LLM-enhanced stereo vision perception and TCP pose generation for embodied AI-enabled robotic welding

Research question

How can stereo images with reflections, weak texture and similar edges be converted into usable pre-weld robot paths?

Method

Stereo depth and open-vision candidates are checked against groove geometry, with an LLM assisting interpretation and rechecking. The pipeline outputs weld centerlines, local groove point clouds and TCP pose sequences for pre-weld path preparation.

Main findings

Across 13 open welding scenes, ROI intersection-over-union reached 0.8743. After common-start alignment, generated and laser-reference paths differed by a mean of 1.100 mm over a 341.61 mm shared segment, evaluating pre-weld perception and trajectory consistency.