Dynamic Pick-and-Place System for a Manipulator on a Quadruped Using Object Detection

The growing demand for versatile and efficient robotic systems in industries such as logistics, manufacturing, and service robotics underscores the importance of advancing mobile manipulation technologies. These systems must navigate and operate within unstructured and dynamic environments, requiring seamless integration of locomotion, manipulation, and perception capabilities. This thesis presents the development of a dynamic pick-and-place system for a manipulator mounted on a quadruped robot. The system employs real-time object detection using a custom-trained model, which leverages synthetic images generated in Blender to enable precise identification and tracking of objects. A key feature is the system's ability to perform real-time detection, sorting of alphabets and numbers, and manipulation of multiple objects, showcasing its capability to dynamically track and handle varying items. The manipulator uses inverse kinematics and closed-loop control algorithms to adjust its movements in real-time, ensuring high precision in complex and changing environments. To verify the system's performance, the sorting results were compared between actual positions and motion capture (mocap) data, while the number of successful real-time object tracking attempts was recorded. These evaluations demonstrated that the system is robust and reliable in dynamic environments, advancing mobile manipulation for automated tasks in logistics, manufacturing, and other applications requiring high adaptability and precision. The combination of advanced computer vision, robust control algorithms, and real-time adaptability positions this system as a significant advancement in the field of mobile manipulation robotics.