Researchers at Carnegie Mellon University teach robots to pick up transparent or reflective objects

2024-01-11

Researchers at Carnegie Mellon University have been able to teach robots to pick up transparent or reflective objects. This has always been a challenge faced by robots in the past, and researchers have solved this problem by teaching robots to infer shapes from color images. For a long time, picking up transparent and reflective objects has been hindering the development of robots, but new systems are expected to alleviate this problem.

The cool thing about the new technology developed by the team is that it does not require complex sensors, extensive training, or human guidance. On the contrary, it mainly relies on color cameras embedded in the robot arm. Researcher David Held stated that previously, depth cameras shone infrared light on objects to determine their shape, which was an effective technique for opaque objects. The challenge for transparent and reflective objects is that light either passes directly through or scatters from the surface, making it difficult for depth cameras to calculate accurate shapes.

However, color cameras can see transparent and reflective objects, as well as opaque objects. Researchers have developed a color camera system that can recognize shapes based on color. Using this technology, researchers can train the system to mimic depth systems and implicitly infer shapes to grasp objects. The team pairs depth camera images for opaque objects with color images of the same object. After training, the color camera system is applied to transparent and shiny objects. Based on these images, combined with the information provided by depth cameras, the system can successfully capture challenging objects.

Researchers have found that sometimes the arm makes mistakes, but it performs better than any other system in grasping transparent or reflective objects. The system is still more effective than transparent or reflective objects when grasping opaque objects. The system can also grab cluttered and stacked objects.

Mitsubishi Electric's development of scene perception interaction technology can provide driver