Robots who tidy up your room learning to handle new objects
Researchers in Cornell’s Personal Robotics Lab have developed a new algorithm which enables a robot to rely on its artificial intelligence to look at a group of objects instead recognizing single objects placed in front of its sensors. The algorithm enables the robot to survey its surroundings, identify all the objects, figure out where they belong and hopefully estimate the adequate place where it should move them.
The researchers tested placing dishes, books, clothing and toys on tables and in bookshelves, dish racks, refrigerators and closets. The robot was up to 98 percent successful in identifying and placing objects it had seen before. It was able to place objects it had never seen before, but success rates fell to an average of 80 percent. Ambiguously shaped objects, such as clothing and shoes, were most often misidentified.
The robot uses a Microsoft Kinect 3D camera to survey the room and creates an overall image of its environment by overlapping, or “stitching “, individual images together. Formed image is divided into blocks by the robot’s computer, and the blocks are based on potential displacements detected in discontinuities of color and shape. For each block it computes the probability of a match with each object in its database and chooses the most likely match.
For each recognized object the robot examines the target area to decide on an appropriate and stable placement in a process similar to displacement detection. It divides the recorded 3D image of the target space into small chunks and computes how items fit into that free space by considering the shape of the object it’s placing.
The robot learns how to behave because it is fed with graphic simulations in which placement sites are labeled as good and bad, and it builds a model of what good placement sites have in common, and chooses the spot with the closest fit to that model. Finally the robot creates a graphic simulation of how to move the object to its final location and carries out those movements.
According to Ashutosh Saxena, assistant professor of computer science at Cornell who leads the group that developed the algorithm, the robot relies on the captured image so a bowl could be detected as a globe. Performance could be improved with cameras that provide higher-resolution images, introduction of tactile feedback, and by pre-programming the robot with 3D models of the objects it is going to handle, rather than leaving it to create its own model from what it sees.
As one of their next steps, Saxena and his Personal Robotics team plan to add contextual decisions when objects are placed. It could lead to more advanced tiding of the room because it would allow the robot to place objects that are related in some way. For instance, a computer mouse can be placed anywhere on a table, but its place is usually beside the keyboard.
For more information, you can read the paper published in International Journal of Robotics Research: “Learning to Place New Objects in a Scene” [6.51MB PDF].