Project Summary

In order to work naturally in human environments such as offices and homes, robots of the future will need to be much more flexible and robust in the face of novelty than those of today. In GeRT we will develop new methods to cope with novelty in manipulation tasks.

Humans cope so seamlessly with novel objects that we do not think of grasping a new cup, or screwing the lid off a jar we haven't seen before as challenging. But this kind of everyday novelty in manipulation tasks is hard for a robot. Currently the most advanced robots can perform a task such as making a drink, which involves grasping, pouring, and twisting off a cap from a jar. But the rules for how to pick up every single object must be programmed. So the ability to manipulate fifty different objects means writing fifty different programs. Also when the robot encounters an object it hasn't seen before, it can't grasp the object. Even worse than this is the fact that if one object in a task changes then the program for the whole task may need to be rewritten. This is because how we manipulate an object depends on the task, and the other objects involved. Suppose, for example, that we substituted a mug for a glass in a task that involved pouring liquid from the mug or glass into another object. The pouring position would change, as would the grasping position. Perhaps we would grasp the mug by the handle, and then tip it sideways to pour.

All of this means that if robots are ever going to be useful in natural settings where manipulation is involved that they need ways of generalising on the fly to cope with novel objects, and perhaps novel tasks. Our approach is to take a small set of existing robot programs, for a certain robot manipulation task, such as serving a drink and to give the robot the ability to adapt them to a novel version of the task. These programs constitute a database of prototypes representing that class of task. When confronted with a novel instance of the same task the robot needs to establish appropriate correspondences between objects and actions in the prototypes and their counterparts in the novel scenario. In this way the robot can solve a task that is physically substantially different but similar at an abstract level. To achieve this we will use a variety of techniques from machine perception, machine learning, and artificial intelligence techniques such as automated planning.