Objectives

The overall approach of GeRT to achieve its goal is as follows: (1) start from existing hand-coded programs as examples; (2) learn or adapt the low-level controllers in these programs to handle greater variation in objects (3) automatically create abstract models of action effects from these, which can be reasoned with by a planner; and (4) use a planner to combine these operators to solve a novel instance of a task in a given domain. As stated in the overview this will require advances in several areas. We have broken the project achievements into a number of measureable objectives. Specifically we will develop:

• Representations of object shape. Building on these we will develop methods for shape category recognition and for making detailed correspondences between known and novel shapes of the same category. This will provide a generic geometric scene representation and allow the transfer of semantic (class labels) and functional (grasp regions or points) knowledge from known to novel objects. (WP2)
• Representations of the abstract effects of manipulation actions that allow an AI planner to reason about them. (WP1)
• An AI planner that allows planning of a whole manipulation task, involving several objects, through sequencing abstract manipulative actions. This planner will account for geometric constraints on individual manipulations that are imposed by the task. The abstract plans generated must be of a form that allows low level parts of the control program to be generated on the fly. (WP1)
• Learning and optimisation techniques that take example grasp locations for known objects and adapt them to a novel object from a similar class. We will also acquire grasp locations via reinforcement learning. (WP3)
• Learning methods that learn models of the effects of different finger closing strategies, and adapt example finger closing controllers to new objects in a way that maximises the robustness of the grasping routine with respect to variations in the location of the object and hand. (WP3)
• Learning methods that can learn models of the effects of manipulation actions that are suitable to act as abstract planning operators. (WP1)
• Adaptation of the control architecture for Justin to allow the on the fly generation and execution of plans and controllers for manipulation tasks. (WP4)
• Integration of the elements into a complete system on the Justin platform and evaluation of the overall approach on a set of tasks of gradually increasing complexity. (WP4)