TR2022-032

Learning robot motor skills with mixed reality


    •  Rosen, E., Rammohan, S., Jha, D.K., "Learning robot motor skills with mixed reality", International Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions ACM/IEEEE International Conference on Human-Robot Collaboration 2022, March 2022.
      BibTeX TR2022-032 PDF
      • @inproceedings{Rosen2022mar,
      • author = {Rosen, Eric and Rammohan, Sreehari and Jha, Devesh K.},
      • title = {Learning robot motor skills with mixed reality},
      • booktitle = {International Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions ACM/IEEEE International Conference on Human-Robot Collaboration 2022},
      • year = 2022,
      • month = mar,
      • url = {https://www.merl.com/publications/TR2022-032}
      • }
  • MERL Contact:
  • Research Area:

    Robotics

Abstract:

Mixed Reality (MR) has recently shown great success as an intuitive interface for enabling end-users to teach robots. Related works have used MR interfaces to communicate robot intents and beliefs to a co-located human, as well as developed algorithms for taking multi-modal human input and learning complex motor behaviors. Even with these successes, enabling end-users to teach robots complex motor tasks still poses a challenge because end-user communication is highly task dependent and world knowledge is highly varied. We propose a learning framework where end-users teach robots a) motion demonstrations, b) task constraints, c) planning representations, and d) object information, all of which are integrated into a single motor skill learning framework based on Dynamic
Movement Primitives (DMPs). We hypothesize that conveying this world knowledge will be intuitive with an MR interface, and that a sample-efficient motor skill learning framework which incorporates varied modalities of world knowledge will enable robots to effectively solve complex tasks.