TR2020-062

Local Policy Optimization for Trajectory-Centric Reinforcement Learning

    •  Jha, D.K., Kolaric, P., Raghunathan, A., Lewis, F., Benosman, M., Romeres, D., Nikovski, D.N., "Local Policy Optimization for Trajectory-Centric Reinforcement Learning", IEEE International Conference on Robotics and Automation (ICRA), Ayanna Howard, Eds., May 2020, pp. 5094-5100.
      BibTeX TR2020-062 PDF
      • @inproceedings{Jha2020may,
      • author = {Jha, Devesh K. and Kolaric, Patrik and Raghunathan, Arvind and Lewis, Frank and Benosman, Mouhacine and Romeres, Diego and Nikovski, Daniel N.},
      • title = {Local Policy Optimization for Trajectory-Centric Reinforcement Learning},
      • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
      • year = 2020,
      • editor = {Ayanna Howard},
      • pages = {5094--5100},
      • month = may,
      • publisher = {IEEE},
      • isbn = {978-1-7281-7395-5},
      • url = {https://www.merl.com/publications/TR2020-062}
      • }
  • MERL Contacts:
  • Research Areas:

    Control, Machine Learning, Robotics

Abstract:

The goal of this paper is to present a method for simultaneous trajectory and local stabilizing policy optimization to generate local policies for trajectory-centric model-based reinforcement learning (MBRL). This is motivated by the fact that global policy optimization for non-linear systems could be a very challenging problem both algorithmically and numerically. However, a lot of robotic manipulation tasks are trajectorycentric, and thus do not require a global model or policy. Due to inaccuracies in the learned model estimates, an openloop trajectory optimization process mostly results in very poor performance when used on the real system. Motivated by these problems, we try to formulate the problem of trajectory optimization and local policy synthesis as a single optimization problem. It is then solved simultaneously as an instance of nonlinear programming. We provide some results for analysis as well as achieved performance of the proposed technique under some simplifying assumptions.

 

  • Related Publications

  •  Kolaric, P., Jha, D.K., Raghunathan, A., Lewis, F., Benosman, M., Romeres, D., Nikovski, D.N., "Local Policy Optimization for Trajectory-Centric Reinforcement Learning", arXiv, January 2020.
    BibTeX arXiv
    • @article{Kolaric2020jan,
    • author = {Kolaric, Patrik and Jha, Devesh K. and Raghunathan, Arvind and Lewis, Frank and Benosman, Mouhacine and Romeres, Diego and Nikovski, Daniel N.},
    • title = {Local Policy Optimization for Trajectory-Centric Reinforcement Learning},
    • journal = {arXiv},
    • year = 2020,
    • month = jan,
    • url = {https://arxiv.org/abs/2001.08092}
    • }
  •  Jha, D.K., Kolaric, P., Romeres, D., Raghunathan, A., Benosman, M., Nikovski, D.N., "Robust Optimization for Trajectory-Centric Model-based Reinforcement Learning", NeurIPS Workshop on Safety and Robustness in Decision Making, December 2019.
    BibTeX TR2019-156 PDF
    • @inproceedings{Jha2019dec2,
    • author = {Jha, Devesh K. and Kolaric, Patrik and Romeres, Diego and Raghunathan, Arvind and Benosman, Mouhacine and Nikovski, Daniel N.},
    • title = {Robust Optimization for Trajectory-Centric Model-based Reinforcement Learning},
    • booktitle = {NeurIPS Workshop on Safety and Robustness in Decision Making},
    • year = 2019,
    • month = dec,
    • url = {https://www.merl.com/publications/TR2019-156}
    • }