Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Arvind
Raghunathan
Stefano
Di Cairano
Siddarth
Jain
William S.
Yerazunis
Radu
Corcodel
Yebin
Wang
Karl
Berntorp
Toshiaki
Koike-Akino
Yuki
Shirai
Mouhacine
Benosman
Abraham P.
Vinod
Avishai
Weiss
Tim K.
Marks
Scott A.
Bortoff
Chiori
Hori
Ye
Wang
Jonathan
Le Roux
Matthew
Brand
Anoop
Cherian
Philip V.
Orlik
Bingnan
Wang
Abraham
Goldsmith
Jianlin
Guo
Sameer
Khurana
Jing
Liu
Hassan
Mansour
Saviz
Mowlavi
Anthony
Vetro
Pedro
Miraldo
James
Queeney
-
Awards
-
AWARD Honorable Mention Award at NeurIPS 23 Instruction Workshop Date: December 15, 2023
Awarded to: Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddharth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka and Diego Romeres
MERL Contacts: Radu Corcodel; Chiori Hori; Siddarth Jain; Devesh K. Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL Researchers received an "Honorable Mention award" at the Workshop on Instruction Tuning and Instruction Following at the NeurIPS 2023 conference in New Orleans. The workshop was on the topic of instruction tuning and Instruction following for Large Language Models (LLMs). MERL researchers presented their work on interactive planning using LLMs for partially observable robotic tasks during the oral presentation session at the workshop.
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
-
AWARD MERL Researchers Win Best Workshop Poster Award at the 2023 IEEE International Conference on Robotics and Automation (ICRA) Date: June 2, 2023
Awarded to: Yuki Shirai, Devesh Jha, Arvind Raghunathan and Dennis Hong
MERL Contacts: Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Optimization, RoboticsBrief- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
The paper presents a technique to manipulate an object using a tool in a closed-loop fashion using vision-based tactile sensors. More information about the workshop and the various speakers can be found here https://sites.google.com/view/icra2023embracingcontacts/home.
- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
See All Awards for Robotics -
-
News & Events
-
NEWS MERL researchers present 9 papers at ACC 2024 Date: July 10, 2024 - July 12, 2024
Where: Toronto, Canada
MERL Contacts: Karl Berntorp; Ankush Chakrabarty; Vedang M. Deshpande; Stefano Di Cairano; Christopher R. Laughman; Arvind Raghunathan; Abraham P. Vinod; Yebin Wang; Avishai Weiss
Research Areas: Artificial Intelligence, Control, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, RoboticsBrief- MERL researchers presented 9 papers at the recently concluded American Control Conference (ACC) 2024 in Toronto, Canada. The papers covered a wide range of topics including data-driven spatial monitoring using heterogenous robots, aircraft approach management near airports, computation fluid dynamics-based motion planning for drones facing winds, trajectory planning for coordinated monitoring using a team of drones and a ground carrier vehicle, ensemble Kalman smoothing-based model predictive control for motion planning for autonomous vehicles, system identification for Lithium-ion batteries, physics-constrained deep Kalman filters for vapor compression systems, switched reference governors for constrained systems, and distributed road-map monitoring using onboard sensors.
As a sponsor of the conference, MERL maintained a booth for open discussions with researchers and students, and hosted a special session to discuss highlights of MERL research and work philosophy.
In addition, Abraham Vinod served as a panelist at the Student Networking Event at the conference. The student networking event provides an opportunity for all interested students to network with professionals working in industry, academia, and national laboratories during a structured event, and encourages their continued participation as the future leaders in the field.
- MERL researchers presented 9 papers at the recently concluded American Control Conference (ACC) 2024 in Toronto, Canada. The papers covered a wide range of topics including data-driven spatial monitoring using heterogenous robots, aircraft approach management near airports, computation fluid dynamics-based motion planning for drones facing winds, trajectory planning for coordinated monitoring using a team of drones and a ground carrier vehicle, ensemble Kalman smoothing-based model predictive control for motion planning for autonomous vehicles, system identification for Lithium-ion batteries, physics-constrained deep Kalman filters for vapor compression systems, switched reference governors for constrained systems, and distributed road-map monitoring using onboard sensors.
-
NEWS MERL at the International Conference on Robotics and Automation (ICRA) 2024 Date: May 13, 2024 - May 17, 2024
Where: Yokohama, Japan
MERL Contacts: Anoop Cherian; Radu Corcodel; Stefano Di Cairano; Chiori Hori; Siddarth Jain; Devesh K. Jha; Jonathan Le Roux; Diego Romeres; William S. Yerazunis
Research Areas: Artificial Intelligence, Machine Learning, Optimization, Robotics, Speech & AudioBrief- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
MERL was a Bronze sponsor of the conference, and exhibited a live robotic demonstration, which attracted a large audience. The demonstration showcased an Autonomous Robotic Assembly technology executed on MELCO's Assista robot arm and was the collaborative effort of the Optimization and Robotics Team together with the Advanced Technology department at Mitsubishi Electric.
MERL researchers from the Optimization and Robotics, Speech & Audio, and Control for Autonomy teams also presented 8 papers and 2 invited talks covering topics on robotic assembly, applications of LLMs to robotics, human robot interaction, safe and robust path planning for autonomous drones, transfer learning, perception and tactile sensing.
- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
See All News & Events for Robotics -
-
Internships
-
OR2103: Human Robot Collaboration in Assembly Tasks
MERL is looking for a self-motivated and qualified candidate to work on human-robot-interaction for manipulation and assembly collaborative scenarios. The ideal candidate is a PhD student and should have experience and records in one or multiple of the following areas. 1) Control, estimation and perception for Robotic manipulation 2) Task and Motion Planning 3) Learning from demonstration algorithms applied to robotic manipulation 4) Machine learning techniques for modeling and control as well as regression and classification problems. 5) Experience in working with robotic systems and familiarity with physics engine simulators like Mujoco, Isaac Gym, PyBullet. The successful candidate will be expected to develop, in collaboration with MERL employees, state of the art algorithms to solve complex manipulation tasks that involve human and robot collaborations. Proficiency in Python and ROS are required. The expectation is that the research will lead to one or more scientific publications. The expected duration s 3-4 months, with a flexible starting date.
-
CA2132: Optimization Algorithms for Motion Planning and Predictive Control
MERL is looking for a highly motivated and qualified individual to work on tailored computational algorithms for optimization-based motion planning and predictive control applications in autonomous systems (vehicles, mobile robots). The ideal candidate should have experience in either one or multiple of the following topics: convex and non-convex optimization, stochastic predictive control (e.g., scenario trees), interaction-aware motion planning, machine learning, learning-based model predictive control, mathematical programs with complementarity constraints (MPCCs), optimal control, and real-time optimization. PhD students in engineering or mathematics, especially with a focus on research related to any of the above topics are encouraged to apply. Publication of relevant results in conference proceedings or journals is expected. Capability of implementing the designs and algorithms in MATLAB/Python is required; coding parts of the algorithms in C/C++ is a plus. The expected duration of the internship is 3 months, and the start date is flexible.
-
CA2213: Mobile robotics: Sensing, Planning, and Control
MERL is seeking a highly motivated intern to collaborate in the development and experimental validation of sensing, planning, and control methods in various robotic testbeds (quadrotors, turtlebots, and mini-cars) at MERL. The ideal candidate is enrolled in a Masters/PhD program in Electrical, Mechanical, Aerospace Engineering, Robotics, Computer Science or related program, with prior experience in some or all of the following: motion planning, control, optimization, learning, computer vision, and their application in mobile robots, including experimental validation. The successful candidate is proficient in ROS2, C/C++, and Python, and at least familiar with MATLAB. The expected duration of the internship is 4-6 months with a flexible start date in the late Fall/Winter 2024.
See All Internships for Robotics -
-
Recent Publications
- "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.BibTeX TR2024-008 PDF
- @article{Shirai2024dec,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Romeres, Diego},
- title = {Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming},
- journal = {Nonlinear Analysis: Hybrid Systems},
- year = 2024,
- month = dec,
- url = {https://www.merl.com/publications/TR2024-008}
- }
, - "Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals", International Conference of the IEEE Engineering in Medicine and Biology Society, July 2024.BibTeX TR2024-103 PDF
- @inproceedings{Bimbraw2024jul3,
- author = {Bimbraw, Keshav and Liu, Jing and Wang, Ye and Koike-Akino, Toshiaki}},
- title = {Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals},
- booktitle = {International Conference of the IEEE Engineering in Medicine and Biology Society},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-103}
- }
, - "Lagrangian Inspired Polynomial Estimator for black-box learning and control of underactuated systems", Learning for Dynamics & Control Conference (L4DC), July 2024.BibTeX TR2024-097 PDF
- @inproceedings{Giacomuzzo2024jul,
- author = {{Giacomuzzo, Giulio and Cescon, Riccardo and Romeres, Diego and Carli, Ruggero and Dalla Libera, Alberto}},
- title = {Lagrangian Inspired Polynomial Estimator for black-box learning and control of underactuated systems},
- booktitle = {Learning for Dynamics \& Control Conference (L4DC)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-097}
- }
, - "Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization", IEEE Transactions on Robotics, DOI: 10.1109/TRO.2024.3422053, pp. 3425-3444, July 2024.BibTeX TR2024-096 PDF Video
- @article{Shirai2024jul,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind}},
- title = {Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization},
- journal = {IEEE Transactions on Robotics},
- year = 2024,
- pages = {3425--3444},
- month = jul,
- doi = {10.1109/TRO.2024.3422053},
- issn = {1941-0468},
- url = {https://www.merl.com/publications/TR2024-096}
- }
, - "Simultaneous Trajectory Optimization and Contact Selection for Contact-rich Manipulation with High-Fidelity Geometry", RSS Workshop on Frontiers of Optimization for Robotics (RSS Workshop FOR), July 2024.BibTeX TR2024-100 PDF
- @inproceedings{Zhang2024jul3,
- author = {Zhang, Mengchao and Jha, Devesh K. and Raghunathan, Arvind and Hauser, Kris}},
- title = {Simultaneous Trajectory Optimization and Contact Selection for Contact-rich Manipulation with High-Fidelity Geometry},
- booktitle = {RSS Workshop on Frontiers of Optimization for Robotics (RSS Workshop FOR)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-100}
- }
, - "Decoupled Trajectory Planning for Monitoring UAVs and UGV Carrier by Reachable Sets", American Control Conference (ACC), July 2024.BibTeX TR2024-092 PDF
- @inproceedings{Kim2024jul,
- author = {Kim, Taewan and Vinod, Abraham P. and Di Cairano, Stefano}},
- title = {Decoupled Trajectory Planning for Monitoring UAVs and UGV Carrier by Reachable Sets},
- booktitle = {American Control Conference (ACC)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-092}
- }
, - "Adaptive Velocity Estimators for Learning Control", International Conference on Control, Decision and Information Technologies (CoDIT), July 2024.BibTeX TR2024-088 PDF
- @inproceedings{Nikovski2024jul,
- author = {{Nikovski, Daniel and Yerazunis, William S.}},
- title = {Adaptive Velocity Estimators for Learning Control},
- booktitle = {International Conference on Control, Decision and Information Technologies (CoDIT)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-088}
- }
, - "Memory-Based Global Iterative Linear Quadratic Control", International Conference on Control, Decision and Information Technologies (CoDIT), July 2024.BibTeX TR2024-089 PDF
- @inproceedings{Nikovski2024jul2,
- author = {{Nikovski, Daniel and Zhong, Junmin and Yerazunis, William S.}},
- title = {Memory-Based Global Iterative Linear Quadratic Control},
- booktitle = {International Conference on Control, Decision and Information Technologies (CoDIT)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-089}
- }
,
- "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.
-
Videos
-
Software & Data Downloads