Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Arvind
Raghunathan
Stefano
Di Cairano
Siddarth
Jain
William S.
Yerazunis
Radu
Corcodel
Yebin
Wang
Karl
Berntorp
Toshiaki
Koike-Akino
Yuki
Shirai
Mouhacine
Benosman
Abraham P.
Vinod
Avishai
Weiss
Tim K.
Marks
Scott A.
Bortoff
Chiori
Hori
Ye
Wang
Jonathan
Le Roux
Matthew
Brand
Anoop
Cherian
Philip V.
Orlik
Bingnan
Wang
Abraham
Goldsmith
Jianlin
Guo
Sameer
Khurana
Jing
Liu
Hassan
Mansour
Saviz
Mowlavi
Anthony
Vetro
Pedro
Miraldo
James
Queeney
-
Awards
-
AWARD Honorable Mention Award at NeurIPS 23 Instruction Workshop Date: December 15, 2023
Awarded to: Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddharth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka and Diego Romeres
MERL Contacts: Radu Corcodel; Chiori Hori; Siddarth Jain; Devesh K. Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL Researchers received an "Honorable Mention award" at the Workshop on Instruction Tuning and Instruction Following at the NeurIPS 2023 conference in New Orleans. The workshop was on the topic of instruction tuning and Instruction following for Large Language Models (LLMs). MERL researchers presented their work on interactive planning using LLMs for partially observable robotic tasks during the oral presentation session at the workshop.
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
-
AWARD MERL Researchers Win Best Workshop Poster Award at the 2023 IEEE International Conference on Robotics and Automation (ICRA) Date: June 2, 2023
Awarded to: Yuki Shirai, Devesh Jha, Arvind Raghunathan and Dennis Hong
MERL Contacts: Devesh K. Jha; Arvind Raghunathan
Research Areas: Artificial Intelligence, Optimization, RoboticsBrief- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
The paper presents a technique to manipulate an object using a tool in a closed-loop fashion using vision-based tactile sensors. More information about the workshop and the various speakers can be found here https://sites.google.com/view/icra2023embracingcontacts/home.
- MERL's paper titled: "Closed-Loop Tactile Controller for Tool Manipulation" Won the Best Poster Award in the workshop on "Embracing contacts : Making robots physically interact with our world". First author and MERL intern, Yuki Shirai, was presented with the award at a ceremony held at ICRA in London. MERL researchers Devesh Jha, Principal Research Scientist, and Arvind Raghunathan, Senior Principal Research Scientist and Senior Team Leader as well as Prof. Dennis Hong of University of California, Los Angeles are also coauthors.
See All Awards for Robotics -
-
News & Events
-
NEWS MERL researchers present 9 papers at ACC 2024 Date: July 10, 2024 - July 12, 2024
Where: Toronto, Canada
MERL Contacts: Karl Berntorp; Ankush Chakrabarty; Vedang M. Deshpande; Stefano Di Cairano; Christopher R. Laughman; Arvind Raghunathan; Abraham P. Vinod; Yebin Wang; Avishai Weiss
Research Areas: Artificial Intelligence, Control, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, RoboticsBrief- MERL researchers presented 9 papers at the recently concluded American Control Conference (ACC) 2024 in Toronto, Canada. The papers covered a wide range of topics including data-driven spatial monitoring using heterogenous robots, aircraft approach management near airports, computation fluid dynamics-based motion planning for drones facing winds, trajectory planning for coordinated monitoring using a team of drones and a ground carrier vehicle, ensemble Kalman smoothing-based model predictive control for motion planning for autonomous vehicles, system identification for Lithium-ion batteries, physics-constrained deep Kalman filters for vapor compression systems, switched reference governors for constrained systems, and distributed road-map monitoring using onboard sensors.
As a sponsor of the conference, MERL maintained a booth for open discussions with researchers and students, and hosted a special session to discuss highlights of MERL research and work philosophy.
In addition, Abraham Vinod served as a panelist at the Student Networking Event at the conference. The student networking event provides an opportunity for all interested students to network with professionals working in industry, academia, and national laboratories during a structured event, and encourages their continued participation as the future leaders in the field.
- MERL researchers presented 9 papers at the recently concluded American Control Conference (ACC) 2024 in Toronto, Canada. The papers covered a wide range of topics including data-driven spatial monitoring using heterogenous robots, aircraft approach management near airports, computation fluid dynamics-based motion planning for drones facing winds, trajectory planning for coordinated monitoring using a team of drones and a ground carrier vehicle, ensemble Kalman smoothing-based model predictive control for motion planning for autonomous vehicles, system identification for Lithium-ion batteries, physics-constrained deep Kalman filters for vapor compression systems, switched reference governors for constrained systems, and distributed road-map monitoring using onboard sensors.
-
NEWS MERL at the International Conference on Robotics and Automation (ICRA) 2024 Date: May 13, 2024 - May 17, 2024
Where: Yokohama, Japan
MERL Contacts: Anoop Cherian; Radu Corcodel; Stefano Di Cairano; Chiori Hori; Siddarth Jain; Devesh K. Jha; Jonathan Le Roux; Diego Romeres; William S. Yerazunis
Research Areas: Artificial Intelligence, Machine Learning, Optimization, Robotics, Speech & AudioBrief- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
MERL was a Bronze sponsor of the conference, and exhibited a live robotic demonstration, which attracted a large audience. The demonstration showcased an Autonomous Robotic Assembly technology executed on MELCO's Assista robot arm and was the collaborative effort of the Optimization and Robotics Team together with the Advanced Technology department at Mitsubishi Electric.
MERL researchers from the Optimization and Robotics, Speech & Audio, and Control for Autonomy teams also presented 8 papers and 2 invited talks covering topics on robotic assembly, applications of LLMs to robotics, human robot interaction, safe and robust path planning for autonomous drones, transfer learning, perception and tactile sensing.
- MERL made significant contributions to both the organization and the technical program of the International Conference on Robotics and Automation (ICRA) 2024, which was held in Yokohama, Japan from May 13th to May 17th.
See All News & Events for Robotics -
-
Internships
-
OR2196: Visuo-tactile Learning for Dexterous Manipulation
MERL is looking for a highly motivated individual to work on robotic manipulation using visuo-tactile learning. The research will develop robot motor skills for complex, dexterous manipulation using vision and tactile perception. The ideal candidate should have experience in either one or multiple of the following topics: manipulation, tactile sensing, Reinforcement Learning, sim-to-real techniques for manipulation, and grasping. Senior PhD students in robotics and engineering with a focus on contact-rich manipulation are encouraged to apply. Prior experience working with physical robotic systems (and vision and tactile sensors) is required as results need to be implemented on a physical hardware. Good coding skills in Python ML libraries like PyTorch etc. is required. A successful internship will result in submission of results to a peer-reviewed robotics journal in collaboration with MERL researchers. The expected duration of internship is 4-5 months with start date in Aug/Sept 2024. This internship is preferred to be onsite at MERL.
-
CA2213: Mobile robotics: Sensing, Planning, and Control
MERL is seeking a highly motivated intern to collaborate in the development and experimental validation of sensing, planning, and control methods in various robotic testbeds (quadrotors, turtlebots, and mini-cars) at MERL. The ideal candidate is enrolled in a Masters/PhD program in Electrical, Mechanical, Aerospace Engineering, Robotics, Computer Science or related program, with prior experience in some or all of the following: motion planning, control, optimization, learning, computer vision, and their application in mobile robots, including experimental validation. The successful candidate is proficient in ROS2, C/C++, and Python, and at least familiar with MATLAB. The expected duration of the internship is 4-6 months with a flexible start date in the late Fall/Winter 2024.
-
CA2131: Collaborative Legged Robots
MERL is seeking a highly motivated and qualified intern to collaborate with the Control for Autonomy team in research on control and planning algorithms for legged robots for support activities of and collaboration with humans. The ideal candidate is expected to be working towards a PhD with strong emphasis in robotics control and planning and to have interest and background in as many as possible of: motion planning algorithms, control for legged robot locomotions, legged robots, perception and sensing with multiple sensors, SLAM, vision-based control. Good programming skills in Python or C/C++ are required. The expected start of of the internship is flexible, with duration of 3--6 months.
See All Internships for Robotics -
-
Recent Publications
- , "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.BibTeX TR2024-008 PDF
- @article{Shirai2024dec,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Romeres, Diego},
- title = {Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming},
- journal = {Nonlinear Analysis: Hybrid Systems},
- year = 2024,
- month = dec,
- url = {https://www.merl.com/publications/TR2024-008}
- }
- , "Real-time Mixed-Integer Quadratic Programming for Vehicle Decision Making and Motion Planning", IEEE Transactions on Control Systems Technology, September 2024.BibTeX TR2024-123 PDF
- @article{Quirynen2024sep,
- author = {Quirynen, Rien and Safaoui, Sleiman and Di Cairano, Stefano}},
- title = {Real-time Mixed-Integer Quadratic Programming for Vehicle Decision Making and Motion Planning},
- journal = {IEEE Transactions on Control Systems Technology},
- year = 2024,
- month = sep,
- url = {https://www.merl.com/publications/TR2024-123}
- }
- , "Reinforcement Learning for Athletic Intelligence: Lessons from the 1st “AI Olympics with RealAIGym” Competition", Internation Joint Conference on Artificial Intelligence (IJCAI), August 2024.BibTeX TR2024-108 PDF
- @inproceedings{Wiebe2024aug,
- author = {Wiebe, Felix and Turcato, Niccolò and Dalla Libera, Alberto and Zhang, Chi and Vincent, Theo and Vyas, Shubham and Giacomuzzo, Giulio and Carli, Ruggero and Romeres, Diego and Sathuluri, Akhil and Zimmermann, Markus and Belousov, Boris and Peters, Jan and Kirchner, Frank and Kumar, Shivesh}},
- title = {Reinforcement Learning for Athletic Intelligence: Lessons from the 1st “AI Olympics with RealAIGym” Competition},
- booktitle = {Internation Joint Conference on Artificial Intelligence (IJCAI)},
- year = 2024,
- month = aug,
- url = {https://www.merl.com/publications/TR2024-108}
- }
- , "Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals", International Conference of the IEEE Engineering in Medicine and Biology Society, July 2024.BibTeX TR2024-103 PDF
- @inproceedings{Bimbraw2024jul3,
- author = {Bimbraw, Keshav and Liu, Jing and Wang, Ye and Koike-Akino, Toshiaki}},
- title = {Random Channel Ablation for Robust Hand Gesture Classification with Multimodal Biosignals},
- booktitle = {International Conference of the IEEE Engineering in Medicine and Biology Society},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-103}
- }
- , "Lagrangian Inspired Polynomial Estimator for black-box learning and control of underactuated systems", Learning for Dynamics & Control Conference (L4DC), July 2024.BibTeX TR2024-097 PDF
- @inproceedings{Giacomuzzo2024jul,
- author = {{Giacomuzzo, Giulio and Cescon, Riccardo and Romeres, Diego and Carli, Ruggero and Dalla Libera, Alberto}},
- title = {Lagrangian Inspired Polynomial Estimator for black-box learning and control of underactuated systems},
- booktitle = {Learning for Dynamics \& Control Conference (L4DC)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-097}
- }
- , "Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization", IEEE Transactions on Robotics, DOI: 10.1109/TRO.2024.3422053, pp. 3425-3444, July 2024.BibTeX TR2024-096 PDF Video
- @article{Shirai2024jul,
- author = {{Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind}},
- title = {Robust Pivoting Manipulation using Contact Implicit Bilevel Optimization},
- journal = {IEEE Transactions on Robotics},
- year = 2024,
- pages = {3425--3444},
- month = jul,
- doi = {10.1109/TRO.2024.3422053},
- issn = {1941-0468},
- url = {https://www.merl.com/publications/TR2024-096}
- }
- , "Simultaneous Trajectory Optimization and Contact Selection for Contact-rich Manipulation with High-Fidelity Geometry", RSS Workshop on Frontiers of Optimization for Robotics (RSS Workshop FOR), July 2024.BibTeX TR2024-100 PDF
- @inproceedings{Zhang2024jul3,
- author = {Zhang, Mengchao and Jha, Devesh K. and Raghunathan, Arvind and Hauser, Kris}},
- title = {Simultaneous Trajectory Optimization and Contact Selection for Contact-rich Manipulation with High-Fidelity Geometry},
- booktitle = {RSS Workshop on Frontiers of Optimization for Robotics (RSS Workshop FOR)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-100}
- }
- , "Decoupled Trajectory Planning for Monitoring UAVs and UGV Carrier by Reachable Sets", American Control Conference (ACC), July 2024.BibTeX TR2024-092 PDF
- @inproceedings{Kim2024jul,
- author = {Kim, Taewan and Vinod, Abraham P. and Di Cairano, Stefano}},
- title = {Decoupled Trajectory Planning for Monitoring UAVs and UGV Carrier by Reachable Sets},
- booktitle = {American Control Conference (ACC)},
- year = 2024,
- month = jul,
- url = {https://www.merl.com/publications/TR2024-092}
- }
- , "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, December 2024.
-
Videos
-
Software & Data Downloads
