Robotics
Where hardware, software and machine intelligence come together.
Our research is interdisciplinary and focuses on sensing, planning, reasoning, and control of single and multi-agent systems, including both manipulation and mobile robots. We strive to develop algorithms and methods for factory automation, smart building and transportation applications using machine learning, computer vision, RF/optical sensing, wireless communications, control theory and signal processing. Key research themes include bin picking and object manipulation, sensing and mapping of indoor areas, coordinated control of robot swarms, as well as robot learning and simulation.
Quick Links
-
Researchers
Devesh K.
Jha
Diego
Romeres
Daniel N.
Nikovski
Stefano
Di Cairano
Arvind
Raghunathan
Siddarth
Jain
William S.
Yerazunis
Radu
Corcodel
Yebin
Wang
Toshiaki
Koike-Akino
Yuki
Shirai
Abraham P.
Vinod
Avishai
Weiss
Tim K.
Marks
Chiori
Hori
Scott A.
Bortoff
Jonathan
Le Roux
Ye
Wang
Anoop
Cherian
Matthew
Brand
Philip V.
Orlik
Alexander
Schperberg
Bingnan
Wang
Purnanand
Elango
Abraham
Goldsmith
Jianlin
Guo
Jing
Liu
Hassan
Mansour
Pedro
Miraldo
Saviz
Mowlavi
Anthony
Vetro
-
Awards
-
AWARD University of Padua and MERL team wins the AI Olympics with RealAIGym competition at IROS24 Date: October 17, 2024
Awarded to: Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Dynamical Systems, Machine Learning, RoboticsBrief- The team composed of the control group at the University of Padua and MERL's Optimization and Robotic team ranked 1st out of the 4 finalist teams that arrived to the 2nd AI Olympics with RealAIGym competition at IROS 24, which focused on control of under-actuated robots. The team was composed by Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli and Diego Romeres. The competition was organized by the German Research Center for Artificial Intelligence (DFKI), Technical University of Darmstadt and Chalmers University of Technology.
The competition and award ceremony was hosted by IEEE International Conference on Intelligent Robots and Systems (IROS) on October 17, 2024 in Abu Dhabi, UAE. Diego Romeres presented the team's method, based on a model-based reinforcement learning algorithm called MC-PILCO.
- The team composed of the control group at the University of Padua and MERL's Optimization and Robotic team ranked 1st out of the 4 finalist teams that arrived to the 2nd AI Olympics with RealAIGym competition at IROS 24, which focused on control of under-actuated robots. The team was composed by Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli and Diego Romeres. The competition was organized by the German Research Center for Artificial Intelligence (DFKI), Technical University of Darmstadt and Chalmers University of Technology.
-
AWARD Honorable Mention Award at NeurIPS 23 Instruction Workshop Date: December 15, 2023
Awarded to: Lingfeng Sun, Devesh K. Jha, Chiori Hori, Siddharth Jain, Radu Corcodel, Xinghao Zhu, Masayoshi Tomizuka and Diego Romeres
MERL Contacts: Radu Corcodel; Chiori Hori; Siddarth Jain; Devesh K. Jha; Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- MERL Researchers received an "Honorable Mention award" at the Workshop on Instruction Tuning and Instruction Following at the NeurIPS 2023 conference in New Orleans. The workshop was on the topic of instruction tuning and Instruction following for Large Language Models (LLMs). MERL researchers presented their work on interactive planning using LLMs for partially observable robotic tasks during the oral presentation session at the workshop.
-
AWARD Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym' Date: August 25, 2023
Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
MERL Contact: Diego Romeres
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.
The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
- A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.
See All Awards for Robotics -
-
News & Events
-
EVENT MERL Contributes to ICASSP 2025 Date: Sunday, April 6, 2025 - Friday, April 11, 2025
Location: Hyderabad, India
MERL Contacts: Wael H. Ali; Petros T. Boufounos; Radu Corcodel; François Germain; Chiori Hori; Siddarth Jain; Devesh K. Jha; Toshiaki Koike-Akino; Jonathan Le Roux; Yanting Ma; Hassan Mansour; Yoshiki Masuyama; Joshua Rapp; Diego Romeres; Anthony Vetro; Pu (Perry) Wang; Gordon Wichern
Research Areas: Artificial Intelligence, Communications, Computational Sensing, Electronic and Photonic Devices, Machine Learning, Robotics, Signal Processing, Speech & AudioBrief- MERL has made numerous contributions to both the organization and technical program of ICASSP 2025, which is being held in Hyderabad, India from April 6-11, 2025.
Sponsorship
MERL is proud to be a Silver Patron of the conference and will participate in the student job fair on Thursday, April 10. Please join this session to learn more about employment opportunities at MERL, including openings for research scientists, post-docs, and interns.
MERL is pleased to be the sponsor of two IEEE Awards that will be presented at the conference. We congratulate Prof. Björn Erik Ottersten, the recipient of the 2025 IEEE Fourier Award for Signal Processing, and Prof. Shrikanth Narayanan, the recipient of the 2025 IEEE James L. Flanagan Speech and Audio Processing Award. Both awards will be presented in-person at ICASSP by Anthony Vetro, MERL President & CEO.
Technical Program
MERL is presenting 15 papers in the main conference on a wide range of topics including source separation, sound event detection, sound anomaly detection, speaker diarization, music generation, robot action generation from video, indoor airflow imaging, WiFi sensing, Doppler single-photon Lidar, optical coherence tomography, and radar imaging. Another paper on spatial audio will be presented at the Generative Data Augmentation for Real-World Signal Processing Applications (GenDA) Satellite Workshop.
MERL Researchers Petros Boufounos and Hassan Mansour will present a Tutorial on “Computational Methods in Radar Imaging” in the afternoon of Monday, April 7.
Petros Boufounos will also be giving an industry talk on Thursday April 10 at 12pm, on “A Physics-Informed Approach to Sensing".
About ICASSP
ICASSP is the flagship conference of the IEEE Signal Processing Society, and the world's largest and most comprehensive technical conference focused on the research advances and latest technological development in signal and information processing. The event has been attracting more than 4000 participants each year.
- MERL has made numerous contributions to both the organization and technical program of ICASSP 2025, which is being held in Hyderabad, India from April 6-11, 2025.
-
NEWS Yuki Shirai appointed as an Associate Editor for IEEE Robotics and Automation Letters (RA-L). Date: March 4, 2025
Where: IEEE Robotics and Automation Society (RAS)
MERL Contact: Yuki Shirai
Research Areas: Artificial Intelligence, Optimization, RoboticsBrief- MERL researcher, Yuki Shirai, has been appointed to the editorial board of the IEEE Robotics and Automation Letters (RA-L) as an Associate Editor.
IEEE RA-L publishes peer-reviewed articles in the areas of robotics and automation which can also be presented at the annual flagship conferences of IEEE Robotics and Automation Society (RAS), including IEEE International Conference on Robotics and Automation (ICRA) and International Conference on Intelligent Robots and Systems (IROS).
- MERL researcher, Yuki Shirai, has been appointed to the editorial board of the IEEE Robotics and Automation Letters (RA-L) as an Associate Editor.
See All News & Events for Robotics -
-
Internships
-
OR0147: Internship - Learning Visuotactile Dexterous Manipulation
MERL is looking for a highly motivated individual to work on designing robot motor skills for contact-rich dexterous manipulation such as tool manipulation using visuotactile sensing. Through the internship, we develop closed-loop controller for dexterous manipulation that can stabilize manipulation even under unexpected contact events. A successful internship will result in the submission of results to a peer-reviewed robotics conference papers (e.g., RSS, ICRA, CoRL) or journal in collaboration with MERL researchers.
The expected duration of this internship is around 3 months with a start date in the Summer/Fall of 2025. This internship will be onsite at MERL.
Required Specific Experience
-
Senior PhD students in robotics and engineering related field.
-
Either (1) experience in machine learning for robotic manipulation such as reinforcement, representation, and imitation learning, with an emphasis on real-world deployments, or (2) experience in model-based optimization for contact-rich robotic manipulation such as trajectory optimization, MPC, and estimators.
-
Experience in visuotactile sensors (e.g., GelSight, GelSlim, DIGIT)
-
Familiarity with related frameworks in either (1) Machine Learning (e.g., PyTorch), or (2) model-based optimization (e.g., IPOPT, Gurobi).
-
Experience working with physical hardware systems is required.
-
Prior publication experience in robotics venues such as ICRA, RSS, IROS, and CoRL.
Additional Desired Experience
-
Experience in computer vision.
-
Experience in dexterous manipulation such as bimanual manipulation, tool manipulation, and whole-body manipulation.
-
Familiarity with robotic simulators (e.g., MuJoCo).
-
Experience in scalable software development (e.g., CI, unittests, OOP) in Python or C++.
-
-
OR0127: Internship - Deep Learning for Robotic Manipulation
MERL is looking for a highly motivated and qualified intern to work on deep learning methods for detection and pose estimation of objects using vision and tactile sensing, in manufacturing and assembly environments. This role involves developing, fine-tuning and deploying models on existing hardware. The method will be applied for robotic manipulation where the knowledge of accurate position and orientation of objects within the scene would allow the robot to interact with the objects. The ideal candidate would be a Ph.D. student familiar with the state-of-the-art methods for pose estimation and tracking of objects. The successful candidate will work closely with MERL researchers to develop and implement novel algorithms, conduct experiments, and publish research findings at a top-tier conference. Start date and expected duration of the internship is flexible. Interested candidates are encouraged to apply with their updated CV and list of relevant publications.
Required Specific Experience
- Prior experience in Computer Vision and Robotic Manipulation.
- Experience with ROS and deep learning frameworks such as PyTorch are essential.
- Strong programming skills in Python.
- Experience with simulation tools, such as PyBullet, Issac Lab, or MuJoCo.
-
CA0148: Internship - Motion Planning and Control for Autonomous Articulated Vehicles
MERL is seeking an outstanding intern to collaborate in the development of motion planning and control for autonomous articulated vehicles. The ideal candidate is expected to be working towards a PhD in electrical, mechanical, aerospace engineering, robotics, control or related areas, with a strong emphasis on motion planning and control, possibly with applications to ground vehicles, to have experience in at least some of path/motion planning algorithms (A*, D*, graph-search) and optimization-based control (e.g., model predictive control), to have excellent coding skills in MATLAB/Simulink and a strong publication record. The expected start date is the Spring/Early Summer 2025 and the expected duration is 6-9 months, depending on candidate availability and interests.
Required Specific Experience
- Path/motion planning algorithms (A*, D*, graph-search)
- Nonlinear model predictive control
- Programming in Matlab/Simulink
- Applications to mobile robots or vehicles
Additional Useful Experience
- Nonlinear MPC Design in CasADi
- Code generation tools and dSPACE
- Applications to autonomous vehicles and articulated vehicles
See All Internships for Robotics -
-
Openings
See All Openings at MERL -
Recent Publications
- "Learning global control of underactuated double pendulum with Model-Based Reinforcement Learning", IEEE International Conference on Robotics and Automation (ICRA) - 3rd AI Olympics with RealAIGym Competition, April 2025.BibTeX TR2025-046 PDF
- @inproceedings{Turcato2025apr,
- author = {Turcato, Niccolò and Cali, Marco and Dalla Libera, Alberto and Giacomuzzo, Giulio and Carli, Ruggero and Romeres, Diego},
- title = {{Learning global control of underactuated double pendulum with Model-Based Reinforcement Learning}},
- booktitle = {IEEE International Conference on Robotics and Automation (ICRA) - 3rd AI Olympics with RealAIGym Competition},
- year = 2025,
- month = apr,
- url = {https://www.merl.com/publications/TR2025-046}
- }
, - "Learning Visual Servoing for Nonholonomic Mobile Robots with Uncalibrated Cameras", The 40th ACM/SIGAPP Symposium On Applied Computing, March 2025.BibTeX TR2025-042 PDF
- @inproceedings{Wang2025mar2,
- author = {Wang, Jen-Wei and Nikovski, Daniel N.},
- title = {{Learning Visual Servoing for Nonholonomic Mobile Robots with Uncalibrated Cameras}},
- booktitle = {The 40th ACM/SIGAPP Symposium On Applied Computing},
- year = 2025,
- month = mar,
- url = {https://www.merl.com/publications/TR2025-042}
- }
, - "Interactive Robot Action Replanning using Multimodal LLM Trained from Human Demonstration Videos", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2025.BibTeX TR2025-034 PDF
- @inproceedings{Hori2025mar,
- author = {Hori, Chiori and Kambara, Motonari and Sugiura, Komei and Ota, Kei and Khurana, Sameer and Jain, Siddarth and Corcodel, Radu and Jha, Devesh K. and Romeres, Diego and {Le Roux}, Jonathan},
- title = {{Interactive Robot Action Replanning using Multimodal LLM Trained from Human Demonstration Videos}},
- booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
- year = 2025,
- month = mar,
- url = {https://www.merl.com/publications/TR2025-034}
- }
, - "Generalized Policy Improvement Algorithms with Theoretically Supported Sample Reuse", IEEE Transactions on Automatic Control, DOI: 10.1109/TAC.2024.3454011, Vol. 70, No. 2, pp. 1236-1243, February 2025.BibTeX TR2025-015 PDF
- @article{Queeney2025feb,
- author = {Queeney, James and Paschalidis, Ioannis Ch. and Cassandras, Christos G.},
- title = {{Generalized Policy Improvement Algorithms with Theoretically Supported Sample Reuse}},
- journal = {IEEE Transactions on Automatic Control},
- year = 2025,
- volume = 70,
- number = 2,
- pages = {1236--1243},
- month = feb,
- doi = {10.1109/TAC.2024.3454011},
- url = {https://www.merl.com/publications/TR2025-015}
- }
, - "Invariant Set Planning for Quadrotors: Design, Analysis, Experiments", IEEE Transactions on Control Systems Technology, DOI: 10.1109/TCST.2024.3492813, Vol. 33, No. 2, pp. 449-462, January 2025.BibTeX TR2025-010 PDF
- @article{Greiff2025jan,
- author = {Greiff, Marcus and Sinhmar, Himani and Weiss, Avishai and Berntorp, Karl and {Di Cairano}, Stefano},
- title = {{Invariant Set Planning for Quadrotors: Design, Analysis, Experiments}},
- journal = {IEEE Transactions on Control Systems Technology},
- year = 2025,
- volume = 33,
- number = 2,
- pages = {449--462},
- month = jan,
- doi = {10.1109/TCST.2024.3492813},
- issn = {1063-6536},
- url = {https://www.merl.com/publications/TR2025-010}
- }
, - "Continuous-Time Successive Convexification for Passively-Safe Six-Degree-of-Freedom Powered-Descent Guidance", AIAA SciTech, DOI: 10.2514/6.2025-1894, January 2025, pp. 1-13.BibTeX TR2025-008 PDF
- @inproceedings{Elango2025jan,
- author = {Elango, Purnanand and Vinod, Abraham P. and {Di Cairano}, Stefano and Weiss, Avishai},
- title = {{Continuous-Time Successive Convexification for Passively-Safe Six-Degree-of-Freedom Powered-Descent Guidance}},
- booktitle = {AIAA SCITECH 2025 Forum},
- year = 2025,
- pages = {1--13},
- month = jan,
- publisher = {AIAA},
- doi = {10.2514/6.2025-1894},
- url = {https://www.merl.com/publications/TR2025-008}
- }
, - "Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming", Nonlinear Analysis: Hybrid Systems, DOI: 10.1016/j.nahs.2024.101466, Vol. 52, December 2024.BibTeX TR2024-008 PDF
- @article{Shirai2024dec,
- author = {Shirai, Yuki and Jha, Devesh K. and Raghunathan, Arvind and Romeres, Diego},
- title = {{Chance-Constrained Optimization for Contact-rich Systems using Mixed Integer Programming}},
- journal = {Nonlinear Analysis: Hybrid Systems},
- year = 2024,
- volume = 52,
- month = dec,
- doi = {10.1016/j.nahs.2024.101466},
- issn = {1751-570X},
- url = {https://www.merl.com/publications/TR2024-008}
- }
, - "Learning Time-Optimal Control of Gantry Cranes", International Conference on Machine Learning and Applications (ICMLA), December 2024.BibTeX TR2024-181 PDF
- @inproceedings{Zhong2024dec,
- author = {Zhong, Junmin and Nikovski, Daniel N. and Yerazunis, William S. and Ando, Taishi},
- title = {{Learning Time-Optimal Control of Gantry Cranes}},
- booktitle = {International Conference on Machine Learning and Applications (ICMLA)},
- year = 2024,
- month = dec,
- url = {https://www.merl.com/publications/TR2024-181}
- }
,
- "Learning global control of underactuated double pendulum with Model-Based Reinforcement Learning", IEEE International Conference on Robotics and Automation (ICRA) - 3rd AI Olympics with RealAIGym Competition, April 2025.
-
Videos
-
Software & Data Downloads
-
Lagrangian Inspired Polynomial for Robot Inverse Dynamics -
Monte Carlo Probabilistic Inference for Learning COntrol -
Python-based Robotic Control & Optimization Package -
Context-Aware Zero Shot Learning -
Online Feature Extractor Network -
Quasi-Newton Trust Region Policy Optimization -
Circular Maze Environment
-