Machine Learning

Data-driven approaches to design intelligent algorithms.

MERL has a long history of research activity in machine learning, including the development of various boosting algorithms and contributing to the theory and practice of highly scalable collaborative filtering. Our recent work has focused on deep learning and reinforcement learning, with application to a wide range of applications including automotive, robotics, factory automation, transportation, as well as building and home systems.

Quick Links
Researchers
Awards
- AWARD MERL Wins Awards at NeurIPS LLM Privacy Challenge
  Date: December 15, 2024
  Awarded to: Jing Liu, Ye Wang, Toshiaki Koike-Akino, Tsunato Nakai, Kento Oonishi, Takuya Higashi
  MERL Contacts: Toshiaki Koike-Akino; Jing Liu; Ye Wang
  Research Areas: Artificial Intelligence, Machine Learning, Information Security
  Brief
  - The Mitsubishi Electric Privacy Enhancing Technologies (MEL-PETs) team, consisting of a collaboration of MERL and Mitsubishi Electric researchers, won awards at the NeurIPS 2024 Large Language Model (LLM) Privacy Challenge. In the Blue Team track of the challenge, we won the 3rd Place Award, and in the Red Team track, we won the Special Award for Practical Attack.
- AWARD University of Padua and MERL team wins the AI Olympics with RealAIGym competition at IROS24
  Date: October 17, 2024
  Awarded to: Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
  MERL Contact: Diego Romeres
  Research Areas: Artificial Intelligence, Dynamical Systems, Machine Learning, Robotics
  Brief
  - The team composed of the control group at the University of Padua and MERL's Optimization and Robotic team ranked 1st out of the 4 finalist teams that arrived to the 2nd AI Olympics with RealAIGym competition at IROS 24, which focused on control of under-actuated robots. The team was composed by Niccolò Turcato, Alberto Dalla Libera, Giulio Giacomuzzo, Ruggero Carli and Diego Romeres. The competition was organized by the German Research Center for Artificial Intelligence (DFKI), Technical University of Darmstadt and Chalmers University of Technology.
    
    The competition and award ceremony was hosted by IEEE International Conference on Intelligent Robots and Systems (IROS) on October 17, 2024 in Abu Dhabi, UAE. Diego Romeres presented the team's method, based on a model-based reinforcement learning algorithm called MC-PILCO.
- AWARD MERL team wins the Listener Acoustic Personalisation (LAP) 2024 Challenge
  Date: August 29, 2024
  Awarded to: Yoshiki Masuyama, Gordon Wichern, Francois G. Germain, Christopher Ick, and Jonathan Le Roux
  MERL Contacts: François Germain; Jonathan Le Roux; Gordon Wichern; Yoshiki Masuyama
  Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
  Brief
  - MERL's Speech & Audio team ranked 1st out of 7 teams in Task 2 of the 1st SONICOM Listener Acoustic Personalisation (LAP) Challenge, which focused on "Spatial upsampling for obtaining a high-spatial-resolution HRTF from a very low number of directions". The team was led by Yoshiki Masuyama, and also included Gordon Wichern, Francois Germain, MERL intern Christopher Ick, and Jonathan Le Roux.
    
    The LAP Challenge workshop and award ceremony was hosted by the 32nd European Signal Processing Conference (EUSIPCO 24) on August 29, 2024 in Lyon, France. Yoshiki Masuyama presented the team's method, "Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization", and received the award from Prof. Michele Geronazzo (University of Padova, IT, and Imperial College London, UK), Chair of the Challenge's Organizing Committee.
    
    The LAP challenge aims to explore challenges in the field of personalized spatial audio, with the first edition focusing on the spatial upsampling and interpolation of head-related transfer functions (HRTFs). HRTFs with dense spatial grids are required for immersive audio experiences, but their recording is time-consuming. Although HRTF spatial upsampling has recently shown remarkable progress with approaches involving neural fields, HRTF estimation accuracy remains limited when upsampling from only a few measured directions, e.g., 3 or 5 measurements. The MERL team tackled this problem by proposing a retrieval-augmented neural field (RANF). RANF retrieves a subject whose HRTFs are close to those of the target subject at the measured directions from a library of subjects. The HRTF of the retrieved subject at the target direction is fed into the neural field in addition to the desired sound source direction. The team also developed a neural network architecture that can handle an arbitrary number of retrieved subjects, inspired by a multi-channel processing technique called transform-average-concatenate.
See All Awards for Machine Learning
News & Events
- TALK [MERL Seminar Series 2025] Andy Zou presents talk titled Red Teaming AI Agents in-the-wild: Revealing Deployment Vulnerabilities
  Date & Time: Wednesday, March 26, 2025; 1:00 PM
  Speaker: Andy Zou, CMU & Gray Swan AI
  MERL Host: Ye Wang
  Research Areas: Artificial Intelligence, Machine Learning, Information Security
  Abstract
  - This presentation demonstrates how red teaming uncovers critical vulnerabilities in AI agents that challenge assumptions about safe deployment. The talk discusses the risks of integrating AI into real-world applications and recommends practical safeguards to enhance resilience and ensure dependable deployment in high-risk settings.
- NEWS MERL Papers and Workshops at AAAI 2025
  Date: February 25, 2025 - March 4, 2025
  Where: The Association for the Advancement of Artificial Intelligence (AAAI)
  MERL Contacts: Ankush Chakrabarty; Toshiaki Koike-Akino; Jing Liu; Kuan-Chuan Peng; Diego Romeres; Ye Wang
  Research Areas: Artificial Intelligence, Machine Learning, Optimization
  Brief
  - MERL researchers presented 2 conference papers, 2 workshop papers, and co-organized 1 workshop at the AAAI 2025 conference, which was held in Philadelphia from Feb. 25 to Mar. 4, 2025. AAAI is one of the most prestigious and competitive international conferences in artificial intelligence (AI). Details of MERL contributions are provided below.
    
    - AAAI Papers in Main Tracks:
    
    1. "Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage" by M.R.U. Rashid, J. Liu, T. Koike-Akino, Y. Wang, and S. Mehnaz. [Oral Presentation]
    
    This work proposes a novel unlearning-based model poisoning method that amplifies privacy breaches during fine-tuning. Extensive empirical studies show the proposed method’s efficacy on both membership inference and data extraction attacks. The attack is stealthy enough to bypass detection based defenses, and differential privacy cannot effectively defend against the attacks without significantly impacting model utility.
    
    Paper: https://www.merl.com/publications/TR2025-017
    
    2. "User-Preference Meets Pareto-Optimality: Multi-Objective Bayesian Optimization with Local Gradient Search" by J.H.S. Ip, A. Chakrabarty, A. Mesbah, and D. Romeres. [Poster Presentation]
    
    This paper introduces a sample-efficient multi-objective Bayesian optimization method that integrates user preferences with gradient-based search to find near-Pareto optimal solutions. The proposed method achieves high utility and reduces distance to Pareto-front solutions across both synthetic and real-world problems, underscoring the importance of minimizing gradient uncertainty during gradient-based optimization. Additionally, the study introduces a novel utility function that respects Pareto dominance and effectively captures diverse user preferences.
    
    Paper: https://www.merl.com/publications/TR2025-018
    
    - AAAI Workshop Papers:
    
    1. "Quantum Diffusion Models for Few-Shot Learning" by R. Wang, Y. Wang, J. Liu, and T. Koike-Akino.
    
    This work presents the quantum diffusion model (QDM) as an approach to overcome the challenges of quantum few-shot learning (QFSL). It introduces three novel algorithms developed from complementary data-driven and algorithmic perspectives to enhance the performance of QFSL tasks. The extensive experiments demonstrate that these algorithms achieve significant performance gains over traditional baselines, underscoring the potential of QDM to advance QFSL by effectively leveraging quantum noise modeling and label guidance.
    
    Paper: https://www.merl.com/publications/TR2025-025
    
    2. "Quantum Implicit Neural Compression", by T. Fujihashi and T., Koike-Akino.
    
    This work introduces a quantum counterpart of implicit neural representation (quINR) which leverages the exponentially rich expressivity of quantum neural networks to improve the classical INR-based signal compression methods. Evaluations using some benchmark datasets show that the proposed quINR-based compression could improve rate-distortion performance in image compression compared with traditional codecs and classic INR-based coding methods.
    
    Paper: https://www.merl.com/publications/TR2025-024
    
    - AAAI Workshops Contributed by MERL:
    
    1. "Scalable and Efficient Artificial Intelligence Systems (SEAS)"
    
    K.-C. Peng co-organized this workshop, which offers a timely forum for experts to share their perspectives in designing and developing robust computer vision (CV), machine learning (ML), and artificial intelligence (AI) algorithms, and translating them into real-world solutions.
    
    Workshop link: https://seasworkshop.github.io/aaai25/index.html
    
    2. "Quantum Computing and Artificial Intelligence"
    
    T. Koike-Akino served a session chair of Quantum Neural Network in this workshop, which focuses on seeking contributions encompassing theoretical and applied advances in quantum AI, quantum computing (QC) to enhance classical AI, and classical AI to tackle various aspects of QC.
    
    Workshop link: https://sites.google.com/view/qcai2025/
See All News & Events for Machine Learning
Research Highlights
Internships
- MS0098: Internship - Control and Estimation for Large-Scale Thermofluid Systems
  
  MERL is seeking a motivated graduate student to research methods for state and parameter estimation and optimization of large-scale systems for process applications. Representative applications include large vapor-compression cycles and other multiphysical systems for energy conversion that couple thermodynamic, fluid, and electrical domains. The ideal candidate would have a solid background in control and estimation, numerical methods, and optimization; strong programming skills and experience with Julia/Python/Matlab are also expected. Knowledge of the fundamental physics of thermofluid flows (e.g., thermodynamics, heat transfer, and fluid mechanics), nonlinear dynamics, or equation-oriented languages (Modelica, gPROMS) is a plus. The expected duration of this internship is 3 months.
- ST0096: Internship - Multimodal Tracking and Imaging
  
  MERL is seeking a motivated intern to assist in developing hardware and algorithms for multimodal imaging applications. The project involves integration of radar, camera, and depth sensors in a variety of sensing scenarios. The ideal candidate should have experience with FMCW radar and/or depth sensing, and be fluent in Python and scripting methods. Familiarity with optical tracking of humans and experience with hardware prototyping is desired. Good knowledge of computational imaging and/or radar imaging methods is a plus.
  Required Specific Experience
  - Experience with Python and Python Deep Learning Frameworks.
  - Experience with FMCW radar and/or Depth Sensors.
- CI0082: Internship - Quantum AI
  
  MERL is excited to announce an internship opportunity in the field of Quantum Machine Learning (QML) and Quantum AI (QAI). We are seeking a highly motivated and talented individual to join our research team. This is an exciting opportunity to make a real impact in the field of quantum computing and AI, with the aim of publishing at leading research venues.
  Responsibilities:
  - Conduct cutting-edge research in quantum machine learning.
  - Collaborate with a team of experts in quantum computing, deep learning, and signal processing.
  - Develop and implement algorithms using PyTorch and PennyLane.
  - Publish research results at leading research venues.
  Qualifications:
  - Currently pursuing a PhD or a post-graduate researcher in a relevant field.
  - Strong background and solid publication records in quantum computing, deep learning, and signal processing.
  - Proficient programming skills in PyTorch and PennyLane are highly desirable.
  What We Offer:
  - An opportunity to work on groundbreaking research in a leading research lab.
  - Collaboration with a team of experienced researchers.
  - A stimulating and supportive work environment.
  If you are passionate about quantum machine learning and meet the above qualifications, we encourage you to apply. Please submit your resume and a brief cover letter detailing your research experience and interests. Join us at MERL and contribute to the future of quantum machine learning!
See All Internships for Machine Learning
Openings
See All Openings at MERL
Recent Publications
- Chakrabarty, A., Vanfretti, L., Wang, Y., Mineyuki, T., Zhan, S., Tang, W.-T., Paulson, J.A., Deshpande, V.M., Bortoff, S.A., Laughman, C.R., "Time-Series Generative Networks for Synthesizing Realistic Scenarios in Occupant-Centric Building Simulation", Building Simulation, April 2025.
  BibTeX TR2025-043 PDF
  - @article{Chakrabarty2025apr,
  - author = {Chakrabarty, Ankush and Vanfretti, Luigi and Wang, Ye and Mineyuki, Takuma and Zhan, Sicheng and Tang, Wei-Ting and Paulson, Joel A. and Deshpande, Vedang M. and Bortoff, Scott A. and Laughman, Christopher R.},
  - title = {{Time-Series Generative Networks for Synthesizing Realistic Scenarios in Occupant-Centric Building Simulation}},
  - journal = {Building Simulation},
  - year = 2025,
  - month = apr,
  - url = {https://www.merl.com/publications/TR2025-043}
  - }
- Smedemark-Margulies, N., Wang, Y., Koike-Akino, T., Liu, J., Parsons, K., Bicer, Y., Erdogmus, D., "Improving Subject Transfer in EEG Classification with Divergence Estimation", Journal of Neural Engineering, DOI: 10.1088/1741-2552/ad9777, Vol. 21, No. 6, April 2025.
  BibTeX TR2025-044 PDF
  - @article{Smedemark-Margulies2025apr,
  - author = {Smedemark-Margulies, Niklas and Wang, Ye and Koike-Akino, Toshiaki and Liu, Jing and Parsons, Kieran and Bicer, Yunus and Erdogmus, Deniz},
  - title = {{Improving Subject Transfer in EEG Classification with Divergence Estimation}},
  - journal = {Journal of Neural Engineering},
  - year = 2025,
  - volume = 21,
  - number = 6,
  - month = apr,
  - doi = {10.1088/1741-2552/ad9777},
  - url = {https://www.merl.com/publications/TR2025-044}
  - }
- Wang, J.-W., Nikovski, D.N., "Learning Visual Servoing for Nonholonomic Mobile Robots with Uncalibrated Cameras", The 40th ACM/SIGAPP Symposium On Applied Computing, March 2025.
  BibTeX TR2025-042 PDF
  - @inproceedings{Wang2025mar2,
  - author = {Wang, Jen-Wei and Nikovski, Daniel N.},
  - title = {{Learning Visual Servoing for Nonholonomic Mobile Robots with Uncalibrated Cameras}},
  - booktitle = {The 40th ACM/SIGAPP Symposium On Applied Computing},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-042}
  - }
- Das, A., Rahman, S., Xiang, X., Palash, R.H., Hossain, T., Sikder, B., Yagyu, E., Nakamura, M., Teo, K.H., Chowdhury, N., "Inverse Design of AlGaN/GaN HEMT RF Device with Source Connected Field Plate", Advanced Theory and Simulations, March 2025.
  BibTeX TR2025-040 PDF
  - @article{Das2025mar,
  - author = {Das, Aurick and Rahman, Saimur and Xiang, Xiaofeng and Palash, Raffd Hassan and Hossain, Toiyob and Sikder, Bejoy and Yagyu, Eiji and Nakamura, Marika and Teo, Koon Hoo and Chowdhury, Nadim},
  - title = {{Inverse Design of AlGaN/GaN HEMT RF Device with Source Connected Field Plate}},
  - journal = {Advanced Theory and Simulations},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-040}
  - }
- Ranade, S., Pais, G., Whitaker, R., Nascimento, J., Miraldo, P., Ramalingam, S., "SurfR: Surface Reconstruction with Multi-scale Attention", International Conference on 3D Vision (3DV), March 2025.
  BibTeX TR2025-039 PDF
  - @inproceedings{Ranade2025mar,
  - author = {Ranade, Siddhant and Pais, Goncalo and Whitaker, Ross and Nascimento, Jacinto and Miraldo, Pedro and Ramalingam, Srikumar},
  - title = {{SurfR: Surface Reconstruction with Multi-scale Attention}},
  - booktitle = {International Conference on 3D Vision (3DV)},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-039}
  - }
- Araki, S., Ito, N., Haeb-Umbach, R., Wichern, G., Wang, Z.-Q., Mitsufuji, Y., "30+ Years of Source Separation Research: Achievements and Future Challenges", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2025.
  BibTeX TR2025-036 PDF
  - @inproceedings{Araki2025mar,
  - author = {Araki, Shoko and Ito, Nobutaka and Haeb-Umbach, Reinhold and Wichern, Gordon and Wang, Zhong-Qiu and Mitsufuji, Yuki},
  - title = {{30+ Years of Source Separation Research: Achievements and Future Challenges}},
  - booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-036}
  - }
- Ebbers, J., Germain, F.G., Wilkinghoff, K., Wichern, G., Le Roux, J., "No Class Left Behind: A Closer Look at Class Balancing for Audio Tagging", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2025.
  BibTeX TR2025-037 PDF
  - @inproceedings{Ebbers2025mar,
  - author = {Ebbers, Janek and Germain, François G and Wilkinghoff, Kevin and Wichern, Gordon and {Le Roux}, Jonathan},
  - title = {{No Class Left Behind: A Closer Look at Class Balancing for Audio Tagging}},
  - booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-037}
  - }
- Gruttadauria, E., Fontaine, M., Le Roux, J., Essid, S., "O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), March 2025.
  BibTeX TR2025-031 PDF
  - @inproceedings{Gruttadauria2025mar,
  - author = {Gruttadauria, Elio and Fontaine, Mathieu and {Le Roux}, Jonathan and Essid, Slim},
  - title = {{O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization}},
  - booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
  - year = 2025,
  - month = mar,
  - url = {https://www.merl.com/publications/TR2025-031}
  - }
See All Publications for Machine Learning
Videos

[MERL Seminar Series Spring 2025] Red Teaming AI Agents in-the-wild: Revealing Deployment Vulnerabilities

[MERL Seminar Series Spring 2025] The Emergence of Generalizability and Semantic Low-Dim Subspaces in Diffusion Models

[WACV 2025] Towards Zero-shot 3D Anomaly Localization

Data-driven Spatial Classification using Multi-Arm Bandits for Monitoring with Robot Teams

[NeurIPS 2024] MEL-PETs Defense for the NeurIPS 2024 LLM Privacy Challenge Blue Team Track

[NeurIPS 2024] MEL-PETs Joint-Context Attack for the NeurIPS 2024 LLM Privacy Challenge Red Team Track

[NeurIPS 2024] Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads

[MERL Seminar Series Fall 2024] Audio for Object and Spatial Awareness

[IROS 2024] Few-shot Transparent Instance Segmentation for Bin Picking

[MERL Seminar Series Fall 2024] Tools from cognitive science to understand the behavior of large language models

[ECCV 2024] PS-NEUS: A Probability-guided Sampler for Neural Implicit Surface Rendering

[ECCV 2024] Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection

[CVPR 2024] SIRA: Scalable Inter-frame Relation and Association for Radar Perception

Decentralized, Safe, Multi-agent Motion Planning for Drones Under Uncertainty via Filtered Reinforcement Learning

[MERL Seminar Series Spring 2024] Neural Certificates and LLMs in Large-Scale Autonomy Design

[MERL Seminar Series Spring 2024] Decoding Hidden Worlds: Unprecedented Sensing and Connectivity for Climate, Robotics, & Smart Environments

Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling

[MERL Seminar Series Spring 2024] Are Emergent Abilities of Large Language Models a Mirage?

MERL's Quantum AI Technology

[GLOBECOM 2022/VCC 2023 Tutorial] Post-Deep Learning Era: Emerging Quantum Machine Learning for Sensing and Communications (Session 2)

[GLOBECOM 2022/VCC 2023 Tutorial] Post-Deep Learning Era: Emerging Quantum Machine Learning for Sensing and Communications (Session I)

[MERL Seminar Series Fall 2023] Robust and Physics-informed machine learning for low light imaging

[MERL Seminar Series Fall 2023] Multiplicity in Machine Learning

Multi-level Reasoning for Robotic Assembly: From Sequence Inference to Contact Selection

[MERL Seminar Series Fall 2023] Visual Programming - A compositional approach to building General Purpose Vision Systems

[MERL Seminar Series Fall 2023] The Confluence of Vision, Language, and Robotics

[MERL Seminar Series Fall 2023] A Process Systems Engineering Perspective on Carbon Capture: Key Challenges and Opportunities

[TRO 2024] Safe Multiagent Motion Planning Under Uncertainty for Drones Using Filtered Reinforcement Learning

Are Deep Neural Networks SMARTer than Second Graders?

[CVPR 2023] EVAL: Explainable Video Anomaly Localization

[MERL Seminar Series Spring 2023] Learning and Dynamical Systems

[MERL Seminar Series Spring 2023] Investigating Multi-Agent Reinforcement Learning for Grid-Interactive Smart Communities using CityLearn

[MERL Seminar Series Spring 2023] Pitfalls and Opportunities in Interpretable Machine Learning

[MERL Seminar Series Spring 2023] Neural Implicit Flow

[MERL Seminar Series Spring 2023] Towards Complex Language in Partially Observed Environments

[MERL Seminar Series Spring 2022] Hybrid robotics and implicit learning

Toshiaki Koike-Akino Gives Seminar Talk at IEEE Boston Photonics

[MERL Seminar Series Spring 2022] RLMPC: An Ideal Combination of Formal Optimal Control and Reinforcement Learning?

[MERL Seminar Series Spring 2022] Self-Supervised Scene Representation Learning

[MERL Seminar Series Spring 2022] Learning Speech Representations with Multimodal Self-Supervision

[MERL Seminar Series Spring 2022] Extreme optics design as a large-scale optimization problem

HealthCam: A system for non-contact monitoring of vital signs

[MERL Seminar Series 2021] Harnessing machine learning to build better Earth system models for climate projection

[MERL Seminar Series 2021] Deep probabilistic regression

[MERL Seminar Series 2021] Learning to See by Moving: Self-supervising 3D scene representations for perception, control, and visual reasoning

Control of Mechanical Systems via Feedback Linearization Based on Black-Box Gaussian Process Models

Application of Deep Learning for Nanophotonic Device Design (Invited)

Towards Human-Level Learning of Complex Physical Puzzles

Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry

Scene-Aware Interaction Technology

Action Detection Using A Deep Recurrent Neural Network

Semantic Scene Labeling

Obstacle Detection

MERL Research on Autonomous Vehicles

Deep Hierarchical Parsing for Semantic Segmentation

Global Local Face Upsampling Network
Software & Data Downloads

Software & Data Downloads

MERL is making Machine Learning software and data available to the research community:

Self-Monitored Inference-Time INtervention for Generative Music Transformers (SMITIN)
Retrieval-Augmented Neural Field for HRTF Upsampling and Personalization (ranf-hrtf)
Radar dEtection TRansformer (RETR)
Meta-Learning State Space Models (MetaLIC)
MEL-PETs Joint-Context Attack for LLM Privacy Challenge (melpets-llmpc2024-red-team)
MEL-PETs Defense for LLM Privacy Challenge (melpets-llmpc2024-blue-team)

See All Downloads