Machine Learning
Data-driven approaches to design intelligent algorithms.
MERL has a long history of research activity in machine learning, including the development of various boosting algorithms and contributing to the theory and practice of highly scalable collaborative filtering. Our recent work has focused on deep learning and reinforcement learning, with application to a wide range of applications including automotive, robotics, factory automation, transportation, as well as building and home systems.
Quick Links
-
Researchers
Jonathan
Le Roux
Takaaki
Hori
Devesh
Jha
Toshiaki
Koike-Akino
Michael
Jones
Tim
Marks
Philip
Orlik
Ye
Wang
Anoop
Cherian
Daniel
Nikovski
Gordon
Wichern
Chiori
Hori
Yebin
Wang
Diego
Romeres
Karl
Berntorp
Ankush
Chakrabarty
Stefano
Di Cairano
Niko
Moritz
Arvind
Raghunathan
Alan
Sullivan
Jeroen
van Baar
Keisuke
Kojima
Kieran
Parsons
Pu
(Perry)
WangHassan
Mansour
Bingnan
Wang
Mouhacine
Benosman
Uroš
Kalabić
Petros
Boufounos
Rien
Quirynen
Hongbo
Sun
Radu
Corcodel
Kyeong Jin
(K.J.)
KimWilliam
Yerazunis
Matthew
Brand
Jianlin
Guo
Chungwei
Lin
Dehong
Liu
Suhas
Lohit
Marcel
Menner
Jing
Zhang
Jinyun
Zhang
Siddarth
Jain
Christopher
Laughman
Rui
Ma
Koon Hoo
Teo
Varun
Haritsa
Kuan-Chuan
Peng
Abraham
P. Vinod
-
Awards
-
AWARD Excellent Presentation Award Date: January 25, 2021
Awarded to: Takenori Sumi, Yukimasa Nagai, Jianlin Guo, Philip Orlik, Tatsuya Yokoyama, Hiroshi Mineno
MERL Contacts: Jianlin Guo; Philip Orlik
Research Areas: Communications, Machine Learning, Signal ProcessingBrief- MELCO and MERL researchers have won "Excellent Presentation Award" at the IPSJ/CDS30 (Information Processing Society of Japan/Consumer Devices and Systems 30th conferences) held on January 25, 2021. The paper titled "Sub-1 GHz Coexistence Using Reinforcement Learning Based IEEE 802.11ah RAW Scheduling" addresses coexistence between IEEE 802.11ah and IEEE 802.15.4g systems in the Sub-1 GHz frequency bands. This paper proposes a novel method to allocate IEEE 802.11 RAW time slots using a Q-Learning technique. MERL and MELCO have been leading IEEE 802.19.3 coexistence standard development and this paper is a good candidate for future standard enhancement. The authors are Takenori Sumi, Yukimasa Nagai, Jianlin Guo, Philip Orlik, Tatsuya Yokoyama and Hiroshi Mineno.
-
AWARD Best Paper - Honorable Mention Award at WACV 2021 Date: January 6, 2021
Awarded to: Rushil Anirudh, Suhas Lohit, Pavan Turaga
MERL Contact: Suhas Lohit
Research Areas: Computational Sensing, Computer Vision, Machine LearningBrief- A team of researchers from Mitsubishi Electric Research Laboratories (MERL), Lawrence Livermore National Laboratory (LLNL) and Arizona State University (ASU) received the Best Paper Honorable Mention Award at WACV 2021 for their paper "Generative Patch Priors for Practical Compressive Image Recovery".
The paper proposes a novel model of natural images as a composition of small patches which are obtained from a deep generative network. This is unlike prior approaches where the networks attempt to model image-level distributions and are unable to generalize outside training distributions. The key idea in this paper is that learning patch-level statistics is far easier. As the authors demonstrate, this model can then be used to efficiently solve challenging inverse problems in imaging such as compressive image recovery and inpainting even from very few measurements for diverse natural scenes.
- A team of researchers from Mitsubishi Electric Research Laboratories (MERL), Lawrence Livermore National Laboratory (LLNL) and Arizona State University (ASU) received the Best Paper Honorable Mention Award at WACV 2021 for their paper "Generative Patch Priors for Practical Compressive Image Recovery".
-
AWARD Best Poster Award and Best Video Award at the International Society for Music Information Retrieval Conference (ISMIR) 2020 Date: October 15, 2020
Awarded to: Ethan Manilow, Gordon Wichern, Jonathan Le Roux
MERL Contacts: Jonathan Le Roux; Gordon Wichern
Research Areas: Artificial Intelligence, Machine Learning, Speech & AudioBrief- Former MERL intern Ethan Manilow and MERL researchers Gordon Wichern and Jonathan Le Roux won Best Poster Award and Best Video Award at the 2020 International Society for Music Information Retrieval Conference (ISMIR 2020) for the paper "Hierarchical Musical Source Separation". The conference was held October 11-14 in a virtual format. The Best Poster Awards and Best Video Awards were awarded by popular vote among the conference attendees.
The paper proposes a new method for isolating individual sounds in an audio mixture that accounts for the hierarchical relationship between sound sources. Many sounds we are interested in analyzing are hierarchical in nature, e.g., during a music performance, a hi-hat note is one of many such hi-hat notes, which is one of several parts of a drumkit, itself one of many instruments in a band, which might be playing in a bar with other sounds occurring. Inspired by this, the paper re-frames the audio source separation problem as hierarchical, combining similar sounds together at certain levels while separating them at other levels, and shows on a musical instrument separation task that a hierarchical approach outperforms non-hierarchical models while also requiring less training data. The paper, poster, and video can be seen on the paper page on the ISMIR website.
- Former MERL intern Ethan Manilow and MERL researchers Gordon Wichern and Jonathan Le Roux won Best Poster Award and Best Video Award at the 2020 International Society for Music Information Retrieval Conference (ISMIR 2020) for the paper "Hierarchical Musical Source Separation". The conference was held October 11-14 in a virtual format. The Best Poster Awards and Best Video Awards were awarded by popular vote among the conference attendees.
See All Awards for Machine Learning -
-
News & Events
-
NEWS Ankush Chakrabarty gave an invited talk at University of Illinois at Chicago Date: April 9, 2021
MERL Contact: Ankush Chakrabarty
Research Areas: Control, Machine Learning, Multi-Physical Modeling, OptimizationBrief- Ankush Chakrabarty, a Research Scientist at MERL's Multiphysical Systems (MS) Team, gave an invited talk on "Learning for Control and Estimation using Digital Twins" at the Department of Electrical and Computer Engineering Seminar Series organized at UIC. The talk proposed new learning-based control/estimation architectures that can utilize simulation data obtained from digital twins to add self-optimization and constraint-enforcement features to grey/black-box control systems.
-
NEWS Invited talk at University of Leeds Date: April 7, 2021
Where: Online
MERL Contact: Devesh Jha
Research Areas: Artificial Intelligence, Machine Learning, RoboticsBrief- Devesh Jha, a Principal Research Scientist in MERL's Data Analytics group, gave an invited talk at the robotics seminar series at the University of Leeds. The talk presented some of the recent work done at MERL in the areas of robotic manipulation and robot learning.
See All News & Events for Machine Learning -
-
Research Highlights
-
Internships
-
MD1300: Compiler Optimizations for Linear Algebra Kernels
MERL is looking for a highly motivated individual to work on automatic, compiler based techniques for optimizing linear algebra kernels. The ideal candidate is a Ph.D. student in computer science with extensive experience in compiler design and source code optimization techniques. In particular, the successful candidate will have a strong working knowledge of polyhedral optimization techniques, the LLVM compiler, and Polly. Strong C/C++ skills and knowledge of LLVM at the source level are required. Publication of results in conference proceedings and journals is expected. The expected duration of the internship is 3 months and the start date is flexible.
-
CV1568: Uncertainty Estimation in 3D Face Landmark Tracking
We are seeking a highly motivated intern to conduct original research extending MERL's work on uncertainty estimation in face landmark localization (the LUVLi model) to the domains of 3D faces and video sequences. The successful candidate will collaborate with MERL researchers to design and implement new models, conduct experiments, and prepare results for publication. The candidate should be a PhD student in computer vision and machine learning with a strong publication record. Experience in deep learning-based face landmark estimation, video tracking, and 3D face modeling is preferred. Strong programming skills, experience developing and implementing new models in deep learning platforms such as PyTorch, and broad knowledge of machine learning and deep learning methods are expected.
-
CA1529: Energy Management for Electric Vehicles
MERL is looking for a highly motivated intern to conduct research on data-driven energy management strategies for (hybrid) electric vehicles. The candidate will develop methods that use data, e.g., of human drivers or traffic conditions, in order to improve the control of electric vehicles. The ideal candidate will have experience in either one or multiple of the following topics: model predictive control, machine learning, statistical learning, numerical optimization, and (inverse) optimal control. Prior experience with (hybrid) electric vehicles is a plus. Good programming skills in MATLAB, Python, or C/C++ are required. PhD students in engineering or mathematics with a focus on control theory or numerical optimization are encouraged to apply. Publication of relevant results in conference proceedings or journals is expected. The expected duration of the internship is 3-6 months. The start date is flexible. This internship is preferred to be onsite at MERL, but may be done remotely where you live if the COVID pandemic makes it necessary.
See All Internships for Machine Learning -
-
Recent Publications
- "Remaining Useful Life Estimation for LFP Cells in Second Life Applications", IEEE Transactions on Instrumentation and Measurement, DOI: 10.1109/TIM.2021.3055791, March 2021.BibTeX TR2021-023 PDF
- @article{Sanz-Gorrachategui2021mar,
- author = {Sanz-Gorrachategui, Ivan and Pastor-Flores, Pablo and Pajovic, Milutin and Wang, Ye and Orlik, Philip V. and Bernal-Ruiz, Carlos and Bono-Nuez, Antonio and Artal-Sevil, Jesús Sergio},
- title = {Remaining Useful Life Estimation for LFP Cells in Second Life Applications},
- journal = {IEEE Transactions on Instrumentation and Measurement},
- year = 2021,
- month = mar,
- doi = {10.1109/TIM.2021.3055791},
- url = {https://www.merl.com/publications/TR2021-023}
- }
, - "AutoBayes: Automated Bayesian Graph Exploration for Nuisance-Robust Inference", IEEE Access, DOI: 10.1109/ACCESS.2021.3064530, Vol. 9, pp. 39955-39972, March 2021.BibTeX TR2021-016 PDF
- @article{Demir2021mar,
- author = {Demir, Andac and Koike-Akino, Toshiaki and Wang, Ye and Erdogmus, Deniz},
- title = {AutoBayes: Automated Bayesian Graph Exploration for Nuisance-Robust Inference},
- journal = {IEEE Access},
- year = 2021,
- volume = 9,
- pages = {39955--39972},
- month = mar,
- doi = {10.1109/ACCESS.2021.3064530},
- issn = {2169-3536},
- url = {https://www.merl.com/publications/TR2021-016}
- }
, - "Application of Deep Learning for Nanophotonic Device Design", SPIE Photonics West, Bahram Jalali and Ken-ichi Kitayama, Eds., DOI: 10.1117/12.2579104, March 2021.BibTeX TR2020-182 PDF Video
- @inproceedings{Kojima2021mar,
- author = {Kojima, Keisuke and Tang, Yingheng and Koike-Akino, Toshiaki and Wang, Ye and Jha, Devesh and TaherSima, Mohammad and Parsons, Kieran},
- title = {Application of Deep Learning for Nanophotonic Device Design},
- booktitle = {SPIE Photonics West},
- year = 2021,
- editor = {Bahram Jalali and Ken-ichi Kitayama},
- month = mar,
- publisher = {SPIE},
- doi = {10.1117/12.2579104},
- url = {https://www.merl.com/publications/TR2020-182}
- }
, - "Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers", AAAI Conference on Artificial Intelligence, February 2021.BibTeX TR2021-010 PDF
- @inproceedings{Geng2021feb,
- author = {Geng, Shijie and Gao, Peng and Chatterjee, Moitreya and Hori, Chiori and Le Roux, Jonathan and Zhang, Yongfeng and Li, Hongsheng and Cherian, Anoop},
- title = {Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers},
- booktitle = {AAAI Conference on Artificial Intelligence},
- year = 2021,
- month = feb,
- url = {https://www.merl.com/publications/TR2021-010}
- }
, - "Deep Neural Networks for Inverse Design of Nanophotonic Devices", IEEE Journal of Lightwave Technology, DOI: 10.1109/JLT.2021.3050083, January 2021.BibTeX TR2021-001 PDF
- @article{Kojima2021jan,
- author = {Kojima, Keisuke and TaherSima, Mohammad and Koike-Akino, Toshiaki and Jha, Devesh and Tang, Yingheng and Wang, Ye and Parsons, Kieran},
- title = {Deep Neural Networks for Inverse Design of Nanophotonic Devices},
- journal = {IEEE Journal of Lightwave Technology},
- year = 2021,
- month = jan,
- doi = {10.1109/JLT.2021.3050083},
- issn = {1558-2213},
- url = {https://www.merl.com/publications/TR2021-001}
- }
, - "Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization", IEEE Winter Conference on Applications of Computer Vision (WACV), January 2021.BibTeX TR2021-004 PDF
- @inproceedings{Lohit2021jan,
- author = {Lohit, Suhas and Anirudh, Rushil and Turaga, Pavan},
- title = {Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization},
- booktitle = {IEEE Winter Conference on Applications of Computer Vision (WACV)},
- year = 2021,
- month = jan,
- url = {https://www.merl.com/publications/TR2021-004}
- }
,
- "Remaining Useful Life Estimation for LFP Cells in Second Life Applications", IEEE Transactions on Instrumentation and Measurement, DOI: 10.1109/TIM.2021.3055791, March 2021.
-
Videos
-
Control of Mechanical Systems via Feedback Linearization Based on Black-Box Gaussian Process Models
-
Application of Deep Learning for Nanophotonic Device Design (Invited)
-
Towards Human-Level Learning of Complex Physical Puzzles
-
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry
-
[ACP 2020] Inverse Design of Nanophotonic Devices using Deep Neural Networks
-
Action Detection Using A Deep Recurrent Neural Network
-
MERL Research on Autonomous Vehicles
-
Obstacle Detection
-
Semantic Scene Labeling
-
Deep Hierarchical Parsing for Semantic Segmentation
-
Global Local Face Upsampling Network
-
-
Software Downloads
-
Generating Visual Dynamics from Sound and Context
-
Adversarially-Contrastive Optimal Transport
-
Online Feature Extractor Network
-
MotionNet
-
FoldingNet++
-
Quasi-Newton Trust Region Policy Optimization
-
Landmarks’ Location, Uncertainty, and Visibility Likelihood
-
Robust Iterative Data Estimation
-
Gradient-based Nikaido-Isoda
-
Circular Maze Environment
-
Discriminative Subspace Pooling
-
Kernel Correlation Network
-
Fast Resampling on Point Clouds via Graphs
-
FoldingNet
-
Deep Category-Aware Semantic Edge Detection
-