News & Events

1,516 News items, Awards, Events and Talks related to MERL and its staff.


  •  TALK    [MERL Seminar Series 2023] Prof. Flavio Calmon presents talk titled Multiplicity in Machine Learning
    Date & Time: Tuesday, November 7, 2023; 12:00 PM
    Speaker: Flavio Calmon, Harvard University
    MERL Host: Ye Wang
    Research Areas: Artificial Intelligence, Machine Learning
    Abstract
    • This talk reviews the concept of predictive multiplicity in machine learning. Predictive multiplicity arises when different classifiers achieve similar average performance for a specific learning task yet produce conflicting predictions for individual samples. We discuss a metric called “Rashomon Capacity” for quantifying predictive multiplicity in multi-class classification. We also present recent findings on the multiplicity cost of differentially private training methods and group fairness interventions in machine learning.

      This talk is based on work published at ICML'20, NeurIPS'22, ACM FAccT'23, and NeurIPS'23.
  •  
  •  NEWS    Invited talk given by Diego Romeres at Bentley University
    Date: November 1, 2023
    MERL Contact: Diego Romeres
    Research Areas: Artificial Intelligence, Machine Learning, Robotics
    Brief
    • Principal Research Scientist and Team Leader Diego Romeres gave an invited talk with title 'Applications of Machine Learning to Robotics' in the Machine Learning graduate course at Bentley University. The presentation focused mainly on Reinforcement Learning research applied to robotics. The audience consisted mostly of Master’s in Business Analytics (MSBA) students and students in the MBA w/ Business Analytics Concentration program.
  •  
  •  TALK    [MERL Seminar Series 2023] Dr. Tanmay Gupta presents talk titled Visual Programming - A compositional approach to building General Purpose Vision Systems
    Date & Time: Tuesday, October 31, 2023; 2:00 PM
    Speaker: Tanmay Gupta, Allen Institute for Artificial Intelligence
    MERL Host: Moitreya Chatterjee
    Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
    Abstract
    • Building General Purpose Vision Systems (GPVs) that can perform a huge variety of tasks has been a long-standing goal for the computer vision community. However, end-to-end training of these systems to handle different modalities and tasks has proven to be extremely challenging. In this talk, I will describe a lucrative neuro-symbolic alternative to the common end-to-end learning paradigm called Visual Programming. Visual Programming is a general framework that leverages the code-generation abilities of LLMs, existing neural models, and non-differentiable programs to enable powerful applications. Some of these applications continue to remain elusive for the current generation of end-to-end trained GPVs.
  •  
  •  EVENT    Prof. Yuejie Chi of Carnegie Mellon University to Deliver Keynote at MERL's Virtual Open House
    Date & Time: Wednesday, November 15, 2023; 3:00-3:40pm (EST)
    Location: Virtual Event
    Speaker: Prof. Yuejie Chi, Carnegie Mellon University
    MERL Contact: Bingnan Wang
    Brief
    • MERL is excited to announce the featured keynote speaker for our Virtual Open House 2023: Prof. Yuejie Chi from Carnegie Mellon University.

      Our virtual open house this year will take place on November 15, 2023, 1:00pm - 5:30pm (EST). Prof. Chi’s talk is scheduled for 3:00-3:40pm (EST). For details and agenda of the event, please visit: https://merl.com/events/voh23

      Join us to learn more about who we are, what we do, and discuss our internship, post-doc, and full-time employment opportunities. To register, go to: https://mailchi.mp/merl/voh23


      Title: Sample Complexity of Q-learning: from Single-agent to Federated Learning

      Abstract: Q-learning, which seeks to learn the optimal Q-function of a Markov decision process (MDP) in a model-free fashion, lies at the heart of reinforcement learning practices. However, theoretical understandings on its non-asymptotic sample complexity remain unsatisfactory, despite significant recent efforts. In this talk, we first show a tight sample complexity bound of Q-learning in the single-agent setting, together with a matching lower bound to establish its minimax sub-optimality. We then show how federated versions of Q-learning allow collaborative learning using data collected by multiple agents without central sharing, where an importance averaging scheme is introduced to unveil the blessing of heterogeneity.
  •  
  •  NEWS    MERL co-organizes the 2023 Sound Demixing (SDX2023) Challenge and Workshop
    Date: January 23, 2023 - November 4, 2023
    Where: International Symposium of Music Information Retrieval (ISMR)
    MERL Contacts: Jonathan Le Roux; Gordon Wichern
    Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
    Brief
    • MERL Speech & Audio team members Gordon Wichern and Jonathan Le Roux co-organized the 2023 Sound Demixing Challenge along with researchers from Sony, Moises AI, Audioshake, and Meta.

      The SDX2023 Challenge was hosted on the AI Crowd platform and had a prize pool of $42,000 distributed to the winning teams across two tracks: Music Demixing and Cinematic Sound Demixing. A unique aspect of this challenge was the ability to test the audio source separation models developed by challenge participants on non-public songs from Sony Music Entertainment Japan for the music demixing track, and movie soundtracks from Sony Pictures for the cinematic sound demixing track. The challenge ran from January 23rd to May 1st, 2023, and had 884 participants distributed across 68 teams submitting 2828 source separation models. The winners will be announced at the SDX2023 Workshop, which will take place as a satellite event at the International Symposium of Music Information Retrieval (ISMR) in Milan, Italy on November 4, 2023.

      MERL’s contribution to SDX2023 focused mainly on the cinematic demixing track. In addition to sponsoring the prizes awarded to the winning teams for that track, the baseline system and initial training data were MERL’s Cocktail Fork separation model and Divide and Remaster dataset, respectively. MERL researchers also contributed to a Town Hall kicking off the challenge, co-authored a scientific paper describing the challenge outcomes, and co-organized the SDX2023 Workshop.
  •  
  •  EVENT    MERL's Virtual Open House 2023
    Date & Time: Wednesday, November 15, 2023; 1:00 - 5:30 EST
    Location: Virtual Event
    MERL Contact: Bingnan Wang
    Brief
    • Join us for MERL's Virtual Open House (VOH) 2023 on November 15th. Live sessions will be held from 1:00-5:30pm EST, including an overview of recent activities by our research groups, a featured guest speaker and live interaction with our research staff through the Gather platform. Registered attendees will be able to browse our virtual booths at their convenience and connect with our research staff to learn about engagement opportunities, including internship/post-doc openings as well as visiting faculty positions.

      For agenda and details of the event: https://www.merl.com/events/voh23

      To register for the VOH, go to:
      https://mailchi.mp/merl/voh23
  •  
  •  TALK    [MERL Seminar Series 2023] Prof. Shaoshuai Mou presents talk titled Inverse Optimal Control for Autonomous Systems
    Date & Time: Tuesday, October 10, 2023; 1:00 PM
    Speaker: Shaoshuai Mou, Purdue University
    MERL Host: Yebin Wang
    Research Areas: Control, Dynamical Systems, Robotics
    Abstract
    • Inverse Optimal Control (IOC) aims to achieve an objective function corresponding to a certain task from an expert robot driven by optimal control, which has become a powerful tool in many applications in robotics. We will present our recent solutions to IOC based on incomplete observations of systems' trajectories, which enables an autonomous system to “sense-and-adapt", i.e., incrementally improving the learning of objective functions as new data arrives. This also leads to a distributed algorithm to solve IOC in multi-agent systems, in which each agent can only access part of the overall trajectory of an optimal control system and cannot solve IOC by itself. This is perhaps the first distributed method to IOC. Applications of IOC into human prediction will also be given.
  •  
  •  NEWS    MERL researchers presenting four papers and organizing the VLAR-SMART101 Workshop at ICCV 2023
    Date: October 2, 2023 - October 6, 2023
    Where: Paris/France
    MERL Contacts: Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Toshiaki Koike-Akino; Suhas Lohit; Tim K. Marks; Pedro Miraldo; Kuan-Chuan Peng; Ye Wang
    Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
    Brief
    • MERL researchers are presenting 4 papers and organizing the VLAR-SMART-101 workshop at the ICCV 2023 conference, which will be held in Paris, France October 2-6. ICCV is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.

      1. Conference paper: “Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis,” by Nithin Gopalakrishnan Nair, Anoop Cherian, Suhas Lohit, Ye Wang, Toshiaki Koike-Akino, Vishal Patel, and Tim K. Marks

      Conditional generative models typically demand large annotated training sets to achieve high-quality synthesis. As a result, there has been significant interest in plug-and-play generation, i.e., using a pre-defined model to guide the generative process. In this paper, we introduce Steered Diffusion, a generalized framework for fine-grained photorealistic zero-shot conditional image generation using a diffusion model trained for unconditional generation. The key idea is to steer the image generation of the diffusion model during inference via designing a loss using a pre-trained inverse model that characterizes the conditional task. Our model shows clear qualitative and quantitative improvements over state-of-the-art diffusion-based plug-and-play models, while adding negligible computational cost.

      2. Conference paper: "BANSAC: A dynamic BAyesian Network for adaptive SAmple Consensus," by Valter Piedade and Pedro Miraldo

      We derive a dynamic Bayesian network that updates individual data points' inlier scores while iterating RANSAC. At each iteration, we apply weighted sampling using the updated scores. Our method works with or without prior data point scorings. In addition, we use the updated inlier/outlier scoring for deriving a new stopping criterion for the RANSAC loop. Our method outperforms the baselines in accuracy while needing less computational time.

      3. Conference paper: "Robust Frame-to-Frame Camera Rotation Estimation in Crowded Scenes," by Fabien Delattre, David Dirnfeld, Phat Nguyen, Stephen Scarano, Michael J. Jones, Pedro Miraldo, and Erik Learned-Miller

      We present a novel approach to estimating camera rotation in crowded, real-world scenes captured using a handheld monocular video camera. Our method uses a novel generalization of the Hough transform on SO3 to efficiently find the camera rotation most compatible with the optical flow. Because the setting is not addressed well by other data sets, we provide a new dataset and benchmark, with high-accuracy and rigorously annotated ground truth on 17 video sequences. Our method is more accurate by almost 40 percent than the next best method.

      4. Workshop paper: "Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection" by Manish Sharma*, Moitreya Chatterjee*, Kuan-Chuan Peng, Suhas Lohit, and Michael Jones

      While state-of-the-art object detection methods for RGB images have reached some level of maturity, the same is not true for Infrared (IR) images. The primary bottleneck towards bridging this gap is the lack of sufficient labeled training data in the IR images. Towards addressing this issue, we present TensorFact, a novel tensor decomposition method which splits the convolution kernels of a CNN into low-rank factor matrices with fewer parameters. This compressed network is first pre-trained on RGB images and then augmented with only a few parameters. This augmented network is then trained on IR images, while freezing the weights trained on RGB. This prevents it from over-fitting, allowing it to generalize better. Experiments show that our method outperforms state-of-the-art.

      5. “Vision-and-Language Algorithmic Reasoning (VLAR) Workshop and SMART-101 Challenge” by Anoop Cherian,  Kuan-Chuan Peng, Suhas Lohit, Tim K. Marks, Ram Ramrakhya, Honglu Zhou, Kevin A. Smith, Joanna Matthiesen, and Joshua B. Tenenbaum

      MERL researchers along with researchers from MIT, GeorgiaTech, Math Kangaroo USA, and Rutgers University are jointly organizing a workshop on vision-and-language algorithmic reasoning at ICCV 2023 and conducting a challenge based on the SMART-101 puzzles described in the paper: Are Deep Neural Networks SMARTer than Second Graders?. A focus of this workshop is to bring together outstanding faculty/researchers working at the intersections of vision, language, and cognition to provide their opinions on the recent breakthroughs in large language models and artificial general intelligence, as well as showcase their cutting edge research that could inspire the audience to search for the missing pieces in our quest towards solving the puzzle of artificial intelligence.

      Workshop link: https://wvlar.github.io/iccv23/
  •  
  •  TALK    [MERL Seminar Series 2023] Prof. Komei Sugiura presents talk titled The Confluence of Vision, Language, and Robotics
    Date & Time: Thursday, September 28, 2023; 12:00 PM
    Speaker: Komei Sugiura, Keio University
    MERL Host: Chiori Hori
    Research Areas: Artificial Intelligence, Machine Learning, Robotics, Speech & Audio
    Abstract
    • Recent advances in multimodal models that fuse vision and language are revolutionizing robotics. In this lecture, I will begin by introducing recent multimodal foundational models and their applications in robotics. The second topic of this talk will address our recent work on multimodal language processing in robotics. The shortage of home care workers has become a pressing societal issue, and the use of domestic service robots (DSRs) to assist individuals with disabilities is seen as a possible solution. I will present our work on DSRs that are capable of open-vocabulary mobile manipulation, referring expression comprehension and segmentation models for everyday objects, and future captioning methods for cooking videos and DSRs.
  •  
  •  TALK    [MERL Seminar Series 2023] Prof. Zac Manchester presents talk titled Composable Optimization for Robotic Simulation, Planning, and Control
    Date & Time: Wednesday, September 27, 2023; 1:00 PM
    Speaker: Zac Manchester, Carnegie Mellon University
    MERL Host: Devesh K. Jha
    Research Areas: Optimization, Robotics
    Abstract
    • Contact interactions are pervasive in key real-world robotic tasks like manipulation and walking. However, the non-smooth dynamics associated with impacts and friction remain challenging to model, and motion planning and control algorithms that can fluently and efficiently reason about contact remain elusive. In this talk, I will share recent work from my research group that takes an “optimization-first” approach to these challenges: collision detection, physics, motion planning, and control are all posed as constrained optimization problems. We then build a set of algorithmic and numerical tools that allow us to flexibly compose these optimization sub-problems to solve complex robotics problems involving discontinuous, unplanned, and uncertain contact mechanics.
  •  
  •  TALK    [MERL Seminar Series 2023] Prof. Faruque Hasan presents talk titled A Process Systems Engineering Perspective on Carbon Capture: Key Challenges and Opportunities
    Date & Time: Tuesday, September 19, 2023; 1:00 PM
    Speaker: Faruque Hasan, Texas A&M University
    MERL Host: Scott A. Bortoff
    Research Areas: Applied Physics, Machine Learning, Multi-Physical Modeling, Optimization
    Abstract
    • Carbon capture, utilization, and storage (CCUS) is a promising pathway to decarbonize fossil-based power and industrial sectors and is a bridging technology for a sustainable transition to a net-zero emission energy future. This talk aims to provide an overview of design and optimization of CCUS systems. I will also attempt to give a brief perspective on emerging interests in process systems engineering research (e.g., systems integration, multiscale modeling, strategic planning, and optimization under uncertainty). The purpose is not to cover all aspects of PSE research for CCUS but rather to foster discussion by presenting some plausible future directions and ideas.
  •  
  •  AWARD    Best paper award at PHMAP 2023
    Date: September 14, 2023
    Awarded to: Dehong Liu, Anantaram Varatharajan, and Abraham Goldsmith
    MERL Contacts: Abraham Goldsmith; Dehong Liu
    Research Areas: Electric Systems, Signal Processing
    Brief
    • MERL researchers Dehong Liu, Anantaram Varatharajan, and Abraham Goldsmith were awarded one of three best paper awards at Asia Pacific Conference of the Prognostics and Health Management Society 2023 (PHMAP23) held in Tokyo from September 11th to 14th, 2023, for their co-authored paper titled 'Extracting Broken-Rotor-Bar Fault Signature of Varying-Speed Induction Motors.'

      PHMAP is a biennial international conference specialized in prognostics and health management. PHMAP23 attracted more than 300 attendees from worldwide and published more than 160 regular papers from academia and industry including aerospace, production, civil engineering, electronics, and so on.
  •  
  •  AWARD    Joint University of Padua-MERL team wins Challenge 'AI Olympics With RealAIGym'
    Date: August 25, 2023
    Awarded to: Alberto Dalla Libera, Niccolo' Turcato, Giulio Giacomuzzo, Ruggero Carli, Diego Romeres
    MERL Contact: Diego Romeres
    Research Areas: Artificial Intelligence, Machine Learning, Robotics
    Brief
    • A joint team consisting of members of University of Padua and MERL ranked 1st in the IJCAI2023 Challenge "Al Olympics With RealAlGym: Is Al Ready for Athletic Intelligence in the Real World?". The team was composed by MERL researcher Diego Romeres and a team from University Padua (UniPD) consisting of Alberto Dalla Libera, Ph.D., Ph.D. Candidates: Niccolò Turcato, Giulio Giacomuzzo and Prof. Ruggero Carli from University of Padua.

      The International Joint Conference on Artificial Intelligence (IJCAI) is a premier gathering for AI researchers and organizes several competitions. This year the competition CC7 "AI Olympics With RealAIGym: Is AI Ready for Athletic Intelligence in the Real World?" consisted of two stages: simulation and real-robot experiments on two under-actuated robotic systems. The two robotics systems were treated as separate tracks and one final winner was selected for each track based on specific performance criteria in the control tasks.

      The UniPD-MERL team competed and won in both tracks. The team's system made strong use of a Model-based Reinforcement Learning algorithm called (MC-PILCO) that we recently published in the journal IEEE Transaction on Robotics.
  •  
  •  AWARD    Best Paper Award at SDEMPED 2023
    Date: August 30, 2023
    Awarded to: Bingnan Wang, Hiroshi Inoue, and Makoto Kanemaru
    MERL Contact: Bingnan Wang
    Research Areas: Applied Physics, Data Analytics, Multi-Physical Modeling
    Brief
    • MERL and Mitsubishi Electric's paper titled “Motor Eccentricity Fault Detection: Physics-Based and Data-Driven Approaches” was awarded one of three best paper awards at the 14th IEEE International Symposium on Diagnostics for Electric Machines, Power Electronics and Drives (SDEMPED 2023). MERL Senior Principal Research Scientist Bingnan Wang presented the paper and received the award at the symposium. Co-authors of the paper include Mitsubishi Electric researchers Hiroshi Inoue and Makoto Kanemaru.

      SDEMPED was established as the only international symposium entirely devoted to the diagnostics of electrical machines, power electronics and drives. It is now a regular biennial event. The 14th version, SDEMPED 2023 was held in Chania, Greece from August 28th to 31st, 2023.
  •  
  •  EVENT    MERL's Annual Women in Science Luncheon
    Date: Friday, August 4, 2023
    Location: MERL's Offices, 201 Broadway, Cambridge, MA
    Speaker: Carole-Jean Wu, PhD, Meta AI / Fair
    MERL Contacts: Elizabeth Phillips; Anthony Vetro
    Brief
    • MERL hosted its annual Women in Science luncheon. Carole-Jean Wu, PhD, joined our event to lead a talk on Scaling AI Computing Sustainably. She shared key challenges across the many dimensions of AI, on what and how at-scale optimization can help reduce the overall carbon footprint of AI and computing. Dr. Wu is a Research Scientist and Technical Lead Manager at Meta AI / FAIR. Prior to Meta/Facebook, she was an Associate Professor at ASU.

      As part of this celebration, MERL will be making a donation to Science Club for Girls in Cambridge, MA.

      Science Club for Girls' mission is to foster excitement, confidence, and literacy in science, technology, engineering, and mathematics (STEM) for girls and gender-expansive youth from underrepresented communities by providing free, experiential programs and by maximizing meaningful interactions with women-in-STEM mentors.

      https://www.scienceclubforgirls.org
  •  
  •  NEWS    MERL presents 9 papers at 2023 IFAC World Congress
    Date: July 9, 2023 - July 14, 2023
    MERL Contacts: Karl Berntorp; Scott A. Bortoff; Ankush Chakrabarty; Stefano Di Cairano; Christopher R. Laughman; Diego Romeres; Abraham P. Vinod
    Research Areas: Control, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, Robotics
    Brief
    • MERL researchers presented 9 papers and organized 2 invited/workshop sessions at the 2023 IFAC World Congress held in Yokohama, JP.

      MERL's contributions covered topics including decision-making for autonomous vehicles, statistical and learning-based estimation for GNSS and energy systems, impedance control for delta robots, learning for system identification of rigid body dynamics and time-varying systems, and meta-learning for deep state-space modeling using data from similar systems. The invited session (MERL co-organizer: Ankush Chakrabarty) was on the topic of “Estimation and observer design: theory and applications” and the workshop (MERL co-organizer: Karl Berntorp) was on “Gaussian Process Learning for Systems and Control”.
  •  
  •  NEWS    MERL researchers present 3 papers on Dexterous Manipulation at RSS 23.
    Date: July 11, 2023
    Where: Daegu, Korea
    MERL Contacts: Siddarth Jain; Devesh K. Jha; Arvind Raghunathan
    Research Areas: Artificial Intelligence, Machine Learning, Robotics
    Brief
    • MERL researchers presented 3 papers at the 19th edition of Robotics:Science and Systems Conference in Daegu, Korea. RSS is the flagship conference of the RSS foundation and is run as a single track conference presenting a limited number of high-quality papers. This year the main conference had a total of 112 papers presented. MERL researchers presented 2 papers in the main conference on planning and perception for dexterous manipulation. Another paper was presented in a workshop of learning for dexterous manipulation. More details can be found here https://roboticsconference.org.
  •  
  •  NEWS    Keynote address given by Philip Orlik at 9th annual IEEE Smartcomp conference
    Date: June 26, 2023
    Where: International Conference on Smart Computing (SMARTCOMP), Vanderbilt University, Nashville, Tennessee
    MERL Contact: Philip V. Orlik
    Research Areas: Communications, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Signal Processing
    Brief
    • VP & Research Director, Philip Orlik, gave a keynote titled, "Smart Technologies for Smarter Buildings" at the 9th edition of the IEEE International Conference on Smart Computing (SMARTCOMP) focusing on some of the research challenges and opportunities that arise as we seek to achieve net-zero emissions in Smart building environments.

      SMARTCOMP is the premier conference on smart computing. Smart computing is a multidisciplinary domain based on the synergistic influence of advances in sensor-based technologies, Internet of Things, cyber-physical systems, edge computing, big data analytics, machine learning, cognitive computing, and artificial intelligence.
  •  
  •  AWARD    MERL Intern and Researchers Win ICASSP 2023 Best Student Paper Award
    Date: June 9, 2023
    Awarded to: Darius Petermann, Gordon Wichern, Aswin Subramanian, Jonathan Le Roux
    MERL Contacts: Jonathan Le Roux; Gordon Wichern
    Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
    Brief
    • Former MERL intern Darius Petermann (Ph.D. Candidate at Indiana University) has received a Best Student Paper Award at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023) for the paper "Hyperbolic Audio Source Separation", co-authored with MERL researchers Gordon Wichern and Jonathan Le Roux, and former MERL researcher Aswin Subramanian. The paper presents work performed during Darius's internship at MERL in the summer 2022. The paper introduces a framework for audio source separation using embeddings on a hyperbolic manifold that compactly represent the hierarchical relationship between sound sources and time-frequency features. Additionally, the code associated with the paper is publicly available at https://github.com/merlresearch/hyper-unmix.

      ICASSP is the flagship conference of the IEEE Signal Processing Society (SPS). ICASSP 2023 was held in the Greek island of Rhodes from June 04 to June 10, 2023, and it was the largest ICASSP in history, with more than 4000 participants, over 6128 submitted papers and 2709 accepted papers. Darius’s paper was first recognized as one of the Top 3% of all papers accepted at the conference, before receiving one of only 5 Best Student Paper Awards during the closing ceremony.
  •  
  •  AWARD    MERL’s Paper on Wi-Fi Sensing Earns Top 3% Paper Recognition at ICASSP 2023, Selected as a Best Student Paper Award Finalist
    Date: June 9, 2023
    Awarded to: Cristian J. Vaca-Rubio, Pu Wang, Toshiaki Koike-Akino, Ye Wang, Petros Boufounos and Petar Popovski
    MERL Contacts: Petros T. Boufounos; Toshiaki Koike-Akino; Pu (Perry) Wang; Ye Wang
    Research Areas: Artificial Intelligence, Communications, Computational Sensing, Dynamical Systems, Machine Learning, Signal Processing
    Brief
    • A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.

      Performed during Cristian’s stay at MERL first as a visiting Marie Skłodowska-Curie Fellow and then as a full-time intern in 2022, this work capitalizes on standards-compliant Wi-Fi signals to perform indoor localization and sensing. The paper uses a neural dynamic learning framework to address technical issues such as low sampling rate and irregular sampling intervals.

      ICASSP, a flagship conference of the IEEE Signal Processing Society (SPS), was hosted on the Greek island of Rhodes from June 04 to June 10, 2023. ICASSP 2023 marked the largest ICASSP in history, boasting over 4000 participants and 6128 submitted papers, out of which 2709 were accepted.
  •  
  •  AWARD    Joint CMU-MERL team wins DCASE2023 Challenge on Automated Audio Captioning
    Date: June 1, 2023
    Awarded to: Shih-Lun Wu, Xuankai Chang, Gordon Wichern, Jee-weon Jung, Francois Germain, Jonathan Le Roux, Shinji Watanabe
    MERL Contacts: François Germain; Jonathan Le Roux; Gordon Wichern
    Research Areas: Artificial Intelligence, Machine Learning, Speech & Audio
    Brief
    • A joint team consisting of members of CMU Professor and MERL Alumn Shinji Watanabe's WavLab and members of MERL's Speech & Audio team ranked 1st out of 11 teams in the DCASE2023 Challenge's Task 6A "Automated Audio Captioning". The team was led by student Shih-Lun Wu and also featured Ph.D. candidate Xuankai Chang, Postdoctoral research associate Jee-weon Jung, Prof. Shinji Watanabe, and MERL researchers Gordon Wichern, Francois Germain, and Jonathan Le Roux.

      The IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE Challenge), started in 2013, has been organized yearly since 2016, and gathers challenges on multiple tasks related to the detection, analysis, and generation of sound events. This year, the DCASE2023 Challenge received over 428 submissions from 123 teams across seven tasks.

      The CMU-MERL team competed in the Task 6A track, Automated Audio Captioning, which aims at generating informative descriptions for various sounds from nature and/or human activities. The team's system made strong use of large pretrained models, namely a BEATs transformer as part of the audio encoder stack, an Instructor Transformer encoding ground-truth captions to derive an audio-text contrastive loss on the audio encoder, and ChatGPT to produce caption mix-ups (i.e., grammatical and compact combinations of two captions) which, together with the corresponding audio mixtures, increase not only the amount but also the complexity and diversity of the training data. The team's best submission obtained a SPIDEr-FL score of 0.327 on the hidden test set, largely outperforming the 2nd best team's 0.315.
  •  
  •  NEWS    MERL researchers presenting four papers and co-organizing a workshop at CVPR 2023
    Date: June 18, 2023 - June 22, 2023
    Where: Vancouver/Canada
    MERL Contacts: Anoop Cherian; Michael J. Jones; Suhas Lohit; Kuan-Chuan Peng
    Research Areas: Artificial Intelligence, Computer Vision, Machine Learning
    Brief
    • MERL researchers are presenting 4 papers and co-organizing a workshop at the CVPR 2023 conference, which will be held in Vancouver, Canada June 18-22. CVPR is one of the most prestigious and competitive international conferences in computer vision. Details are provided below.

      1. “Are Deep Neural Networks SMARTer than Second Graders,” by Anoop Cherian, Kuan-Chuan Peng, Suhas Lohit, Kevin Smith, and Joshua B. Tenenbaum

      We present SMART: a Simple Multimodal Algorithmic Reasoning Task and the associated SMART-101 dataset for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed for children in the 6-8 age group. Our experiments using SMART-101 reveal that powerful deep models are not better than random accuracy when analyzed for generalization. We also evaluate large language models (including ChatGPT) on a subset of SMART-101 and find that while these models show convincing reasoning abilities, their answers are often incorrect.

      Paper: https://arxiv.org/abs/2212.09993

      2. “EVAL: Explainable Video Anomaly Localization,” by Ashish Singh, Michael J. Jones, and Erik Learned-Miller

      This work presents a method for detecting unusual activities in videos by building a high-level model of activities found in nominal videos of a scene. The high-level features used in the model are human understandable and include attributes such as the object class and the directions and speeds of motion. Such high-level features allow our method to not only detect anomalous activity but also to provide explanations for why it is anomalous.

      Paper: https://arxiv.org/abs/2212.07900

      3. "Aligning Step-by-Step Instructional Diagrams to Video Demonstrations," by Jiahao Zhang, Anoop Cherian, Yanbin Liu, Yizhak Ben-Shabat, Cristian Rodriguez, and Stephen Gould

      The rise of do-it-yourself (DIY) videos on the web has made it possible even for an unskilled person (or a skilled robot) to imitate and follow instructions to complete complex real world tasks. In this paper, we consider the novel problem of aligning instruction steps that are depicted as assembly diagrams (commonly seen in Ikea assembly manuals) with video segments from in-the-wild videos. We present a new dataset: Ikea Assembly in the Wild (IAW) and propose a contrastive learning framework for aligning instruction diagrams with video clips.

      Paper: https://arxiv.org/pdf/2303.13800.pdf

      4. "HaLP: Hallucinating Latent Positives for Skeleton-Based Self-Supervised Learning of Actions," by Anshul Shah, Aniket Roy, Ketul Shah, Shlok Kumar Mishra, David Jacobs, Anoop Cherian, and Rama Chellappa

      In this work, we propose a new contrastive learning approach to train models for skeleton-based action recognition without labels. Our key contribution is a simple module, HaLP: Hallucinating Latent Positives for contrastive learning. HaLP explores the latent space of poses in suitable directions to generate new positives. Our experiments using HaLP demonstrates strong empirical improvements.

      Paper: https://arxiv.org/abs/2304.00387

      The 4th Workshop on Fair, Data-Efficient, and Trusted Computer Vision

      MERL researcher Kuan-Chuan Peng is co-organizing the fourth Workshop on Fair, Data-Efficient, and Trusted Computer Vision (https://fadetrcv.github.io/2023/) in conjunction with CVPR 2023 on June 18, 2023. This workshop provides a focused venue for discussing and disseminating research in the areas of fairness, bias, and trust in computer vision, as well as adjacent domains such as computational social science and public policy.
  •  
  •  NEWS    Mitsubishi Electric Corporation Press Release Announces Worlds First GaN Power Amplifier Capable of Wideband Operation for 4G, 5G and Beyond 5G/6G.
    Date: June 8, 2023
    MERL Contact: Toshiaki Koike-Akino
    Research Areas: Communications, Electronic and Photonic Devices, Machine Learning, Signal Processing
    Brief
    • Mitsubishi Electric Corporation announced today it has developed what is believed to be the world's first gallium nitride (GaN) power amplifier that achieves a frequency range of 3,400MHz using a single power amplifier, which the company has demonstrated can be used for 4G, 5G and Beyond 5G/6G communication systems operating at different frequencies in a single base station. The amplifier is expected to enable the radio unit (transceiver) to be shared with different communication systems and lead to more power-efficient base stations.

      Mitsubishi Electric Researchers, Toshiaki Koike-Akino and Koon Hoo Teo helped developed the technology and device. Technical details will be presented at the IEEE International Microwave Symposium 2023 this month.

      Please see the link below for the full press release from Mitsubishi Electric.
  •  
  •  NEWS    Abraham Vinod gave an invited talk at the University of California Santa Cruz
    Date: June 8, 2023
    Where: Zoom
    MERL Contact: Abraham P. Vinod
    Research Areas: Artificial Intelligence, Control, Dynamical Systems, Optimization, Robotics
    Brief
    • Abraham Vinod gave an invited talk at the Electrical and Computer Engineering Department, the University of California Santa Cruz, titled "Motion Planning under Constraints and Uncertainty using Data and Reachability". His presentation covered recent work on fast and safe motion planners that can allow for coordination among agents, mitigate uncertainty arising from sensing limitations and simplified models, and tolerate the possibility of failures.
  •  
  •  NEWS    Ankush Chakrabarty co-organized three sessions at the ACC2023, and was nominated for Best Energy Systems Paper.
    Date: June 30, 2023 - June 2, 2023
    Where: San Diego, CA
    MERL Contact: Ankush Chakrabarty
    Research Areas: Applied Physics, Artificial Intelligence, Control, Data Analytics, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, Robotics
    Brief
    • Ankush Chakrabarty (researcher, Multiphysical Systems Team) co-organized and spoke at 3 sessions at the 2023 American Control Conference in San Diego, CA. These include: (1) A tutorial session (w/ Stefano Di Cairano) on "Physics Informed Machine Learning for Modeling and Control": an effort with contributions from multiple academic institutes and US research labs; (2) An invited session on "Energy Efficiency in Smart Buildings and Cities" in which his paper (w/ Chris Laughman) on "Local Search Region Constrained Bayesian Optimization for Performance Optimization of Vapor Compression Systems" was nominated for Best Energy Systems Paper Award; and, (3) A special session on Diversity, Equity, and Inclusion to improve recruitment and retention of underrepresented groups in STEM research.
  •