Computational Sensing

Utilizing computation to improve sensing capabilities.

Our research in the area of computational sensing focuses on signal acquisition and design, signal modeling and reconstruction algorithms, including inverse problems, as well as array signal processing techniques.

  • Quick Links

  • Researchers

  • Awards

    •  AWARD    MERL’s Paper on Wi-Fi Sensing Earns Top 3% Paper Recognition at ICASSP 2023, Selected as a Best Student Paper Award Finalist
      Date: June 9, 2023
      Awarded to: Cristian J. Vaca-Rubio, Pu Wang, Toshiaki Koike-Akino, Ye Wang, Petros Boufounos and Petar Popovski
      MERL Contacts: Petros T. Boufounos; Toshiaki Koike-Akino; Pu (Perry) Wang; Ye Wang
      Research Areas: Artificial Intelligence, Communications, Computational Sensing, Dynamical Systems, Machine Learning, Signal Processing
      Brief
      • A MERL Paper on Wi-Fi sensing was recognized as a Top 3% Paper among all 2709 accepted papers at the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023). Co-authored by Cristian Vaca-Rubio and Petar Popovski from Aalborg University, Denmark, and MERL researchers Pu Wang, Toshiaki Koike-Akino, Ye Wang, and Petros Boufounos, the paper "MmWave Wi-Fi Trajectory Estimation with Continous-Time Neural Dynamic Learning" was also a Best Student Paper Award finalist.

        Performed during Cristian’s stay at MERL first as a visiting Marie Skłodowska-Curie Fellow and then as a full-time intern in 2022, this work capitalizes on standards-compliant Wi-Fi signals to perform indoor localization and sensing. The paper uses a neural dynamic learning framework to address technical issues such as low sampling rate and irregular sampling intervals.

        ICASSP, a flagship conference of the IEEE Signal Processing Society (SPS), was hosted on the Greek island of Rhodes from June 04 to June 10, 2023. ICASSP 2023 marked the largest ICASSP in history, boasting over 4000 participants and 6128 submitted papers, out of which 2709 were accepted.
    •  
    •  AWARD    Joshua Rapp wins Best Dissertation Award from the IEEE Signal Processing Society
      Date: December 20, 2021
      Awarded to: Joshua Rapp
      MERL Contact: Joshua Rapp
      Research Areas: Computational Sensing, Signal Processing
      Brief
      • Joshua Rapp has won the 2021 Best PhD Dissertation Award from the IEEE Signal Processing Society.
        The award recognizes a PhD thesis completed on a signal processing subject within the past three years for its relevant work in signal processing while stimulating further research in the field.

        Dr. Rapp completed his PhD at Boston University in 2020 with a thesis entitled "Probabilistic Modeling for Single-Photon Lidar." The dissertation tackles challenges of the acquisition and processing of 3D depth maps reconstructed from time-of-flight data captured one photon at a time.
        The award will be presented at the 2022 IEEE International Conference on Image Processing (ICIP) in France.
    •  
    •  AWARD    Petros Boufounos Elevated to IEEE Fellow
      Date: January 1, 2022
      Awarded to: Petros T. Boufounos
      MERL Contact: Petros T. Boufounos
      Research Areas: Computational Sensing, Signal Processing
      Brief
      • MERL’s Petros Boufounos has been elevated to IEEE Fellow, effective January 2022, for “contributions to compressed sensing.”

        IEEE Fellow is the highest grade of membership of the IEEE. It honors members with an outstanding record of technical achievements, contributing importantly to the advancement or application of engineering, science and technology, and bringing significant value to society. Each year, following a rigorous evaluation procedure, the IEEE Fellow Committee recommends a select group of recipients for elevation to IEEE Fellow. Less than 0.1% of voting members are selected annually for this member grade elevation.
    •  

    See All Awards for Computational Sensing

  • News & Events

    •  TALK    [MERL Seminar Series 2025] David Lindell presents talk titled Imaging Dynamic Scenes from Seconds to Picoseconds
      Date & Time: Wednesday, January 29, 2025; 1:00 PM
      Speaker: David Lindell, University of Toronto
      MERL Host: Joshua Rapp
      Research Areas: Computational Sensing, Computer Vision, Signal Processing
      Abstract
      • The observed timescales of the universe span from the exasecond scale (~1e18 seconds) down to the zeptosecond scale (~1e-21 seconds). While specialized imaging systems can capture narrow slices of this temporal spectrum in the ultra-fast regime (e.g., nanoseconds to picoseconds; 1e-9 to 1e-12 s), they cannot simultaneously capture both slow (> 1 second) and ultra-fast events (< 1 nanosecond). Further, ultra-fast imaging systems are conventionally limited to single-viewpoint capture, hindering 3D visualization at ultra-fast timescales. In this talk, I discuss (1) new computational algorithms that turn a single-photon detector into an "ultra-wideband" imaging system that captures events from seconds to picoseconds; and (2) a method for neural rendering using multi-viewpoint, ultra-fast videos captured using single-photon detectors. The latter approach enables rendering videos of propagating light from novel viewpoints, observation of viewpoint-dependent changes in light transport predicted by Einstein, recovery of material properties, and accurate 3D reconstruction from multiply scattered light. Finally, I discuss future directions in ultra-wideband imaging.
    •  
    •  NEWS    MERL Researchers to Present 2 Conference and 11 Workshop Papers at NeurIPS 2024
      Date: December 10, 2024 - December 15, 2024
      Where: Advances in Neural Processing Systems (NeurIPS)
      MERL Contacts: Petros T. Boufounos; Matthew Brand; Ankush Chakrabarty; Anoop Cherian; François Germain; Toshiaki Koike-Akino; Christopher R. Laughman; Jonathan Le Roux; Jing Liu; Suhas Lohit; Tim K. Marks; Yoshiki Masuyama; Kieran Parsons; Kuan-Chuan Peng; Diego Romeres; Pu (Perry) Wang; Ye Wang; Gordon Wichern
      Research Areas: Artificial Intelligence, Communications, Computational Sensing, Computer Vision, Control, Data Analytics, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Optimization, Robotics, Signal Processing, Speech & Audio, Human-Computer Interaction, Information Security
      Brief
      • MERL researchers will attend and present the following papers at the 2024 Advances in Neural Processing Systems (NeurIPS) Conference and Workshops.

        1. "RETR: Multi-View Radar Detection Transformer for Indoor Perception" by Ryoma Yataka (Mitsubishi Electric), Adriano Cardace (Bologna University), Perry Wang (Mitsubishi Electric Research Laboratories), Petros Boufounos (Mitsubishi Electric Research Laboratories), Ryuhei Takahashi (Mitsubishi Electric). Main Conference. https://neurips.cc/virtual/2024/poster/95530

        2. "Evaluating Large Vision-and-Language Models on Children's Mathematical Olympiads" by Anoop Cherian (Mitsubishi Electric Research Laboratories), Kuan-Chuan Peng (Mitsubishi Electric Research Laboratories), Suhas Lohit (Mitsubishi Electric Research Laboratories), Joanna Matthiesen (Math Kangaroo USA), Kevin Smith (Massachusetts Institute of Technology), Josh Tenenbaum (Massachusetts Institute of Technology). Main Conference, Datasets and Benchmarks track. https://neurips.cc/virtual/2024/poster/97639

        3. "Probabilistic Forecasting for Building Energy Systems: Are Time-Series Foundation Models The Answer?" by Young-Jin Park (Massachusetts Institute of Technology), Jing Liu (Mitsubishi Electric Research Laboratories), François G Germain (Mitsubishi Electric Research Laboratories), Ye Wang (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Gordon Wichern (Mitsubishi Electric Research Laboratories), Navid Azizan (Massachusetts Institute of Technology), Christopher R. Laughman (Mitsubishi Electric Research Laboratories), Ankush Chakrabarty (Mitsubishi Electric Research Laboratories). Time Series in the Age of Large Models Workshop.

        4. "Forget to Flourish: Leveraging Model-Unlearning on Pretrained Language Models for Privacy Leakage" by Md Rafi Ur Rashid (Penn State University), Jing Liu (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Shagufta Mehnaz (Penn State University), Ye Wang (Mitsubishi Electric Research Laboratories). Workshop on Red Teaming GenAI: What Can We Learn from Adversaries?

        5. "Spatially-Aware Losses for Enhanced Neural Acoustic Fields" by Christopher Ick (New York University), Gordon Wichern (Mitsubishi Electric Research Laboratories), Yoshiki Masuyama (Mitsubishi Electric Research Laboratories), François G Germain (Mitsubishi Electric Research Laboratories), Jonathan Le Roux (Mitsubishi Electric Research Laboratories). Audio Imagination Workshop.

        6. "FV-NeRV: Neural Compression for Free Viewpoint Videos" by Sorachi Kato (Osaka University), Takuya Fujihashi (Osaka University), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Takashi Watanabe (Osaka University). Machine Learning and Compression Workshop.

        7. "GPT Sonography: Hand Gesture Decoding from Forearm Ultrasound Images via VLM" by Keshav Bimbraw (Worcester Polytechnic Institute), Ye Wang (Mitsubishi Electric Research Laboratories), Jing Liu (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories). AIM-FM: Advancements In Medical Foundation Models: Explainability, Robustness, Security, and Beyond Workshop.

        8. "Smoothed Embeddings for Robust Language Models" by Hase Ryo (Mitsubishi Electric), Md Rafi Ur Rashid (Penn State University), Ashley Lewis (Ohio State University), Jing Liu (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Kieran Parsons (Mitsubishi Electric Research Laboratories), Ye Wang (Mitsubishi Electric Research Laboratories). Safe Generative AI Workshop.

        9. "Slaying the HyDRA: Parameter-Efficient Hyper Networks with Low-Displacement Rank Adaptation" by Xiangyu Chen (University of Kansas), Ye Wang (Mitsubishi Electric Research Laboratories), Matthew Brand (Mitsubishi Electric Research Laboratories), Pu Wang (Mitsubishi Electric Research Laboratories), Jing Liu (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories). Workshop on Adaptive Foundation Models.

        10. "Preference-based Multi-Objective Bayesian Optimization with Gradients" by Joshua Hang Sai Ip (University of California Berkeley), Ankush Chakrabarty (Mitsubishi Electric Research Laboratories), Ali Mesbah (University of California Berkeley), Diego Romeres (Mitsubishi Electric Research Laboratories). Workshop on Bayesian Decision-Making and Uncertainty. Lightning talk spotlight.

        11. "TR-BEACON: Shedding Light on Efficient Behavior Discovery in High-Dimensions with Trust-Region-based Bayesian Novelty Search" by Wei-Ting Tang (Ohio State University), Ankush Chakrabarty (Mitsubishi Electric Research Laboratories), Joel A. Paulson (Ohio State University). Workshop on Bayesian Decision-Making and Uncertainty.

        12. "MEL-PETs Joint-Context Attack for the NeurIPS 2024 LLM Privacy Challenge Red Team Track" by Ye Wang (Mitsubishi Electric Research Laboratories), Tsunato Nakai (Mitsubishi Electric), Jing Liu (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Kento Oonishi (Mitsubishi Electric), Takuya Higashi (Mitsubishi Electric). LLM Privacy Challenge. Special Award for Practical Attack.

        13. "MEL-PETs Defense for the NeurIPS 2024 LLM Privacy Challenge Blue Team Track" by Jing Liu (Mitsubishi Electric Research Laboratories), Ye Wang (Mitsubishi Electric Research Laboratories), Toshiaki Koike-Akino (Mitsubishi Electric Research Laboratories), Tsunato Nakai (Mitsubishi Electric), Kento Oonishi (Mitsubishi Electric), Takuya Higashi (Mitsubishi Electric). LLM Privacy Challenge. Won 3rd Place Award.

        MERL members also contributed to the organization of the Multimodal Algorithmic Reasoning (MAR) Workshop (https://marworkshop.github.io/neurips24/). Organizers: Anoop Cherian (Mitsubishi Electric Research Laboratories), Kuan-Chuan Peng (Mitsubishi Electric Research Laboratories), Suhas Lohit (Mitsubishi Electric Research Laboratories), Honglu Zhou (Salesforce Research), Kevin Smith (Massachusetts Institute of Technology), Tim K. Marks (Mitsubishi Electric Research Laboratories), Juan Carlos Niebles (Salesforce AI Research), Petar Veličković (Google DeepMind).
    •  

    See All News & Events for Computational Sensing

  • Research Highlights

  • Internships

    • ST0096: Internship - Multimodal Tracking and Imaging

      MERL is seeking a motivated intern to assist in developing hardware and algorithms for multimodal imaging applications. The project involves integration of radar, camera, and depth sensors in a variety of sensing scenarios. The ideal candidate should have experience with FMCW radar and/or depth sensing, and be fluent in Python and scripting methods. Familiarity with optical tracking of humans and experience with hardware prototyping is desired. Good knowledge of computational imaging and/or radar imaging methods is a plus.

      Required Specific Experience

      • Experience with Python and Python Deep Learning Frameworks.
      • Experience with FMCW radar and/or Depth Sensors.

    • ST0126: Internship - Particle-Efficient Interacting Particle Systems for Inverse Problems

      The Computational Sensing Team at MERL is seeking an intern to work with MERL researchers on algorithms based on interacting particle systems for solving inverse problems. The focus of the project is particle-efficiency and applicability to non-log-concave posterior distributions (which may result from nonlinear forward operators). The project includes algorithm design, (finite-particle) convergence analysis, and/or empirical evaluation for challenging inverse problems such as full waveform inversion. The ideal candidate would be a PhD student with a solid background in applied probability, nonconvex optimization, or Bayesian sampling. Programming skills in Python or MATLAB are required. The duration is anticipated to be at least 3 months with a flexible start date.

    • ST0141: Internship - Uncertainty Quantification in Computational Physics

      The Computational Sensing team at MERL is seeking a highly motivated PhD student for an internship focused on uncertainty quantification (UQ) in computational modeling of physical systems. The goal of this project is to advance the methodology and practice of UQ, with a focus on reduced-order stochastic modeling and optimal sensor placement for Bayesian inverse problems. The research will draw upon foundational ideas and techniques in applied mathematics and statistics for applications in wave propagation, fluid dynamics, and more generally high-dimensional systems. The ideal candidate will be a PhD student in engineering, applied mathematics, computer science, or related fields with a solid background and publication record in any of the following areas: stochastic modeling, dimensionality reduction, Bayesian inference, optimal experimental design, and tensor methods. Programming skills in Python or MATLAB are required. Publication of the results obtained during the internship is expected. The duration is anticipated to be at least 3 months with a flexible start date.


    See All Internships for Computational Sensing

  • Openings


    See All Openings at MERL

  • Recent Publications

    •  Rapp, J., Mansour, H., Boufounos, P.T., Koike-Akino, T., Parsons, K., "ulti-layered Surface Estimation for Low-cost Optical Coherence Tomography", IEEE Transactions on Computational Imaging, DOI: 10.1109/​TCI.2024.3497602, Vol. 10, pp. 1706-1721, December 2024.
      BibTeX TR2024-164 PDF
      • @article{Rapp2024dec,
      • author = {Rapp, Joshua and Mansour, Hassan and Boufounos, Petros T. and Koike-Akino, Toshiaki and Parsons, Kieran}},
      • title = {ulti-layered Surface Estimation for Low-cost Optical Coherence Tomography},
      • journal = {IEEE Transactions on Computational Imaging},
      • year = 2024,
      • volume = 10,
      • pages = {1706--1721},
      • month = dec,
      • doi = {10.1109/TCI.2024.3497602},
      • url = {https://www.merl.com/publications/TR2024-164}
      • }
    •  Yataka, R., Cardace, A., Wang, P., Boufounos, P.T., Takahashi, R., "RETR: Multi-View Radar Detection Transformer for Indoor Perception", Advances in Neural Information Processing Systems (NeurIPS), A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang, Eds., November 2024, pp. 19839-19869.
      BibTeX TR2024-159 PDF Software
      • @inproceedings{Yataka2024nov3,
      • author = {Yataka, Ryoma and Cardace, Adriano and Wang, Pu and Boufounos, Petros T. and Takahashi, Ryuhei}},
      • title = {RETR: Multi-View Radar Detection Transformer for Indoor Perception},
      • booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
      • year = 2024,
      • editor = {A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang},
      • pages = {19839--19869},
      • month = nov,
      • publisher = {Curran Associates, Inc.},
      • url = {https://www.merl.com/publications/TR2024-159}
      • }
    •  Sholokhov, A., Nabi, S., Rapp, J., Brunton, S., Kutz, N., Boufounos, P.T., Mansour, H., "Single-pixel imaging of spatio-temporal flows using differentiable latent dynamics", IEEE Transactions on Computational Imaging, DOI: 10.1109/​TCI.2024.3434541, Vol. 10, pp. 1124-1138, October 2024.
      BibTeX TR2024-151 PDF
      • @article{Sholokhov2024oct,
      • author = {{Sholokhov, Aleksei and Nabi, Saleh and Rapp, Joshua and Brunton, Steven and Kutz, Nathan and Boufounos, Petros T. and Mansour, Hassan}},
      • title = {Single-pixel imaging of spatio-temporal flows using differentiable latent dynamics},
      • journal = {IEEE Transactions on Computational Imaging},
      • year = 2024,
      • volume = 10,
      • pages = {1124--1138},
      • month = oct,
      • doi = {10.1109/TCI.2024.3434541},
      • url = {https://www.merl.com/publications/TR2024-151}
      • }
    •  Jin, S., Wang, P., Boufounos, P.T., Orlik, P.V., Takahashi, R., Roy, S., "Spatial-Domain Mutual Interference Mitigation for MIMO-FMCW Automotive Radar", IEEE Transactions on Vehicular Technology, DOI: 10.1109/​TVT.2024.3467917, September 2024.
      BibTeX TR2024-148 PDF
      • @article{Jin2024sep,
      • author = {Jin, Sian and Wang, Pu and Boufounos, Petros T. and Orlik, Philip V. and Takahashi, Ryuhei and Roy, Sumit}},
      • title = {Spatial-Domain Mutual Interference Mitigation for MIMO-FMCW Automotive Radar},
      • journal = {IEEE Transactions on Vehicular Technology},
      • year = 2024,
      • month = sep,
      • doi = {10.1109/TVT.2024.3467917},
      • issn = {1939-9359},
      • url = {https://www.merl.com/publications/TR2024-148}
      • }
    •  Rahman, M., Yataka, R., Kato, S., Wang, P., Li, P., Cardace, A., Boufounos, P.T., "MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception", European Conference on Computer Vision (ECCV), DOI: 10.1007/​978-3-031-72986-7_18, September 2024, pp. 306–322.
      BibTeX TR2024-117 PDF Data
      • @inproceedings{Rahman2024sep,
      • author = {Rahman, Mahbub and Yataka, Ryoma and Kato, Sorachi and Wang, Pu and Li, Peizhao and Cardace, Adriano and Boufounos, Petros T.}},
      • title = {MMVR: Millimeter-wave Multi-View Radar Dataset and Benchmark for Indoor Perception},
      • booktitle = {European Conference on Computer Vision (ECCV)},
      • year = 2024,
      • pages = {306–322},
      • month = sep,
      • publisher = {Springer},
      • doi = {10.1007/978-3-031-72986-7_18},
      • url = {https://www.merl.com/publications/TR2024-117}
      • }
    •  Shastri, S., Ma, Y., Boufounos, P.T., Mansour, H., "Deep Calibration and Operator Learning for Ground Penetrating Radar Imaging", European Signal Processing Conference (EUSIPCO), August 2024.
      BibTeX TR2024-128 PDF
      • @inproceedings{Shastri2024aug,
      • author = {Shastri, Saurav and Ma, Yanting and Boufounos, Petros T. and Mansour, Hassan}},
      • title = {Deep Calibration and Operator Learning for Ground Penetrating Radar Imaging},
      • booktitle = {European Signal Processing Conference (EUSIPCO)},
      • year = 2024,
      • month = aug,
      • url = {https://www.merl.com/publications/TR2024-128}
      • }
    •  Zhang, X., Mao, W., Mowlavi, S., Benosman, M., Basar, T., "Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms", Learning for Dynamics & Control Conference (L4DC), July 2024, pp. 181-196.
      BibTeX TR2024-098 PDF
      • @inproceedings{Zhang2024jul2,
      • author = {Zhang, Xiangyuan and Mao, Weichao and Mowlavi, Saviz and Benosman, Mouhacine and Basar, Tamer}},
      • title = {Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms},
      • booktitle = {Learning for Dynamics & Control Conference (L4DC)},
      • year = 2024,
      • pages = {181--196},
      • month = jul,
      • publisher = {PMLR},
      • url = {https://www.merl.com/publications/TR2024-098}
      • }
    •  Yataka, R., Wang, P., Boufounos, P.T., Takahashi, R., "SIRA: Scalable Inter-frame Relation and Association for Radar Perception", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2024, pp. 15024-15034.
      BibTeX TR2024-041 PDF Video
      • @inproceedings{Yataka2024jun,
      • author = {Yataka, Ryoma and Wang, Pu and Boufounos, Petros T. and Takahashi, Ryuhei},
      • title = {SIRA: Scalable Inter-frame Relation and Association for Radar Perception},
      • booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      • year = 2024,
      • pages = {15024--15034},
      • month = jun,
      • url = {https://www.merl.com/publications/TR2024-041}
      • }
    See All Publications for Computational Sensing

  • Videos

  • Software & Data Downloads