- Date: June 17, 2024 - June 21, 2024
Where: Seattle, WA
MERL Contacts: Petros T. Boufounos; Moitreya Chatterjee; Anoop Cherian; Michael J. Jones; Toshiaki Koike-Akino; Jonathan Le Roux; Suhas Lohit; Tim K. Marks; Pedro Miraldo; Jing Liu; Kuan-Chuan Peng; Pu (Perry) Wang; Ye Wang; Matthew Brand
Research Areas: Artificial Intelligence, Computational Sensing, Computer Vision, Machine Learning, Speech & Audio
Brief - MERL researchers are presenting 5 conference papers, 3 workshop papers, and are co-organizing two workshops at the CVPR 2024 conference, which will be held in Seattle, June 17-21. CVPR is one of the most prestigious and competitive international conferences in computer vision. Details of MERL contributions are provided below.
CVPR Conference Papers:
1. "TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models" by H. Ni, B. Egger, S. Lohit, A. Cherian, Y. Wang, T. Koike-Akino, S. X. Huang, and T. K. Marks
This work enables a pretrained text-to-video (T2V) diffusion model to be additionally conditioned on an input image (first video frame), yielding a text+image to video (TI2V) model. Other than using the pretrained T2V model, our method requires no ("zero") training or fine-tuning. The paper uses a "repeat-and-slide" method and diffusion resampling to synthesize videos from a given starting image and text describing the video content.
Paper: https://www.merl.com/publications/TR2024-059
Project page: https://merl.com/research/highlights/TI2V-Zero
2. "Long-Tailed Anomaly Detection with Learnable Class Names" by C.-H. Ho, K.-C. Peng, and N. Vasconcelos
This work aims to identify defects across various classes without relying on hard-coded class names. We introduce the concept of long-tailed anomaly detection, addressing challenges like class imbalance and dataset variability. Our proposed method combines reconstruction and semantic modules, learning pseudo-class names and utilizing a variational autoencoder for feature synthesis to improve performance in long-tailed datasets, outperforming existing methods in experiments.
Paper: https://www.merl.com/publications/TR2024-040
3. "Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling" by X. Liu, Y-W. Tai, C-T. Tang, P. Miraldo, S. Lohit, and M. Chatterjee
This work presents a new strategy for rendering dynamic scenes from novel viewpoints. Our approach is based on stratifying the scene into regions based on the extent of motion of the region, which is automatically determined. Regions with higher motion are permitted a denser spatio-temporal sampling strategy for more faithful rendering of the scene. Additionally, to the best of our knowledge, ours is the first work to enable tracking of objects in the scene from novel views - based on the preferences of a user, provided by a click.
Paper: https://www.merl.com/publications/TR2024-042
4. "SIRA: Scalable Inter-frame Relation and Association for Radar Perception" by R. Yataka, P. Wang, P. T. Boufounos, and R. Takahashi
Overcoming the limitations on radar feature extraction such as low spatial resolution, multipath reflection, and motion blurs, this paper proposes SIRA (Scalable Inter-frame Relation and Association) for scalable radar perception with two designs: 1) extended temporal relation, generalizing the existing temporal relation layer from two frames to multiple inter-frames with temporally regrouped window attention for scalability; and 2) motion consistency track with a pseudo-tracklet generated from observational data for better object association.
Paper: https://www.merl.com/publications/TR2024-041
5. "RILA: Reflective and Imaginative Language Agent for Zero-Shot Semantic Audio-Visual Navigation" by Z. Yang, J. Liu, P. Chen, A. Cherian, T. K. Marks, J. L. Roux, and C. Gan
We leverage Large Language Models (LLM) for zero-shot semantic audio visual navigation. Specifically, by employing multi-modal models to process sensory data, we instruct an LLM-based planner to actively explore the environment by adaptively evaluating and dismissing inaccurate perceptual descriptions.
Paper: https://www.merl.com/publications/TR2024-043
CVPR Workshop Papers:
1. "CoLa-SDF: Controllable Latent StyleSDF for Disentangled 3D Face Generation" by R. Dey, B. Egger, V. Boddeti, Y. Wang, and T. K. Marks
This paper proposes a new method for generating 3D faces and rendering them to images by combining the controllability of nonlinear 3DMMs with the high fidelity of implicit 3D GANs. Inspired by StyleSDF, our model uses a similar architecture but enforces the latent space to match the interpretable and physical parameters of the nonlinear 3D morphable model MOST-GAN.
Paper: https://www.merl.com/publications/TR2024-045
2. “Tracklet-based Explainable Video Anomaly Localization” by A. Singh, M. J. Jones, and E. Learned-Miller
This paper describes a new method for localizing anomalous activity in video of a scene given sample videos of normal activity from the same scene. The method is based on detecting and tracking objects in the scene and estimating high-level attributes of the objects such as their location, size, short-term trajectory and object class. These high-level attributes can then be used to detect unusual activity as well as to provide a human-understandable explanation for what is unusual about the activity.
Paper: https://www.merl.com/publications/TR2024-057
MERL co-organized workshops:
1. "Multimodal Algorithmic Reasoning Workshop" by A. Cherian, K-C. Peng, S. Lohit, M. Chatterjee, H. Zhou, K. Smith, T. K. Marks, J. Mathissen, and J. Tenenbaum
Workshop link: https://marworkshop.github.io/cvpr24/index.html
2. "The 5th Workshop on Fair, Data-Efficient, and Trusted Computer Vision" by K-C. Peng, et al.
Workshop link: https://fadetrcv.github.io/2024/
3. "SuperLoRA: Parameter-Efficient Unified Adaptation for Large Vision Models" by X. Chen, J. Liu, Y. Wang, P. Wang, M. Brand, G. Wang, and T. Koike-Akino
This paper proposes a generalized framework called SuperLoRA that unifies and extends different variants of low-rank adaptation (LoRA). Introducing new options with grouping, folding, shuffling, projection, and tensor decomposition, SuperLoRA offers high flexibility and demonstrates superior performance up to 10-fold gain in parameter efficiency for transfer learning tasks.
Paper: https://www.merl.com/publications/TR2024-062
-
- Date: December 2, 2022 - December 8, 2022
MERL Contacts: Matthew Brand; Toshiaki Koike-Akino; Jing Liu; Saviz Mowlavi; Kieran Parsons; Ye Wang
Research Areas: Artificial Intelligence, Control, Dynamical Systems, Machine Learning, Signal Processing
Brief - In addition to 5 papers in recent news (https://www.merl.com/news/news-20221129-1450), MERL researchers presented 2 papers at the NeurIPS Conference Workshop, which was held Dec. 2-8. NeurIPS is one of the most prestigious and competitive international conferences in machine learning.
- “Optimal control of PDEs using physics-informed neural networks” by Saviz Mowlavi and Saleh Nabi
Physics-informed neural networks (PINNs) have recently become a popular method for solving forward and inverse problems governed by partial differential equations (PDEs). By incorporating the residual of the PDE into the loss function of a neural network-based surrogate model for the unknown state, PINNs can seamlessly blend measurement data with physical constraints. Here, we extend this framework to PDE-constrained optimal control problems, for which the governing PDE is fully known and the goal is to find a control variable that minimizes a desired cost objective. We validate the performance of the PINN framework by comparing it to state-of-the-art adjoint-based optimization, which performs gradient descent on the discretized control variable while satisfying the discretized PDE.
- “Learning with noisy labels using low-dimensional model trajectory” by Vasu Singla, Shuchin Aeron, Toshiaki Koike-Akino, Matthew E. Brand, Kieran Parsons, Ye Wang
Noisy annotations in real-world datasets pose a challenge for training deep neural networks (DNNs), detrimentally impacting generalization performance as incorrect labels may be memorized. In this work, we probe the observations that early stopping and low-dimensional subspace learning can help address this issue. First, we show that a prior method is sensitive to the early stopping hyper-parameter. Second, we investigate the effectiveness of PCA, for approximating the optimization trajectory under noisy label information. We propose to estimate the low-rank subspace through robust and structured variants of PCA, namely Robust PCA, and Sparse PCA. We find that the subspace estimated through these variants can be less sensitive to early stopping, and can outperform PCA to achieve better test error when trained on noisy labels.
- In addition, new MERL researcher, Jing Liu, also presented a paper entitled “CoPur: Certifiably Robust Collaborative Inference via Feature Purification" based on his previous work before joining MERL. His paper was elected as a spotlight paper to be highlighted in lightening talks and featured paper panel.
-
- Date & Time: Tuesday, February 8, 2022; 1:00 PM EST
Speaker: Raphaël Pestourie, MIT
MERL Host: Matthew Brand
Research Areas: Applied Physics, Electronic and Photonic Devices, Optimization
Abstract 
Thin large-area structures with aperiodic subwavelength patterns can unleash the full power of Maxwell’s equations for focusing light and a variety of other wave transformation or optical applications. Because of their irregularity and large scale, capturing the full scattering through these devices is one of the most challenging tasks for computational design: enter extreme optics! This talk will present ways to harness the full computational power of modern large-scale optimization in order to design optical devices with thousands or millions of free parameters. We exploit various methods of domain-decomposition approximations, supercomputer-scale topology optimization, laptop-scale “surrogate” models based on Chebyshev interpolation and/or new scientific machine learning models, and other techniques to attack challenging problems: achromatic lenses that simultaneously handle many wavelengths and angles, “deep” images, hyperspectral imaging, and more.
-
- Date & Time: Monday, July 11, 2016; 10 AM - 9:15 PM
Location: Westin Boston Waterfront Pavilion, Boston, Massachusetts
MERL Contacts: Matthew Brand; Arvind Raghunathan Brief - MERL researchers participate in SIAM Job fair to showcase MERL's research and highlight employment and intern opportunities at MERL. The Career Fair emphasizes careers in business, industry, and government, and takes place during the SIAM Annual Meeting.
The SIAM Applied Mathematics and Computational Science Career Fair is an informational and interactive event at which employers and prospective employees can discuss careers. It is a great opportunity for prospective employees to meet government and industry representatives and discuss what they are looking for and what each employer has to offer.
-
- Date: February 13, 2014
MERL Contact: Matthew Brand Brief - Mitsubishi Electric Corporation announced its development of advanced optimization algorithms and high-speed calculation methods aimed at optimizing the performance of three practical systems: laser-processing machines for high-speed cutting of sheet metal using the shortest possible trajectories, moon probes achieved with minimized fuel consumption, and particle beam therapies for prompt medical treatments.
-
- Date & Time: Wednesday, November 6, 2013; 1:00 PM - 3:00 PM
Location: MIT, Kiva room @CSAIL
MERL Contact: Matthew Brand Brief - Every year, MIT neighbor Mitsubishi Electric Research Labs hires 40-60 talented and motivated graduate students for summer internships in CS/EE-oriented research projects, aimed at real-world impact and high-quality publications. Join us for our kick-off recruiting reception at the Kiva/Patel room Wednesday 1-3pm. There will be a short overview of current research areas, confections, and MERL researchers on hand to discuss research opportunities. .
-
- Date: July 9, 2013
Where: International Journal of Control
MERL Contacts: Stefano Di Cairano; Matthew Brand; Scott A. Bortoff
Research Area: Control
Brief - The article "Projection-free Parallel Quadratic Programming for Linear Model predictive Control" by Di Cairano, S., Brand, M. and Bortoff, S.A. was published in International Journal of Control.
-
- Date & Time: Thursday, December 1, 2011; 11:00 AM
Speaker: Gregg Favalora, Optics for Hire (OFH)
MERL Host: Matthew Brand Abstract - I'll give an information-rich survey presentation on "interesting and unusual" forms of autostereo display. It will assume basic knowledge of autostereo, e.g. lenticular and parallax barrier displays [unless, of course, you'd like a few minutes going over the basics.] I will discuss: spatially-multiplexed, time-multiplexed, and multi-projector systems. This includes: non-obvious depth cues, advances in parallax barrier displays, lenticulars, multi-projector / projection onto corrugated screens, scanned illumination, volumetric, and electro-holographic techniques.
-
- Date: September 11, 2011
Where: IEEE International Conference on Image Processing (ICIP)
MERL Contacts: Matthew Brand; Anthony Vetro; Petros T. Boufounos Brief - The papers "Distributed Compression of Zerotrees of Wavelet Coefficients" by Wang, Y., Rane, S., Boufounos, P. and Vetro, A., "A Trellis-based Approach for Robust View Synthesis" by Tian, D., Vetro, A. and Brand, M., "Concentric Ring Signature Descriptor for 3D Objects" by Nguyen, H.V. and Porikli, F. and "Parallel Quadratic Programming for Image Processing" by Brand, M. and Chen, D. were presented at the IEEE International Conference on Image Processing (ICIP).
-
- Date: August 28, 2011
Where: World Congress of the International Federation of Automatic Control (IFAC)
MERL Contacts: Scott A. Bortoff; Matthew Brand Brief - The papers "Integrated Design and Control of Flexure-Based Nanopositioning Systems - Part I: Methodology" by Shilpiekandula, V. and Youcel-Toumi, K. and "A Parallel Quadratic Programming Algorithm for Model Predictive Control" by Brand, M., Shilpiekandula, V. and Bortoff, S.A. were presented at the World Congress of the International Federation of Automatic Control (IFAC).
-
- Date: October 18, 2010
Where: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
MERL Contact: Matthew Brand Brief - The paper "SKYLINE2GPS: Localization in Urban Canyons Using Omni-skylines" by Ramalingam, S., Bouaziz, S., Sturm, P. and Brand, M. was presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
-
- Date: September 27, 2009
Where: IEEE International Conference on Computer Vision Workshops (ICCV)
MERL Contact: Matthew Brand Brief - The paper "Geolocalization using Skylines from Omni-Images" by Ramalingam, S., Bouaziz, S., Sturm, P. and Brand, M.E. was presented at the IEEE International Conference on Computer Vision Workshops (ICCV).
-
- Date: July 7, 2009
Where: International Conference on Image Analysis and Recognition (ICIAR)
MERL Contact: Matthew Brand Brief - The paper "Image and Video Retargetting by Darting" by Brand, M.E. was presented at the International Conference on Image Analysis and Recognition (ICIAR).
-
- Date: November 30, 2008
Where: IEEE Global Telecommunications Conference (GLOBECOM)
MERL Contacts: Matthew Brand; Jinyun Zhang Brief - The papers "Routing with Probabilistic Delay Guarantees in Wireless Ad-Hoc Networks" by Brand, M., Maymounkov, P. and Molisch, A.F., "Delay-Energy Tradeoffs in Wireless Ad-Hoc Networks with Partial Channel State Information" by Brand, M. and Molisch, A.F., "Low-Complexity Hybrid QRD-MCMC MIMO Detection" by Peng, R., Teo, K.H., Zhang, J. and Chen, R.-R., "Adaptive Soft Frequency Reuse for Inter-cell Interference Coordination in SC-FDMA based 3GPP LTE Uplinks" by Mao, X., Maaref, A. and Teo, K.H., "Impact of Mobility on the Behavior of Interference in Cellular Wireless Networks" by Yarkan, S., Maaref, A. and Teo, K.H. and "Modified Beacon-Enabled IEEE 802.15.4 MAC for Lower Latency" by Bhatti, G., Mehta, A., Sahinoglu, Z., Zhang, J. and Viswanathan, R. were presented at the IEEE Global Telecommunications Conference (GLOBECOM).
-
- Date: June 28, 2008
Where: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
MERL Contacts: Matthew Brand; Anthony Vetro Brief - The papers "Non-Refractive Modulators for Encoding and Capturing Scene Appearance and Depth" by Veeraraghavan, A., Agrawal, A., Raskar, R., Mohan, A. and Tumblin, J., "Feature Transformation of Biometric Templates for Secure Biometric Systems based on Error Correcting Codes" by Sutcu, Y., Rane, S., Yedidia, J.S., Draper, S.C. and Vetro, A., "Constant Time O(1) Bilateral Filtering" by Porikli, F., "Learning on Lie Groups for Invariant Detection and Tracking" by Tuzel, O., Porikli, F. and Meer, P., "Kernel Integral Images: A Framework for Fast non-Uniform Filtering" by Hussein, M., Porikli, F. and Davis, L., "A Conditional Random Field for Automatic Photo Editing" by Brand, M. and Pletscher, P., "Boosting Adaptive Linear Weak Classifiers for Online Learning and Tracking" by Parag, T., Porikli, F. and Elgammai, A. and "Sensing Increased Image Resolution Using Aperture Masks" by Mohan, A., Huang, X., Tumblin, J. and Raskar, R. were presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
-
- Date: September 1, 2006
Where: European Symposium on Algorithms (ESA)
MERL Contact: Matthew Brand Brief - The paper "Stochastic Shortest Paths Via Quasi-convex Maximization" by Nikolova, E., Kelner, J., Brand, M. and Mitzenmacher, M. was presented at the European Symposium on Algorithms (ESA).
-
- Date: June 6, 2006
Where: International Conference on Automated Planning and Scheduling (ICAPS)
MERL Contact: Matthew Brand Brief - The paper "Optimal Route Planning under Uncertainty" by Nikolova, E., Brand, M. and Karger, D.R. was presented at the International Conference on Automated Planning and Scheduling (ICAPS).
-
- Date: May 1, 2006
Where: Linear Algebra and Its Applications
MERL Contact: Matthew Brand Brief - The article "Fast Low-Rank Modifications of the Thin Singular Value Decomposition" by Brand, M. was published in Linear Algebra and Its Applications.
-
- Date: October 3, 2005
Where: European Conference on Machine Learning (ECML)
MERL Contact: Matthew Brand Brief - The paper "Nonrigid Embeddings for Dimensionality Reduction" by Brand, M. was presented at the European Conference on Machine Learning (ECML).
-
- Date: July 31, 2005
Where: ACM Transactions on Graphics (TOG)
MERL Contact: Matthew Brand Brief - The articles "Face Transfer with Multilinear Models" by Vlasic, D., Brand, M., Pfister, H. and Popovic, J. and "Removing Photography Artifacts Using Gradient Projection and Flash-Exposure Sampling" by Agrawal, A., Raskar, R., Nayar, S.K. and Li, Y. were published in ACM Transactions on Graphics (TOG).
-
- Date: June 20, 2005
Where: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
MERL Contact: Matthew Brand Brief - The papers "A Direct Method for 3D Factorization of Nonrigid Motion Observed in 2D" by Brand, M., "Integral Histogram: A Fast Way to Extract Histograms in Cartesian Spaces" by Porikli, F., "Why I Want a Gradient Camera" by Tumblin, J., Agrawal, A. and Raskar, R., "Videoshop: A New Framework for Spatio-Temporal Video Editing in Gradient Domain" by Wang, H., Xu, N., Raskar, R. and Ahuja, N. and "Handheld Projectors for Mixing Physical and Digital Textures" by Beardsley, P., Forlines, C., Raskar, R. and van Baar, J. were presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
-
- Date: May 15, 2005
Where: SIAM Conference on Optimization
MERL Contact: Matthew Brand Brief - The paper "A Random Walks Perspective on Maximizing Satisfaction and Profit" by Brand, M. was presented at the SIAM Conference on Optimization.
-
- Date: October 31, 2004
Where: IEEE Transactions on Automatic Control
MERL Contacts: Matthew Brand; Daniel N. Nikovski
Research Area: Optimization
Brief - The article "Exact Calculation of Expected Waiting Times for Group Elevator Control" by Nikovski, D. and Brand, M. was published in IEEE Transactions on Automatic Control.
-
- Date: October 18, 2004
Where: World Congress and Exhibition on Intelligent Transport Systems and Services (ITS)
MERL Contacts: Matthew Brand; Daniel N. Nikovski
Research Area: Data Analytics
Brief - The paper "Traffic Prediction using Singular Value Decomposition" by Nishiuma, N., Goto, Y., Kumazawa, H., Nikovski, D. and Brand, M. was presented at the World Congress and Exhibition on Intelligent Transport Systems and Services (ITS).
-
- Date: September 17, 2004
Where: British Machine Vision Conference (BMVC)
MERL Contact: Matthew Brand Brief - The paper "From Subspaces to Submanifolds" by Brand, M. was presented at the British Machine Vision Conference (BMVC).
-