Multi-Band Wi-Fi Sensing with Matched Feature Granularity


Complementary to the fine-grained channel state information (CSI) and coarse-grained received signal strength indicator (RSSI) measurements, the mid-grained spatial beam attributes (i.e., beam SNR) during the millimeter-wave (mmWave) beam training phase were recently repurposed for Wi-Fi sensing applications such as human activity recognition and indoor localization. This paper proposes a multi-band Wi-Fi sensing framework to fuse features from both CSI from 5 GHz bands and the mid-grained beam SNR at 60 GHz with feature granularity matching that pairs feature maps from the CSI and beam SNR at different granularity levels with learnable weights. To address the issue of limited labeled training data, we propose to pretrain an autoencoderbased multi-band Wi-Fi fusion network in an unsupervised fashion. For specific sensing tasks, separate sensing heads can be attached to the pretrained fusion network with fine-tuning. The proposed framework is thoroughly validated for three sensing applications using in-house experimental datasets: 1) pose recognition; 2) occupancy sensing; and 3) indoor localization. Comparison to a list of baseline methods demonstrates the effectiveness of granularity matching. An ablation study is performed as a function of the amount of labeled data, the latent space dimension, and learning rates


  • Related News & Events

    •  NEWS    Keynote address given by Philip Orlik at 9th annual IEEE Smartcomp conference
      Date: June 26, 2023
      Where: International Conference on Smart Computing (SMARTCOMP), Vanderbilt University, Nashville, Tennessee
      MERL Contact: Philip V. Orlik
      Research Areas: Communications, Dynamical Systems, Machine Learning, Multi-Physical Modeling, Signal Processing
      • VP & Research Director, Philip Orlik, gave a keynote titled, "Smart Technologies for Smarter Buildings" at the 9th edition of the IEEE International Conference on Smart Computing (SMARTCOMP) focusing on some of the research challenges and opportunities that arise as we seek to achieve net-zero emissions in Smart building environments.

        SMARTCOMP is the premier conference on smart computing. Smart computing is a multidisciplinary domain based on the synergistic influence of advances in sensor-based technologies, Internet of Things, cyber-physical systems, edge computing, big data analytics, machine learning, cognitive computing, and artificial intelligence.
    •  NEWS    MERL Researchers gave a Tutorial Talk on Quantum Machine Learning for Sensing and Communications at IEEE GLOBECOM
      Date: December 8, 2022
      MERL Contacts: Toshiaki Koike-Akino; Pu (Perry) Wang
      Research Areas: Artificial Intelligence, Communications, Computational Sensing, Machine Learning, Signal Processing
      • On December 8, 2022, MERL researchers Toshiaki Koike-Akino and Pu (Perry) Wang gave a 3.5-hour tutorial presentation at the IEEE Global Communications Conference (GLOBECOM). The talk, titled "Post-Deep Learning Era: Emerging Quantum Machine Learning for Sensing and Communications," addressed recent trends, challenges, and advances in sensing and communications. P. Wang presented on use cases, industry trends, signal processing, and deep learning for Wi-Fi integrated sensing and communications (ISAC), while T. Koike-Akino discussed the future of deep learning, giving a comprehensive overview of artificial intelligence (AI) technologies, natural computing, emerging quantum AI, and their diverse applications. The tutorial was conducted remotely. MERL's quantum AI technology was partly reported in the recent press release (

        The IEEE GLOBECOM is a highly anticipated event for researchers and industry professionals in the field of communications. Organized by the IEEE Communications Society, the flagship conference is known for its focus on driving innovation in all aspects of the field. Each year, over 3,000 scientific researchers submit proposals for program sessions at the annual conference. The theme of this year's conference was "Accelerating the Digital Transformation through Smart Communications," and featured a comprehensive technical program with 13 symposia, various tutorials and workshops.
  • Related Publication

  •  Yu, J., Pu, W., Koike-Akino, T., Wang, Y., Orlik, P.V., "Multi-Band Wi-Fi Sensing with Granularity Matching", arXiv, November 2021.
    BibTeX arXiv
    • @article{Yu2021nov,
    • author = {Yu, Jianyuan and Pu, Wang and Koike-Akino, Toshiaki and Wang, Ye and Orlik, Philip V.},
    • title = {Multi-Band Wi-Fi Sensing with Granularity Matching},
    • journal = {arXiv},
    • year = 2021,
    • month = nov,
    • url = {}
    • }