Sensing and Machine Learning for Automotive Perception: A Review

    •  Pandharipande, A., Cheng, C.-H., Dauwels, J., Gurbuz, S., Ibanez-Guzman, J., Li, G., Piazzoni, A., Wang, P., Santra, A., "Sensing and Machine Learning for Automotive Perception: A Review", IEEE Sensors Journal, DOI: 10.1109/​JSEN.2023.3262134, Vol. 23, No. 11, pp. 11097-11115, June 2023.
      BibTeX TR2023-089 PDF
      • @article{Pandharipande2023jun,
      • author = {Pandharipande, Ashish and Cheng, Chih-Hong and Dauwels, Justin and Gurbuz, Sevgi and Ibanez-Guzman, Javier and Li, Guofa and Piazzoni, Andrea and Wang, Pu and Santra, Avik},
      • title = {Sensing and Machine Learning for Automotive Perception: A Review},
      • journal = {IEEE Sensors Journal},
      • year = 2023,
      • volume = 23,
      • number = 11,
      • pages = {11097--11115},
      • month = jun,
      • doi = {10.1109/JSEN.2023.3262134},
      • issn = {1558-1748},
      • url = {}
      • }
  • MERL Contact:
  • Research Areas:

    Artificial Intelligence, Computational Sensing, Machine Learning, Signal Processing


Automotive perception involves understanding the external driving environment as well as the internal state of the vehicle cabin and occupants using sensor data. It is critical to achieving high levels of safety and autonomy in driving. This paper provides an overview of different sensor modalities like cameras, radars, and LiDARs used commonly for perception, along with the associated data processing techniques. Critical aspects in perception are considered, like architectures for processing data from single or multiple sensor modalities, sensor data processing algorithms and the role of machine learning techniques, methodologies for validating the performance of perception systems, and safety. The technical challenges for each aspect are analyzed, emphasizing machine learning approaches given their potential impact on improving perception. Finally, future research opportunities in automotive perception for their wider deployment are outlined.