Research Progress of Millimeter-wave Radar and Multi-sensor Fusion Algorithm in Assisted Driving

Chao Chen

doi:10.54097/xtg16027

Authors

Chao Chen

DOI:

https://doi.org/10.54097/xtg16027

Keywords:

Millimeter-wave radar, multi-sensor fusion, Advanced driver assistance systems (ADAS), Bird's-Eye-View (BEV) space, End-to-end optimization.

Abstract

With the rapid advancement of intelligent transportation technologies, market demands for advanced driver assistance systems (ADAS) have grown increasingly stringent. Environmental perception, as the core technical component of autonomous driving systems, faces complex real-world challenges. Traditional single-modality perception solutions exhibit inherent limitations, prompting the emergence of multi-modal perception systems. Among these, millimeter-wave (mmWave) radar-a cost-effective and high-performance sensor-transcends human perceptual constraints and is pivotal in ADAS. This paper summarizes recent technological progress in fusion algorithms integrating mmWave radar with diverse sensors for ADAS applications, with a focused analysis of fusion mechanisms and functional roles between mmWave radar and visual sensors, LiDAR, and infrared sensors. The study reveals intrinsic connections and evolutionary patterns across different technical pathways. The research aims to provide theoretical references for designing intelligent driving perception systems and facilitate breakthroughs in multi-modal fusion challenges.

Downloads

Download data is not yet available.

References

[1] S&P Global Mobility. "2025 Auto Sales Forecast: 89.6M Vehicle Sales Worldwide," S&P Global Automotive Insights, Dec. 20, 2024. [Online]. Available: https://www.spglobal.com/automotive-insights/en/blogs/2025-auto-sales-forecast-global. [Accessed: Mar. 20, 2025].

[2] McKinsey & Company. "McKinsey China Auto Consumer Insights 2024: Evolution, not involution," McKinsey & Company, May 31, 2024. [Online]. Available: https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/mckinsey-china-auto-consumer-insights-2024. [Accessed: 20-Mar-2025].

[3] Tesla, Inc. Vehicle Hardware 4.0 Technical Disclosure [Internal Document]. Fremont, CA: Tesla Engineering Department, 2023.

[4] B. Steux, C. Laurgeau, L. Salesse, D. Wautier. "Fade: a vehicle detection and tracking system featuring monocular color vision and radar data fusion," in Intelligent Vehicle Symposium, 2002. IEEE, Versailles, France, 2002, pp. 632-639 vol.2, doi: 10.1109/IVS.2002.1188022.

[5] IEEE, "IEEE Standard for Automotive Vision Systems in Advanced Driver Assistance Systems," IEEE Std 2745 - 2745, 2024, doi: 10.1109/IEEESTD.2024.7245829.

[6] L. Shi et al. "Tianmu: A Hybrid Vision Chip for Robust and Efficient Autonomous Driving Perception," Tsinghua Science and Technology, vol. 29, no. 3, pp. 385 - 394, 2024, doi: 10.1109/TST.2024.3376542.

[7] X. Li et al. "FUTR3D: A Unified Sensor Fusion Framework for 3D Detection," in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 1-10, doi: 10.1109/CVPR52688.2023.00087.

[8] F. Zhao, D. Li, W. Yu. "Data Fusion and Speed Tagging Alignment Based on Millimeter Wave Radar Detection and Camera Images," in 2020 International Conference on Microwave and Millimeter Wave Technology (ICMMT), Shanghai, China, 2020, pp. 1-3, doi: 10.1109/ICMMT49418.2020.9386772.

[9] Y. Lin, W. Zhou, S. Wang, W. Ruan. "Multi-Target Pedestrian Tracking Using Radar and Visual Detection Information Fusion," in 2023 8th International Conference on Power and Renewable Energy (ICPRE), Shanghai, China, 2023, pp. 2046 - 2051, doi: 10.1109/ICPRE59655.2023.10353623.

[10] Z. Liu et al. "BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation," in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 1 - 8, doi: 10.1109/ICRA48891.2023.10160968.

[11] Y. Zhao et al. "Radar-Image Fusion Transformer for Object Detection with Uncertainty Estimation," in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC), 2023, pp. 1 - 8, doi: 10.1109/ITSC57777.2023.10422373.

[12] X. Chen et al. "DeepFusion: A CNN-Based Framework for Radar-Camera Feature Fusion in Autonomous Driving," IEEE Trans. Vehicular Technology, vol. 73, no. 2, pp. 1234 - 1245, 2024, doi: 10.1109/TVT.2023.3321876.

[13] L. Huang et al. "A Collision Warning Method Based on Millimeter Wave Radar and Vision Fusion," IEEE Trans. Intell. Transp. Syst., 2023.

[14] Y. Guo et al. "Camera Radar Fusion for Increased Reliability in ADAS Applications," Electronic Imaging Conference, 2018.

[15] H. Cui et al. "Online Multi-Pedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision," IEEE Sensors J., vol. 23, no. 10, pp. 10245 - 10255, 2023.

[16] J. Wang et al. "Multimodal Sensor Fusion for 3D Object Detection in Autonomous Driving," IEEE Trans. Intell. Vehicles, vol. 8, no. 2, pp. 1123 - 1135, 2023.

[17] A. H. Lang, S. Vora, H. Caesar et al. "PointPillars: Fast encoders for object detection from point clouds," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Piscataway, NJ, USA: IEEE, 2019, pp. 12689 - 12697.

[18] H. Tai, Y. Qian, X. Kang et al. "Fusing LiDAR and Radar with Pillars Attention for 3D Object Detection," in 2024 7th International Symposium on Autonomous Systems (ISAS), Chongqing, China, 2024, pp. 1 - 6, doi: 10.1109/ISAS61044.2024.10552581.

[19] Z. Peng, J. Feng, Y. Wu et al. "Data Fusion Approach with MMW Radar and IR Sensor Based on MEKF," in 2007 International Conference on Mechatronics and Automation, Harbin, China, 2007, pp. 1992 - 1996, doi: 10.1109/ICMA.2007.4303856.

[20] S. Liu, J. Wu, B. Lv, X. Pan et al. "Data Fusion of Roadside Camera, LiDAR, and Millimeter-Wave Radar," IEEE Sensors J., vol. 20, 2024.