Research on the Method of Applying 3D Gaussian Splatting Technology to Help Conduct Automatic Driving Training

Yanxin Wu

doi:10.54097/fgs5rp10

Authors

Yanxin Wu

DOI:

https://doi.org/10.54097/fgs5rp10

Keywords:

Automatic driving; 3D Gaussian; 3D Gaussian Splatting; 3D scene reconstruction technology.

Abstract

The purpose of this study is to explore the application of 3D Gaussian Splatting technology in automated driving training. In reality, the training of automatic driving models will produce great labor and economic consumption, but the use of virtual 3D scenes can significantly reduce this consumption, and at the same time have better training effects. Therefore, how to better use 3D for the training of automatic driving models will also be the future development direction of the field of automatic driving. In this paper, some major 3D Gaussian based dynamic scene modeling methods are summarized and some improvement schemes are proposed, including scene segmentation, new scene generation and dynamic scene rendering. This paper hopes to summarize and improve the current research, and finally propose that the generation target scene can be customized according to the training requirements of users' automatic driving models, so as to achieve more efficient and practical 3D scene generation.

Downloads

Download data is not yet available.

References

[1] Gao R, Qi Y. A Brief Review on Differentiable Rendering: Recent Advances and Challenges. Electronics, 2024, 13(17): 3546.

[2] Kerbl B, Kopanas G, Leimkuhler T, et al. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph., 2023, 42(4): 139:1-139:14.

[3] Zhou X, Lin Z, Shan X, et al. DrivingGaussian: Composite Gaussian splatting for surrounding dynamic autonomous driving scenes//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 21634-21643.

[4] Yan Y, Lin H, Zhou C, et al. Street Gaussians for modeling dynamic urban scenes. arXiv preprint arXiv:2401.01339, 2024.

[5] Sharma G, Liu D, Maji S, et al. Parsenet: A parametric surface fitting network for 3d point clouds//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VII 16. Springer International Publishing, 2020: 261-276.

[6] Tancik M, Casser V, Yan X, Pradhan S, Mildenhall B, Srinivasan P P, Barron J T, Kretzschmar H. Block-nerf: Scalable large scene neural view synthesis. In: CVPR, 2022.

[7] Tang J, Ren J, Zhou H, et al. DreamGaussian: Generative Gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309.16653, 2023Ma Kunlong. Short term distributed load forecasting method based on big data. Changsha: Hunan University, 2014.

[8] Ge C, Xu C, Ji Y, et al. CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussians. arXiv preprint arXiv:2410.20723, 2024.

[9] Huang N, Wei X, Zheng W, et al. $textit {S}^ 3$ Gaussian: Self-Supervised Street Gaussians for Autonomous Driving. arXiv preprint arXiv:2405.20323, 2024.

[10] Luiten J, Kopanas G, Leibe B, et al. Dynamic 3d Gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.09713, 2023.

[11] Shah S, Dey D, Lovett C, et al. Airsim: High-fidelity visual and physical simulation for autonomous vehicles//Field and Service Robotics: Results of the 11th International Conference. Springer International Publishing, 2018: 621-635.

[12] Dosovitskiy A, Ros G, Codevilla F, et al. CARLA: An open urban driving simulator//Conference on robot learning. PMLR, 2017: 1-16.

[13] Ravi N, Reizenstein J, Novotny D, et al. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08501, 2020.