Safety validation for autonomous vehicles with naturalistic and adversarial driving environment

Abstract: Safety validation is critical to the development and deployment of autonomous vehicles. The prevailing approach tests autonomous vehicles in the naturalistic driving environment (NDE). However, due to the high dimensionality of the environment and the rareness of safety-critical events, hundreds of millions of miles would be required to demonstrate the safety performance of autonomous vehicles, which is severely inefficient. We discover that sparse but adversarial adjustments to NDE, resulting in the naturalistic and adversarial driving environment (NADE), can significantly reduce the required test miles without loss of evaluation unbiasedness. By training the background vehicles to learn when to execute what adversarial maneuver, NADE becomes an intelligent environment for safety validation. We demonstrate the effectiveness of the NADE by testing a highly automated vehicle in test tracks with an augmented reality environment, combining simulated background vehicles with a real AV under test. Compared with NDE, NADE can accelerate the evaluation process by multiple orders of magnitude.

Bio: Dr. Shuo Feng is an Assistant Research Scientist at the University of Michigan Transportation Research Institute (UMTRI) and an incoming Assistant Professor at Tsinghua University. He received his bachelor’s and Ph.D. degrees in the Department of Automation at Tsinghua University, China, in 2014 and 2019, respectively. Before joining UMTRI, Dr. Feng was a postdoctoral research fellow in the Department of Civil and Environmental Engineering at the University of Michigan, Ann Arbor. His research interests lie in the development and validation of safety-critical machine learning, particularly for connected and automated vehicles. Dr. Feng has served as the associate editor of IEEE Transactions on Intelligent Vehicles and academic editor of Automotive Innovation. He was the recipient of the “Best Ph.D. Dissertation Award” from the IEEE Intelligent Transportation Systems Society (ITSS) in 2020 and the Intelligent Transportation Systems Best Paper Award from the INFORMS Transportation Science and Logistics society in 2021.