A Scenario-Based Development Framework for Autonomous Driving

Scenario-based testing was first applied to the development of software systems. “Scenarios” were used to describe the use of the system, the requirements for use, the use environment, and the construction of more feasible systems  [5, 17, 8]. Since then, many fields have defined the term scene in their respective disciplines, such as climate change [37], energy industry [6] and so on.

However, in the field of autonomous driving at this stage, “scenario” has not yet been clearly defined. Since Schieben et al. [42] applied the concept of scenario to automatic driving tests, many scholars have put forward their own understanding of the term “scenarios”. Elrofai et al. [10] defined that “the scene is to test the continuous changes of the dynamic environment around the vehicle in a specific time range, including the behavior of the test vehicle in this environment”. Koskimies [25] defined that “a scene is an informal description of a series of events when the system performs a specific task”, and an object-oriented modeling method can be used to describe the scene. RAND proposed in the autonomous driving research report that “scenarios are a combination of a series of elements used to detect and verify the behavioral capabilities of autonomous driving systems in specific driving environments”. The PEGASUS project proposes corresponding functional scenes, logical scenes, and physical scene concepts based on the differences in demand for scenes during the concept phase, system development phase, and test phase of autonomous driving product development [34]. Chinese academician Zheng Nanning of Xi’an University of Communications defines a scene as “a specific situation or scene of a traffic occasion at a specific time and in a specific space. It can be defined as a set of entities that can give a rich description of the current environment with perceptual data.” [53]. Based on the above viewpoints, these scene definitions are consistent in the core elements: they all include road environment elements, other traffic participants, and vehicle driving tasks. At the same time, these elements will last for a certain period of time and have dynamic characteristics.

Therefore, the autonomous driving scenario can be understood as such: a scenario is the dynamic description of the components of the autonomous vehicle and its driving environment over a period of time. The relationship of these components is determined by the functions of the autonomous vehicle to be inspected. In short, the scene can be regarded as a combination of the driving situation and driving scene of an autonomous vehicle.

Autonomous driving scenarios are infinitely rich, extremely complex, difficult to predict, and inexhaustible. Therefore, the scenarios used for developing and testing should meet the requirements of quantifiable (the features of each element of the scenario can be quantified) and reproducible (the scenario is in the current technology The basic and test software can be reproduced) and high-fidelity (can present or reflect the real world scene to a certain extent).

Determining the ontology of the scenario element is the cornerstone of scenario-based techniques. However, there are still disputes among different researchers regarding the types and content of ontology.

Commonly used open source schemas such as OpenDrive and OpenScenario specified their road elements and traffic dynamic elements definitions in detail [31, 23]. Ulbrich et al. [46] proposed that the elements of a scene should include test vehicles, traffic environment elements, driving task information, and specific driving behaviors. Autonomous driving is a part of the test scene. Geyer et al. [15] believe that the scene is the pre-defined driving environment, driving tasks, static elements and dynamic elements during the automatic driving test, and the test vehicle itself is not included in the scene. Korbinian et al. [18] divided the scene elements into three categories: the environmental part (weather, light, wind speed, etc.), the static part (lane lines, trees, obstacles) and the dynamic part (traffic participants, pedestrians). In the latest report of RAND, the scene elements are divided into 5 layers, namely the road information layer (lane line, intersection shape, number of lanes, etc.), road infrastructure layer (traffic signs, trees, guardrails, etc.), road information layer and road The dynamic changes of the facility layer (road maintenance, tree breaking, obstacle movement, etc.), dynamic targets (pedestrians, traffic participants), environmental conditions (light, weather), test vehicles are not included. Matthaei et al. [32] discussed whether weather and light should be included as scene factors. Zhu et al. [54] categorized scenarios into test vehicles and traffic environments. Erwin et al. [7] believe that in the early stage of system development, the scene only needs basic information about the road and other traffic participants.

During testing, the test vehicle itself will have a significant impact on surrounding scene elements, especially other traffic participants. The interaction between the test vehicle and the surrounding driving environment forms a closed loop. At the same time, the property of the test vehicle will have a key impact on the behavioral decision-making of the automatic driving system. For example, the acceleration performance of the vehicle during overtaking plays a decisive role in the execution of the decision. Therefore, the test vehicle should be treated as a part of the scene, and the surrounding driving environment constitutes the whole scene.

Based on this concept, we integrate the above-mentioned research and propose a scenario ontology shown in Fig 1.

In this ontology, the scenario elements have two categories: basic information of the vehicle and environment elements. Among them, the basic information of the vehicle includes three categories: basic elements of the test vehicle, target information, and driving behavior. Traffic environment elements include weather and light, Static road information, dynamic road information and traffic participant information.

It is necessary to collect a large amount of scenario data and establish a scenario library. For example, PEGASUS and KITTI in Germany, NHTSA Autonomous Driving Test Architecture Project in the United States, University of California, Berkeley BDD100K, China’s ”Kunlun Project”, Baidu ApolloScape, etc. are all committed to providing more practical scenario data for autonomous driving research and testing [14].

The data sources mainly include three parts: real data, simulation data and expert experience data. The specific content is shown in Figure 2.

The key to scene data processing is the deconstruction and reconstruction of scene elements.

The German PEGASUS project proposes 7 steps for scene data processing [40]: Generate a general environment description, check the data format, generate additional information, analyze the degree of correlation between the scenes, analyze the possibility of scene occurrence, cluster logical scene data and calculate the frequency distribution, and generate specific test scenes based on the generated logical scenes. Baidu proposed a three-step method of scene clustering including scene classification rule definition, scene labeling (element decomposition, quantification), and label clustering.

According to the existing typical scene data processing methods, this article summarizes and proposes the scene data processing flow shown in Figure 3.

Based on the probability density $\mathcal{K}$ of various scenes in Eq.2, specific scenes can be randomly generated in the virtual environment. The generation methods mainly lies in three categories. 1) Random sampling represented by Monte Carlo sampling and fast random search tree. 2) Importance based sampling such as importance level analysis of scene elements. 3) Machine learning based methods.

Compared with building real test scenarios in the real world, generating test cases in a virtual environment can greatly reduce time and resource consumption. However, due to the low probability of accidents under natural circumstances, the method of using random generation may still face a large number of calculation difficulties. Putting more weight on dangerous scenes generation can alleviate this problem.

First of all, it is necessary to define and classify dangerous scenes. Many projects have conducted research on car dangerous scenes. SeMiFOT divides the risk of driving into 4 levels [1]. The United States NHTSA classifies collisions into 37 categories [36]. Aparicio et al. [2] summarized the types of conflicts between cars and cars, cars and pedestrians. Winkle et al. [49] analyzed accident data in which the line of sight was blocked in different weather conditions from 2004 to 2014, and analyzed the severity of the accident.

The definition of the above-mentioned dangerous scenes is narrow where most of them only analyze the types of their dangers without defining specific parameters of the scene elements. Tang et al. [45] define each attribute parameter of the accident scene, and propose a method for drawing urban traffic accidents. Sven et al. [19] used specific parameters such as TTB, expected braking deceleration, TTC, traffic flow, speed fluctuation, average speed, acceleration change and other specific parameters to find the dangerous scenes from the massive car driving data. Elrofai et al. [10] judged whether there is lane changing behavior by detecting the speed and yaw rate of the vehicle during driving. When the continuous yaw rate exceeds the threshold for a period of time, it is judged as a valuable steering event. Huang et al. [22] proposed a method to accelerate the generation of dangerous scenes based on importance sampling based on the defined dangerous scenes. The core idea is to introduce a new probability density function $f^{*}(x)$ to increase the probability of producing dangerous scenes, thereby reducing the number of tests. When using the randomly scene generation method, the probability density function of the dangerous scene is $f(x)$, and the minimum number of tests $n$ is

where $\gamma$ is the probability of a dangerous scenario, $z$ is related to the inverse cumulative distribution function of $\mathcal{N}(0,1)$.

When importance sampling is used to generate dangerous scenes, the probability density function of the dangerous scenes is $f^{*}(x)$, and the minimum number of tests is

where $I(x)\in[1,0]$ is the index function of dangerous event $\varepsilon$ and $L(x)=\frac{f(x)}{f^{*}(x)}$ is likelihood ratio for using importance sampling. $E_{f^{*}}[I^{2}(x)\cdot L^{2}(x)]$ is the probability of occurrence of the dangerous scene after changing the probability density function to $f^{*}(x)$.

Through the verification of the method for typical scenes such as cut in and AEB, it is proved that the test speed is 7,000 times faster than Monte Carlo test simulation.

There are three technical challenges for auto test scenario generation: authenticity, granularity, and measurement.