New approaches for testing of autonomous driving functions are using Virtual Reality (VR) to analyze the behavior of automated vehicles in various scenarios. The real time simulation of the environment sensors is still a challenge. In this paper, the conception, development and validation of an automotive radar raw data sensor model is shown. For the implementation, the Unreal VR engine developed by Epic Games is used. The model consists of a sending antenna, a propagation and a receiving antenna model. The microwave field propagation is simulated by a raytracing approach. It uses the method of shooting and bouncing rays to cover the field. A diffused scattering model is implemented to simulate the influence of rough structures on the reflection of rays. To parameterize the model, simple reflectors are used. The validation is done by a comparison of the measured radar patterns of pedestrians and cyclists with simulated values. The outcome is that the developed model shows valid results, even if it still has deficits in the context of performance. It shows that the bouncing of diffuse scattered field can only be done once. This produces inadequacies in some scenarios. In summary, the paper shows a high potential for real time simulation of radar sensors by using ray tracing in a virtual reality.
Recently the simulation of sensor data of advanced driver assistance systems (ADAS) becomes more and more important. In [
In the present work, investigations on this topic are conducted. The aim is to implement a physical correct radar model for ADAS functionalities within a virtual reality environment. For this, the overall radar model is separated into sub-models according the physical steps: emit, reflect and receive. Therefore, in Section 2 the theoretical basics of the used models are provided. Section 3 describes the implementation process. Section 4 provides a validation method for the model. The results of this validation are presented and discussed in Section 5 by using real measured values and analytical calculations. Finally, a conclusion for the potentials and further work to be done is given.
At the following chapter the overall radar model is introduced. It is separated into a sending antenna model, a scattering model and a receiving antenna model. Whereby the scattering model is additionally divided into more detailed models for specular reflection, diffuse reflection and directive reflection.
In common automotive radar sensors different types of sending and receiving antennas can be found. They are divided into single beam, multibeam and planar array concepts. For the control of planar array antennas complex algorithms are used. The objective of this work is to model radar antennas and propagations and not to model these algorithms. Hence a single beam antenna approach is chosen, based at [
S ( Φ ) = a i ⋅ ( Φ ± Δ Φ i ) 3 − b i (1)
Φ = ( l A / λ ) ⋅ sin ϕ (2)
Based on the Equations (1) and (2) and the parameters defined in
S ( Φ ) ( W m 2 ) = S ( Φ ) max ⋅ 10 S ( Φ ) [ dB ] 10 (3)
A further problem is that each ray needs a specified power to be send with, not a power density as defined in Equation (3). The concept provides an approach defining the power of one ray by integrating the power density up to the next. Regarding to
Parameter | Main lobe | Side lobe 1 | Side lobe 2 |
---|---|---|---|
a | −40 | −220 | −220 |
b | 0 | −13 | −18 |
Δ Φ i | 0 | 1.5 | 2.5 |
P 1 = ∫ ϕ 1 ϕ 2 S ( Φ ) ≈ S ( Φ ) ⋅ ( ϕ 2 − ϕ 1 ) (4)
With this assumption, it is possible to compute a defined sending power for each discrete ray of the antenna pattern in a two-dimensional case. To include the elevation angle, the propagation of the field in this direction is assumed constant. In this way, the model can be modified to a three-dimensional case through dividing the power of each ray in the azimuth direction by the number of rays in the elevation direction. This method allows to easily rebuild a radiation pattern of an automotive radar sending antenna by a defined number of rays.
To simulate the propagation of the microwaves emitted by the sending antenna in a virtual reality environment, scattering models are necessary. In this case three models are conceived. A specular model simulates the reflection of microwaves on perfectly smooth surfaces. A diffuse Lambertian model represents the opposite, perfectly rough surfaces. A third directive model represents the cases in between, where a scattered lobe is mainly reflected towards the direction of the specular case.
The specular reflection is a model for simulating microwave bounces on perfectly smooth surfaces. It uses a simple approach. Each ray impinging on a surface with an angle of θ i is mirrored on the surface normal at the impact point, as shown in
P s = S S c ⋅ P i (5)
As mentioned above a Lambertian model is used to represent the case of reflection on perfectly rough surfaces. The approach is based on the work [
| E _ S | 2 = | E _ S 0 | 2 ⋅ cos ( θ s ) (6)
| E _ S 0 | 2 = ( K ⋅ S S c ) 2 ⋅ cos ( θ i ) ⋅ cos ( θ s ) π (7)
These equations provide the possibility to compute the diffuse scattered field of an impinging ray. However, as with the antenna model, the problem arises that the resulting field needs to be discrete. Equation (8) shows an approach similar to the one used in the antenna model to numerically integrate the power density.
P 1 = ∫ θ S 1 θ S 2 E S ( θ S ) ≈ E S ( θ S ) ⋅ ( θ S 2 − θ S 1 ) (8)
This assumes that a ray represents the power integrated by the density up to the next ray. So far, a two-dimensional case for the diffuse scattering of a ray is described. To reproduce a three-dimensional scenario, for each discrete position of θ s a defined number of rays rotated around the surfaces normal is created. To get the power of the single rays, the power computed by (8) needs to be divided by the number of new rays. Now it is possible to describe a diffuse scattered field by a defined number of reflected rays. To allow a theoretically multipath propagation each outgoing ray needs to be reflected with the same model at its next bounce. This would result in a high number of rays. The impact of this on the implementation will be clarified in the further course of this study.
Just like the diffuse model the directive reflection model is adopted from [
| E _ S | 2 = | E _ S 0 | 2 ⋅ ( 1 + cos Ψ R 2 ) α R , α R ∈ ℕ (9)
| E _ S 0 | 2 = ( K ⋅ S S c ) 2 ⋅ cos ( θ i ) F α R (10)
F α R = 1 2 α R ⋅ ∑ j = 0 α R ( α R j ) ⋅ I j (11)
I j = 2 π j + 1 ⋅ [ cos θ i ⋅ ∑ w = 0 j − 1 2 sin 2 w θ i 2 2 w ] ( 1 − ( − 1 ) j 2 ) (12)
As with the Lambertian model an approach is needed to describe the scattered field by a defined number of rays. Equivalent to (8) Equation (13) is used to integrate the power for the rays, with the difference that the angle Ψ R of the field is used. This approach based on [
P 1 = ∫ Ψ R 1 Ψ R 2 E S ( Ψ R ) ≈ E S ( Ψ R ) ⋅ ( Ψ R 2 − Ψ R 1 ) (13)
Also, as before this model only covers the two-dimensional case. To derive the three-dimensional scenario each discrete ray needs to be rotated around the vector of the specular direction. The power of each ray can be computed by dividing the result of the two-dimensional case of (13) by the number of rays around the specular direction. For a theoretical multipath propagation each outgoing ray also needs to be reflected with the same model at its next bounce. Again, the impact of this on the implementation will be clarified later. This model allows to simulate cases of reflections that are not perfectly rough or smooth.
To model a receiving antenna a simplified approach is used. A primitive geometry bounded to the position of the virtual sensor represents the antenna. Each ray that hits this geometry on the way through the virtual space passes its data to a memory. To do that a request is done on each bounce of a ray whether the body hit was the antenna. If the request is true, the ray is not reflected anymore. The data that need to be stored are depending on the application case of the model. For example the rays powers, their impinging angle or also metadata like the last body hit can be stored. With this model it is not relevant whether a ray was directly reflected or received multiple bounces. The advantage is that the antenna geometry can be scaled up to perceive lower field strengths, represented by less impinging rays.
For the implementation into a Virtual Reality the Unreal Engine by Epic Games is used. It is a VR engine whose original purpose is the development of computer games. The programming can either be done graphically by using so called blueprint scripts or by textual C++ coding. The classical textual coding shows a much higher performance. With a regular state of the art gaming computer setup it is possible to generate 1.5 million rays per second. Based on this the implementation of the models as defined in the section 2 takes place. For this a simple testing environment is created. In the environment a character carries the model. This allows the user to move the virtual sensor within the scene. The sending and the receiving antenna models as well as the total reflection model can be implemented into the engine without any difficulties. For the diffuse and the directive model a problem arises. Because of the limited number of rays per second the reflection of the rays can only be done once. With these models it is not possible to do multiple bounces. Because of that only the specular and the directive models are implemented. The effect of this must be clarified by the validation of the model. The storage of the generated raw data of each ray is done as explained in Section 2.3. Each timestep arrays with the data of the impinging rays are stored into a matrix column by column. To export the data the MAT file format is used. This allows further processing of the raw data by the using the software MATLAB. For this the third-party C++ library “MAT-File API Library” is included into the Unreal Engine. This makes it possible to process and visualize the results. Thus, the validation of the model can be started.
To validate the implemented model a two-stepped validation concept with an increasing complexity of the considered models is conceived. The benefit of this is, that inadequacies can be systematically detected and analyzed. The individual steps are explained in the following sections.
In the first validation step, simple geometries are used to check the basic functionality of the models and to identify the parameters for further steps. A sphere with a cross section of one square meter and a square plate with the dimensions of 0.7 m × 1 m are used. The advantage if the spherical shape is, that its radar cross section (RCS) corresponds with its physical cross section. Hence a sphere with a cross-sectional area of one square meter is modeled and implemented into the virtual environment. In addition to the sphere a virtual rectangle plate with the given measurements is implemented into the virtual reality. The approximated RCS of this object is known from a measurement campaign with a real radar sensor. The measurement setup of this campaign is shown in
The RCS value of the used plate was determined to be 15.5 m2. To compare the measured and the simulated results a simple form of the radar equation is used (13). It represents the theoretically impinging power on the receiving antenna in connection with sending power of the radar P t , the RCS value of the considered object σ and the distance to the object R. The parameters G t and A e f f are describing the antenna gain of the sending antenna and the effective area of the receiving antenna respectively. As the receiving antenna is loss-free the effective area corresponds to the actual area of the antenna.
P r = P t ⋅ G t ⋅ σ ( 4 π ) 2 ⋅ R 4 ⋅ A e f f (14)
By Equation (14), which based on [
The second validation scenario uses 3D-models of different pedestrians and cyclists to simulate their radar signature and compares it with measured results. The measured data are taken from [
The pedestrian is placed on a rotatable table. The signature of the dummy is recorded on every rotation step and visualized in a heatmap plot. This heatmap shows die RCS value of the object applied in color. Its X-Axis represents the rotation of the dummy, the Y-axis displays the distance on the reflection, whose the origin lies in the rotation center of the rotatable table. To generate comparable results within the Virtual Reality an equivalent test setup and a similar visualization is implemented. The test objects are rotated by one degree on every simulation step. The results of each on the receiving antenna impinging ray are stored into matrices and exported into a Mat-file. In MATLAB the results of the rays are sorted into range gates. Then the RCS value is calculated for each area. So, the results can be plotted as a heatmap, comparable to the paper. In the last step equivalent 3D-models for measured dummies from the paper must be found. For comparison a pedestrian model and a cyclist model are chosen. The selected models are shown in
Even if the poses of the Models are very similar, small differences can be found. For example, the physique of the digital walking pedestrian is stronger, and it has a more open body posture. The physical dummy is smaller. The cyclist and the cycle also show small inadequacies. The frontal wheel fork is not given in the physical arrangement. Also, the posture of the digital cyclist is more open again. The model quality is considered sufficient for a comparison of the simulated data with measured results. Errors due to simplifications are to be expected on the physical measurement side, not on the simulation side.
Based on the in section 4 conceived validation scenarios now the results of the simulations are presented and discussed. This is done individually for each scenario according to the same structure as used before.
The surveying of the simple sphere and plate geometries in the virtual environment is done individually for both implemented reflection models, the total and the directive reflection. The results of the measurement with the total reflection model immediately show its shortcomings. For the sphere geometry no incoming power is registered on the receiving antenna. On closer examination it can be determined that this reflection model is sensitive to small geometric errors.
For the rectangular plate, the results of the total reflection are also to be discussed. With the measured RCS value and the usage of the radar Equation (13) the theoretically impinging power results in 0.0162 W. The simulated impinging power is 0.0434 W. This large deviation can only be reduced by an artificial reduction of the scattering parameter S S c . Summarizing the total reflection model shows weaknesses caused by geometric inadequacies as well as physically incorrect simulations. In conclusion the total reflection model is not valid.
The directive reflection model shows much better results. In contrast to the previous model it is insensitive to geometric inaccuracies. The results for the simple test geometries are summarized in
Object | Pin target size | Pin measured | Relative deviation |
---|---|---|---|
Sphere | 1.044 × 10−3 W | 1.033 × 10−3 W | 1.1% |
Plate | 0.0162 W | 0.0154 W | 4.9% |
Based on the previous results and parameters determined, the simulation for the human models is carried out. As it has been shown the total reflection model does not provide valid results. Hence the following simulations are done with the directive reflection model. The execution of the simulation is done as described in Section 4.2. The simulated results for the pedestrian and the cyclist dummy are shown in
The final step in the validation process is to assess performance of the implemented directive reflection model. The previous validation tests have been done without considering the real-time capability. Assuming that a framerate of at least 30 FPS is needed for further use cases of the model, this framerate is selected as the lower limit. The tests are done on a state-of-the-art gaming computer. It shows that it is possible to do 1.5 Million ray traces per second. The practice shows that 34,400 rays per frame lead to a framerate of 40 FPS. This brings the advantage of having a reserve for drops in the framerate.
The present work shows a new approach for the generation of automotive radar raw data in a virtual environment. The implemented model is divided into three sub-models, a sending antenna, different scattering models and a receiving antenna model. Thereby the overall scattering model includes the cases specular reflection, diffuse reflection and directive reflection. To decrease numerical effort and validate the early stage of the model, multiple reflections for the directive reflection model were neglected. Therefore, a two stepped validation process was conducted. The first step shows a general functionality of the total reflection model, but caused of individual issues of the modelling method of geometries in VR, the overall functionality derivate from physical reality. It is sensitive against small geometric errors and does not validly simulate the microwave physics. The directive scattering model shows much better accordance for the simulation of the reflected power of simple geometries as well as the simulation of the radar signatures of typical road user models. The results turn out to be valid, only the performance causes issues. With a rising framerate the content of information in the virtual raw data is reduced. The influence of this lack of information on further algorithms needs to be clarified regarding the final use case. Summarizing the implemented models give new opportunities for the simulation of radar raw data in virtual environments.
The Project is supported by the Ministry of Economic Affairs, Innovation, Digitization and Energy of North Rhine-Westphalia and by HH Vision, Hoersch und Hennrich Achitekten GbR.
The authors declare no conflicts of interest regarding the publication of this paper.
Degen, R., Ott, H., Overath, F., Schyr, C., Leijon, M. and Ruschitzka, M. (2021) Methodical Approach to the Development of a Radar Sensor Model for the Detection of Urban Traffic Participants Using a Virtual Reality Engine. Journal of Transportation Technologies, 11, 179-195. https://doi.org/10.4236/jtts.2021.112012