_{DDIVA 2021}

Modern supervised machine learning models require large amounts of labelled data for timely convergence during training and good generalization during testing. Thus, labelling large amounts of data has become an inevitable bottleneck in the model production pipeline. In this talk, I will present techniques and workflows that could considerably reduce the labelling time and/or effort, and in some cases remove the need to label altogether. To further illustrate this, I will present three case studies where these ideas have been put to use in practice, and discuss the resulting outcomes of our approach. In particular, this talk will cover concepts such as - simultaneous data and label capture, automatic labelling using auxiliary sensors, and task-specific data augmentation schemes. These techniques are meant to be for general-use, and could be applied to or adapted for tasks beyond the ones covered in this talk.

Most research on data-driven perception in Intelligent Vehicles focuses on camera and lidar perception tasks, including object detection and scene segmentation. In this talk, I will present our group's research on data-driven methods for other sensing modalities that traditionally rely on signal processing (radar), or are even completely ignored in current IV research (acoustics).
First, I will demonstrate how the low-level radar data, represented as a 3D azimuth-range-Doppler tensor, provides valuable information to classify radar targets before clustering them for multi-class object detection [Palffy,RA-L’20]. We train our low-level radar detection network using annotations obtained from a visual detector on synchronized radar and camera recordings with our group's research vehicle. Extrinsic multi-modal sensor calibration for our vehicle will also be shortly discussed [Domhof,T-IV’21].
Second, I present novel research [Schulz,RA-L’21] that shows how a vehicle-mounted microphone array can be used to detect occluded traffic around corners 1 second before the camera-based detection system can. For these experiments we collected a new dataset with our research vehicle at various narrow alleys in the Delft inner city.
Overall, we seek to exploit easily collected multi-modal sensor data. Still, as we lack truly huge datasets and annotations needed for end-to-end learning, we rely on a combination of data-driven learning and classic signal processing techniques.

[Palffy,RA-L’20]: “CNN based Road User Detection using the 3D Radar Cube”, A. Palffy et al., IEEE Robotics and Automation Letters (RA-L), 2020
[Domhof,T-IV’21]: “A Joint Extrinsic Calibration Tool for Radar, Camera and Lidar”, J. Domhof et al., IEEE Trans. on Intelligent Vehicles (T-IV), 2021
[Schulz,RA-L’21]: “Hearing What You Cannot See: Acoustic Vehicle Detection Around Corners”, Y. Schulz et al., IEEE Robotics and Automation Letters (RA-L), 2021

Automated vehicles (AVs) are gaining increasing interest and their development is making great progress. However, assuring safe driving operation under all possible road and environment conditions is still an open challenge. In this regard, vehicle simulation is seen as a major corner stone for test and validation. Recorded real world challenges can be rebuild in simulation (e.g. NHTSA pre-crash scenarios) and in addition artificial corner cases can be added on top. Those can then be utilized to test the complex software stack in various configurations to find possible safety or availability issues. For example, the same situation can be tested with different settings of the planning modules (driving policy) or road conditions to ensure that all possibilities result in safe driving. In this talk, we will present how the CARLA vehicle simulator in combination with an open source scenario editor can be used to re-create traffic scenarios, play those under various operating conditions and by that means make one step towards safe autonomous driving.

Developing reinforcement learning (RL) algorithms for automated tactical decision making has been an attractive topic in recent years. It is evident that designing RL based autonomous driving systems can help tremendously with handling performance and safety-based issues of alternative planning and decision making approaches. That being said, most RL algorithms are trained in simulators, and require large-amounts of data to converge to good solutions. Thus, designing sample-efficient RL algorithms is important for accelerating design cycles and verifying the safety and robustness of RL solutions. In addition, good performance in simulation does not always imply good performance in real life tests. Thus, additional measure needs to be taken to guarantee that trained RL agents generalize to real-life situations. In this talk, we go over three case studies that deals with these issues; i) how to utilize curriculum RL to boost your autonomous driving agent’s performance using limited amount of data; ii) how to take advantage of offline RL to inject external driving demonstrations for improving sample efficiency, iii) how to use multiple fidelity simulators to transfer simulated performance to real-life.