AutoVision – A Run-time Reconfigurable MPSoC Architecture for Future Driver Assistance Systems

The Autovision architecture is a new Multi Processor System-on-Chip (MPSoC) architecture for video-based driver assistance systems, using run-time reconfigurable hardware accelerator engines for video processing. According to various driving conditions (highway, city, sunlight, rain, tunnel entrance) different algorithms have to be used for video processing. These different algorithms require different hardware accelerator engines, which are loaded into the Autovision chip at run-time of the system, triggered by changing driving conditions. We investigate how to use dynamic partial reconfiguration to load and operate the right hardware accelerator engines in time, while removing unused engines in order to save precious chip area.

Introduction

In future automotive systems, video-based driver assistance will improve security. Video processing for driver assistance requires real time implementation of complex algorithms. A pure software implementation does not offer the required real time processing, based on available hardware in automotive environments. Therefore hardware acceleration is necessary. Dedicated hardware circuits (ASICs) can offer the required real time processing, but they do not offer the necessary flexibility. Video algorithms for driver assistance are not standardized, and never might be. Algorithmic research is expected to go on for future years. So a flexible, programmable hardware acceleration is needed. Specific driving conditions, e.g. highway, country side, urban traffic, tunnel, require specific optimized algorithms. Reconfigurable hardware offers high potential for real time video processing and its adaptability to various driving conditions and future algorithms.

Today’s systems for driver assistance offer features as adaptive cruise control and lane departure warning. Both video cameras and radar sensors are used. On highways and two-way primary roads a safe distance to previous cars can be kept automatically over a broad speed range However, for complex driving situations and complex environments, e.g. urban traffic, there are no established and reliable algorithms. This is a topic for future research.

Algorithms for video processing can be grouped into high level application code and low level pixel operations. High level application code requires a high degree of flexibility and, thus, is good for a standard processor implementation. Pixel manipulation on the other hand requires applying the same operation on many pixels and, thus, seems to be a good candidate for hardware acceleration.

Hardware reconfiguration

In general, a large number of different coprocessors could be implemented on a System-on-Chip (SoC) in parallel. However, this is not resource efficient, as depending on the application, just a subset of all coprocessors will be active at the same time. With hardware reconfiguration, hardware resources can be used more efficiently. Coprocessor configurations can be loaded into a FPGA whenever needed, depending on the application.

Let us look at the following driving scenario: A car is driving daytimes on a highway and entering a tunnel. On the highway other road users, e.g. cars, trucks, bikes, motorbikes, pedestrians, animals, have to be recognized and distinguished. We started to use MPEG-7-based Region Shape Descriptors for this task. The pixel-level processing can be accelerated by a coprocessor, called ShapeEngine. If the road is leading into a tunnel, the tunnel entrance will be marked as Region of Interest (ROI).

The tunnel entrance itself is rather dark, the surrounding is bright. The most important area is the road inside the tunnel, behind the entrance, where edge detection could be used to detect obstacles. While vertical edges and vertical luminance structures mainly belong to the road and to lane markers, horizontal edges mainly belong to obstacles, e.g. vehicles in the lane. The related pixel-level processing can be accelerated by a coprocessor, called EdgeEngine.

Inside the tunnel, due to the low luminance level, only the lights of other vehicles can be detected. Headlights, rear lights, and fixed tunnel lights have to be distinguished. This is done by luminance segmentation and measurement of position and movement of light areas. The processing on pixel-level can be accelerated by a coprocessor, called SegmentEngine.

The coprocessors are used as shown in table 1:

 

Our target hardware architecture will use a Xilinx Virtex-II Pro FPGA with two embedded PowerPC (PPC) cores. PPC-1 will be used for high level application code, PPC-2 will be used for control and management functions. The configurable logic parts of the FPGA will be used for coprocessors, on-chip bus, data I/O, and memory interface. On-chip Block RAM (BRAM) will be used for local memory with the coprocessors. Loading and replacement of the Engines will be controlled PPC-2.

Taillightdetection

Optical Flow

Acknowledgments

Exceedingly we want to thank Xilinx research labs for providing development boards for our research activities.

Dissertation

Dissertation Christopher Claus:
"Zum Einsatz dynamisch rekonfigurierbarer eingebetteter Systeme in der Bildverarbeitung"
(ext. Link...)