Duckietown

At LIS, we try to leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebots' control system (https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/).
More information on Duckietown can be found at https://www.duckietown.org/.

In previous work, an LCT agent to steer Duckiebots has been developed using only the angular heading error for the system state. In this Bachelor's thesis, the vehicle steering agent should be improved and its functionality extended.
Starting with the existing Python/ROS implementation of the RL agent and our image processing pipeline, multiple system parts should be enhanced. On the environment side, detecting the lateral offset from the center of a lane should be improved for reliability. This will require an analysis of the current problems and some adaptations in the pipeline, possibly some hardware changes.
With more reliable lane offset values, the agent's state observation can also include it, allowing us to move further from the default PID control towards a purely RL-based steering approach. This will involve modifications to the rule population, the reward function, and potentially the learning method. Different configurations are to be implemented and evaluated in terms of their resulting performance and efficiency.
The thesis aims to shift the vehicle steering entirely to the RL agent, ideally reducing the effort for manual parameter tuning while being comparable in driving performance and computation effort.

Voraussetzungen

At LIS, we try to leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebots' control system (https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/).
More information on Duckietown can be found at https://www.duckietown.org/.
In this student work, a visualization tool for our lab should be developed. This will involve collecting data to evaluate both driving and learning performance, and visualizing the results in a graphical interface. Further, options for interaction with the learning agents controlling Duckiebot steering, speed, and platooning, should be included. An example functionality could be to change learning parameters at runtime in order to observe a difference in driving performance.
Suitable GUI frameworks and approaches to both driving and learning evaluation should be investigated as a start. The result of the thesis should be a complete visualization tool we can use for refinement of our learning agents and for demonstration purposes.

Voraussetzungen

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

Towards this goal, we need a (followup) working student who is improving the current infrastructure.

Towards this goal, the following three major tasks are necessary:

1. Developping an infrastructure to track and visualize measurement data of the platform (e.g. CPU utilization) as well as the executed application.
2. During this task also the source and periodicity of already provided data should be analyzed.
3. Setting up all Duckiebots incl. all their features and a pipeline to reflash them in case it's needed.
4. FPGA-Extension: Searching for a concept, as well as implementing it.
5. Final goal: demonstration of data exchange between NVIDIA Jetson and FPGA including protocol to specify the type of transfered data