Michael Meidinger, M.Sc.

At LIS, we try to leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebots' control system (https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/).
More information on Duckietown can be found at https://www.duckietown.org/.
In this student work, a visualization tool for our lab should be developed. This will involve collecting data to evaluate both driving and learning performance, and visualizing the results in a graphical interface. Further, options for interaction with the learning agents controlling Duckiebot steering, speed, and platooning, should be included. An example functionality could be to change learning parameters at runtime in order to observe a difference in driving performance.
Suitable GUI frameworks and approaches to both driving and learning evaluation should be investigated as a start. The result of the thesis should be a complete visualization tool we can use for refinement of our learning agents and for demonstration purposes.

Prerequisites

With technology nodes approaching their physical limit, Moore’s law becomes continually more difficult to keep up with. As a strategy to allow further scaling, chiplet-based architectures will likely become more prevalent as they offer benefits regarding development effort and manufacturing yield.

Even while reusing IP, creating an entire multi-chiplet system is still a complicated task. Following a top-down approach, a high-level simulation can help design the system architecture before going to the register transfer level. As most available simulators cater to classical SoCs, setting up a simulation for chiplet-based systems might require special attention in selecting a framework and effort in its adaptation.

This seminar work should investigate what needs to be considered when simulating chiplet-based systems compared to SoCs, what simulation frameworks are viable, and what challenges simulation for chiplets and especially their interconnect brings.

A starting point for literature could be the following paper:
https://dl.acm.org/doi/abs/10.1145/3477206.3477459

With technology nodes approaching their physical limit, Moore’s law becomes continually more difficult to keep up with. As a strategy to allow further scaling, chiplet-based architectures will likely become more prevalent as they offer benefits regarding development effort and manufacturing yield.

Even while reusing IP, creating an entire multi-chiplet system is still a complicated task. Following a top-down approach, a high-level simulation can help design the system architecture before going to the register transfer level. As most available simulators cater to classical SoCs, setting up a simulation for chiplet-based systems might require special attention in selecting a framework and effort in its adaptation.

This seminar work should investigate what needs to be considered when simulating chiplet-based systems compared to SoCs, what simulation frameworks are viable, and what challenges simulation for chiplets and especially their interconnect brings.

A starting point for literature could be the following paper:
https://dl.acm.org/doi/abs/10.1145/3477206.3477459

At LIS, we try to leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebots' control system (https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/).
More information on Duckietown can be found at https://www.duckietown.org/.
In this student work, steering Duckiebots should be realized via LCTs. Therefore, a Python implementation of the RL agent needs to be included in the Duckietown pipeline. Replacing the current controller with an RL-based one involves observing suitable sensor values and selecting reasonable actions. Different reward functions and learning methods are to be implemented and evaluated regarding their resulting performance and efficiency.
The thesis aims to shift the vehicle steering entirely to the new RL-based approach, ideally reducing computation effort.

Prerequisites

The goal of this master thesis is to implement and evaluate various topologies for a NoI. This will be done using a chiplet design for Arm-based processors configured with a standardized C2C interface supporting cross chiplet cache coherency.

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

Towards this goal, we need a (followup) working student who is improving the current infrastructure.

Towards this goal, the following three major tasks are necessary:

1. Developping an infrastructure to track and visualize measurement data of the platform (e.g. CPU utilization) as well as the executed application.
2. During this task also the source and periodicity of already provided data should be analyzed.
3. Setting up all Duckiebots incl. all their features and a pipeline to reflash them in case it's needed.
4. FPGA-Extension: Searching for a concept, as well as implementing it.
5. Final goal: demonstration of data exchange between NVIDIA Jetson and FPGA including protocol to specify the type of transfered data