Control for Robotics: from Optimal Control to Reinforcement Learning

Master Wahlmodul, Advanced Control Course with Machine Learning Elements

This course presents optimal control, learning-based control, and reinforcement learning principles from the perspective of robotics applications. The course covers the foundations of optimal control and derives practical control algorithms that leverage first-principle robot models as well as data collected from the robot system. Real-world challenges such as disturbances, state estimation errors and model errors are addressed, and adaptive and reinforcement learning approaches are derived to address these challenges.

Learning Objectives

At the end of the course, students are able to:

• derive optimal controller equations for mobile robots,
• implement optimal controllers in practice and analyse their properties,
• understand state-of-the-art approaches to learning-based control and reinforcement learning,
• design learning-based controllers to cope with non-idealities such as model errors.

Prerequisites

Motivation and interest in robot learning and control, as well as knowledge in dynamic systems, control theory (including state-space approaches), calculus, probability theory, linear algebra, and programming (Matlab and Python). Additional knowledge and courses on the following topics will be beneficial: basics of optimization, mobile robotics, machine learning, and optimal control.

Teaching and Learning Approach 

The core components of this course are:

• Lectures (introducing the core concepts)
• In-Class Exercises and Tutorials (working through examples)
• Optional assignments (practising the application of the concepts to practical problems and gaining hands-on implementation experience)

Evaluation

The students will demonstrate their knowledge in a two-hour written final exam at the end of the course. The exam will assess the students’ understanding of optimal control, learning-based control, and reinforcement learning, with a focus on applying these approaches in robotics. Exam questions will cover the lecture content as well as the practical content from the course assignments.