Reinforcement learning (RL) is one approach for solving sequential decision making problems. A reinforcement learning agent interacts with its environment and uses its experience to make decisions towards solving the problem. The technique has succeeded in various applications of operation research, robotics, game playing, network management, and computational intelligence.

This lecture provides an overview of basic concepts, practical techniques, and programming tools used in reinforcement learning. Specifically, it focuses on the application aspects of the subject, such as problem solving and implementations. By design, it aims to complement the theoretical treatment of the subject, such as mathematical derivation, convergence proves, and bound analysis, which are covered in the lecture "Approximate Dynamic Programming and Reinforcement Learning" in winter semesters.

In this lecture, we will cover the following topics:

• Reinforcement learning problems as Markov decision processes
• Dynamic programming (value iteration and policy iteration)
• Monte Carlo reinforcement learning methods
• Temporal difference learning (SARSA and Q-learning)
• Simulation-based reinforcement learning algorithms
• Linear value function approximation, e.g. tile coding

We will not cover:

• Deep Reinforcement Learning in any flavor
• Deep function approximation architectures that change during the learning process

The excessive tuning of hyper parameters exceeds the time and computational constraints of the lecture.

The goal of the lecture is, that students are able to master the application of reinforcement learning algorithms for solving continuous control problems. This involves:

• describe classic scenarios of reinforcement learning problems
• explain basics of reinforcement learning methods
• select proper reinforcement learning algorithms in accordance with specific problems, and argue their choices
• construct and implement reinforcement learning algorithms
• model an engineering problem as Markov Decision Process
• design and plan experiments
• analyze and evaluate systematically the outcome of experiments
• compare performance of the reinforcement learning algorithms covered by the course
• creating a concise report / documentation of the project and their results.

The reinforcement learning algorithms will be applied on a simulated game in groups of three.