Reinforcement Learning Beginner to Master

Duration

2 days

What you'll learn

• Understand the Reinforcement Learning paradigm and the tasks that it's best suited to solve.
• Understand the process of solving a cognitive task using Reinforcement Learning
• Understand the different approaches to solving a task using Reinforcement Learning and choose the most fitting
• Implement Reinforcement Learning algorithms completely from scratch
• Fundamentally understand the learning process for each algorithm
• Debug and extend the algorithms presented
• Understand and implement new algorithms from research papers

Course Outline

The Markove Decision Process (MDP)

• Elements common to all control tasks
• The Markov decision process (MDP)
• Types of Markov decision process
• Trajectory vs episode
• Reward vs Return
• Policy
• State values v(s) and action values q(s,a)
• Bellman equations
• Solving a Markov decision process
• Setup - MDP in code
• MDP in code

Dynamic Programming

• Introduction to Dynamic Programming
• Value iteration
• Setup - Value iteration
• Coding - Value iteration
• Policy iteration
• Setup - Policy iteration
• Coding - Policy iteration
• Policy evaluation
• Coding - Policy iteration
• Policy Improvement
• Coding - Policy iteration
• Policy iteration in practice
• Generalized Policy Iteration (GPI)

Monte Carlo methods

• Solving control tasks with Monte Carlo methods
• On-policy Monte Carlo control
• Setup - On-policy Monte Carlo control
• Coding - On-policy Monte Carlo
• Setup - Constant alpha Monte Carlo
• Coding - Constant alpha Monte Carlo
• Off-policy Monte Carlo control
• Setup - Off-policy Monte Carlo control
• Coding - Off-policy Monte Carlo

Temporal Difference Methods

• Temporal difference methods
• Solving control tasks with temporal difference methods
• Monte Carlo vs temporal difference methods
• SARSA
• Setup - SARSA
• Coding - SARSA
• Q-Learning
• Setup - Q-Learning
• Coding - Q-Learning
• Advantages of temporal difference methods

N-step bootstrapping

• N-step temporal difference methods
• Where do n-step methods fit?
• Effect of changing n
• N-step SARSA
• N-step SARSA in action
• Setup - n-step SARSA
• Coding - n-step SARSA

Continues state spaces

• Setup - Classic control tasks
• Coding - Classic control tasks
• Working with continuous state spaces
• State aggregation
• Setup - Continuous state spaces
• Coding - State aggregation
• Tile coding
• Coding - Tile coding

Brief introduction to neural networks

• Function approximators
• Artificial Neural Networks
• Artificial Neurons
• How to represent a Neural Network
• Stochastic Gradient Descent
• Neural Network optimization

Deep SARSA

• Neural Network optimization (Deep Q-Network)
• Experience Replay
• Target Network
• Coding - Deep SARSA

Deep Q-Learning

• Setup - Deep Q-Learning
• Coding - Deep Q-Learning

Reinforce

• Policy gradient methods
• Representing policies using neural networks
• Policy performance
• The policy gradient theorem
• REINFORCE
• Parallel learning
• Entropy regularization
• REINFORCE 2
• Coding - REINFORCE

Advantage Actor-Critic (A2C)

• A2C
• Setup - A2C
• Coding - A2C

Programme scope, delivery arrangements and suitability are confirmed during enquiry.