Neuro RL

University of Amsterdam Neuro-AI Summer School, 2024

made by: Tom George (UCL) and Jesse Geerts (Imperial)

In this tutorial we'll study and build reinforcement learning models inspired by the brain. By the end you'll understand, and be able to construct, a series of simple but surprisingly powerful models of how agents learn to navigate spatial environments and find rewards.

Note: the colab renders better in Safari and Firefox than Chrome.

Figure 1: An agent has learn to navigate around a wall towards a hidden reward using place cell state features and a simple Q-value learning algorithm.

Topics covered:

1. Rescorla-Wagner Model (~60 mins)
2. Temporal Difference Learning (~60 mins)
3. Q-Values and Policy Improvement (~60 mins)
4. State features and function approximation (~60 mins)

Solutions

Solutions to the maths exercises can be found in a seperate solutions.ipynb notebook which may or may not be provided to you by the TAs.