Dueling DDQN

🚧👷🛑 Under Construction!!!

Table of Contents

1. Overview
2. Setup
3. Results
4. Analysis

Overview

This repository contains an implementation of Dueling Double Deep Q-Network (Dueling DDQN) learning using PyTorch.

Setup

Required Dependencies

It's recommended to use a Conda environment to manage dependencies and avoid conflicts. You can create and activate a new Conda environment with the following commands:

conda create -n rl python=3.11
conda activate rl

After activating the environment, install the required dependencies using:

pip install -r requirements.txt

Running the Algorithm

You can run the algorithm on any supported Gymnasium Atari environment with a discrete action space using the following command:

python main.py --env 'MsPacmanNoFrameskip-v4'

Command-Line Arguments

• Environment Selection: Use -e or --env to specify the Gymnasium environment. The default is None, so you must specify an environment.

  Example:

  python main.py --env 'PongNoFrameskip-v4'

• Number of Learning Steps: Use --n_steps to define how many training steps the agent should undergo. The default is 1,000,000 steps.

  Example:

  python main.py --env 'BreakoutNoFrameskip-v4' --n_steps 200000

• Parallel Environments: Use --n_envs to specify the number of parallel environments to run during training. The default is 32 environments, optimizing the training process.

  Example:

  python main.py --env 'AsterixNoFrameskip-v4' --n_envs 16

• Continue Training: Use --continue_training to determine whether to continue training from saved weights. The default is True, allowing you to resume training from where you left off.

  Example:

  python main.py --env 'AsteroidsNoFrameskip-v4' --continue_training False

Using a Conda environment along with these flexible command-line options will help you efficiently manage your dependencies and customize the training process for your specific needs.

Results

AirRaid	Alien	Amidar

Assault	Asterix	Asteroids

Atlantis	BankHeist	BattleZone

BeamRider	Berzerk	Bowling

Boxing	Breakout	Carnival

Centipede	ChopperCommand	CrazyClimber

Defender	DemonAttack	DoubleDunk

ElevatorAction	Enduro	FishingDerby

Freeway	Frostbite	Gopher

Gravitar	Hero	IceHockey

JamesBond	JourneyEscape	Kangaroo

Krull	KungFuMaster	MontezumaRevenge

MsPacman	NameThisGame	Phoenix

Pitfall	Pong	Pooyan

PrivateEye	Qbert	Riverraid

RoadRunner	Robotank	Seaquest

Skiing	Solaris	SpaceInvaders

StarGunner	Tennis	TimePilot

Tutankham	UpNDown	Venture

-

VideoPinball	WizardOfWor	YarsRevenge

Zaxxon	Adventure

Analysis

Acknowledgements

Special thanks to Phil Tabor, an excellent teacher! I highly recommend his Youtube channel.