Skip to content

Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets

License

Notifications You must be signed in to change notification settings

BarisYazici/deep-rl-grasping

Repository files navigation

Deep Reinforcement Learning on Robotics Grasping

Train robotics model with integrated curriculum learning-based gripper environment. Choose from different perception layers depth, RGB-D. Run pretrained models with SAC, BDQ and DQN algorithms. Test trained algorithms in different scenes and domains.

Master's thesis PDF

Prerequisites (CPU)

Install anaconda. Start a clean conda environment.

conda create -n grasp_env python=3.6
conda activate grasp_env

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Prerequisites (GPU)

conda create -n grasp_env python=3.6
conda activate grasp_env
conda install -c conda-forge cudatoolkit=10.0 cudnn=7.6.5
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/

In setup.py change:

'tensorflow==1.14.0',
to
'tensorflow_gpu==1.14.0',

Installation

Use pip to install the dependencies.

pip install -e .

If using GPU you can check if it was successful with: ´´´ python -c "import tensorflow as tf; print(tf.config.experimental.list_physical_devices('GPU'))" ´´´

Run Models

train_stable_baselines script provides the functionality of running and training models.

For running models 'manipulation_main/training/train_stable_baselines.py' takes the following arguments

  • --model - trained model file e.g trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip
  • -t - use test dataset if not given runs on training dataset
  • -v - visualize the model (faster without the -v option)
  • -s - run stochastic model if not deterministic

For running functionality run sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py run --model trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip -v -t

Train models

For training models 'manipulation_main/training/train_stable_baselines.py' takes the following arguments

  • --config - config file (e.g 'config/simplified_object_picking.yaml' or 'config/gripper_grasp.yaml')
  • --algo - algorithm to use(e.g BDQ, DQN, SAC, TRPO)
  • --model_dir - name of the folder to host the trained model logs and best performing model on validation set.
  • -sh - use shaped reward function (Only makes sense for Full Environment version)
  • -v - visualize the model

For training functionality train sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Running the tests

To run the gripperEnv related test use

pytest tests_gripper
  • Domain and Scene Transfer

  • Different Perception Layers

  • Ablation Studies

  • Training Environment

  • Domain transfer performance

Authors

Citing the Project

To cite the master's thesis:

@MastersThesis{Yazici2020,
    author     =     {Yazici Baris},
    title     =     {{Branch Dueling Deep Q-Networks for Robotics Applications}},
    school     =     {Technical University of Munich},
    year     =     {2020},
    howpublished = {\url{https://github.com/BarisYazici/tum_masters_thesis}}
}

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments

About

Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages