Skip to content

supertigim/elevator_buttons_recognition

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

This project is to show how to detect and recognize buttons in an elevator for robotics.

  • Button Detection: tensorflow(<2.0) detection API
  • Button Recognition: OCR
  • Button Status (On/Off): Mean color value of each button

Environment

    $ conda create -n detection python=3.7 pyqt=5
    $ conda activate detection  
    (detection)$ git clone https://github.com/supertigim/elevator_buttons_recognition.git  
    (detection)$ cd elevator_buttons_recognition
    (detection)elevator_buttons_recognition$ pip install -r requirements.txt  
    (detection)elevator_buttons_recognition$ mkdir addons && cd addons  
    (detection)elevator_buttons_recognition/addons$ git clone https://github.com/tzutalin/labelImg.git  
    (detection)elevator_buttons_recognition/addons$ cd labelImg  
    (detection)elevator_buttons_recognition/addons/labelImg$ pip install -r requirements/requirements-linux-python3.txt
    (detection)elevator_buttons_recognition/addons/labelImg$ cd ../..
    (detection)elevator_buttons_recognition$ git clone https://github.com/tensorflow/models.git

When labelImge doesn't work properly,

    (detection)elevator_buttons_recognition/addons/labelImg$ sudo apt-get install pyqt5-dev-tools  
    (detection)elevator_buttons_recognition/addons/labelImg$ sudo pip3 install -r requirements/requirements-linux-python3.txt  
    (detection)elevator_buttons_recognition/addons/labelImg$ make qt5py3  
    pyrcc5 -o libs/resources.py resources.qrc  

Training

Before training, environment needs to be setup.

    (detection)elevator_buttons_recognition$ sudo apt-get install protobuf-compiler
    (detection)elevator_buttons_recognition$ cd models/research
      
    # Once done, Don't need to do again
    (detection)elevator_buttons_recognition/models/research$ wget -O protobuf.zip https://github.com/google/protobuf/releases/download/v3.0.0/protoc-3.0.0-linux-x86_64.zip
    (detection)elevator_buttons_recognition/models/research$ unzip protobuf.zip
    (detection)elevator_buttons_recognition/models/research$ protoc object_detection/protos/*.proto --python_out=.
      
    # For each terminal or put it in .bashrc for convenience
    (detection)elevator_buttons_recognition/models/research$ export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim

There are 5 steps with the additional step for monitoring

    # 1. Create xmls with labelImg
    (detection)elevator_buttons_recognition$ python ./addons/labelImg/labelImg.py 

    # 2. Convert xml to csv 
    (detection)elevator_buttons_recognition$  python xml_to_csv.py -i ./images/train/ -o ./annotations/train_labels.csv
    (detection)elevator_buttons_recognition$ python xml_to_csv.py -i ./images/test/ -o ./annotations/test_labels.csv

    # 3. Convert .csv to .record
    (detection)elevator_buttons_recognition$ python generate_tfrecord.py --csv_input=./annotations/train_labes.csv --output_path=./annotations/train.record --img_path=images/train/
    (detection)elevator_buttons_recognition$ python generate_tfrecord.py --csv_input=./annotations/test_labes.csv --output_path=./annotations/test.record --img_path=images/test/

    # 4. Start training   
    (detection)elevator_buttons_recognition$ python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_inception_v2_coco.config

    # (Optional) For visualization 
    (detection)elevator_buttons_recognition$ tensorboard --logdir=training

    # 5. Conversion to .pb file
    (detection)elevator_buttons_recognition$ python freeze_model.py --input_type image_tensor --pipeline_config_path ./training/ssd_inception_v2_coco.config --trained_checkpoint_prefix ./training/model.ckpt-200000 --output_directory ./frozen_model

Inference

    (detection)elevator_buttons_recognition$ python main.py -m cam      # Camera Streaming
    # or 
    (detection)elevator_buttons_recognition$ python main.py -m image    # Images Files
    # or 
    (detection)elevator_buttons_recognition$ python main.py -m video    # Video File 

The pressed button on the image is recognized in red.

To-do List

  • Press Button Detection Improvement
  • Tensorflow 2.0 Implementation using Model Garden

Reference

How to build my own button detector

Papers in regard to elevator buttons detection

ETC

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages