This is a project preseneted by Prof. Dr. Andreas Kist at FAU. The aim of this project is to detect the precise location of all Hexbugs in videos. We are provided with 100 videos, containing ~ 101 frame each, alognside the annotations of Hexbugs' heads' location at each frame.
Given data are in mp4 format. We need to extract every single frame and save it in specific directories.
We are trying to detect the Hexbugs in the each frame using Yolo v5 algorithm. By doing so, we are left with cropped images containing the Hexbugs. By doing so, we got rid of void disturbing areas in the image.
After obtaining the cropped images, we use Resnet50 model for feature extraction, and then we add Dense layers at the top of the model to perform regression. Now, we have created a model that is capable of estimating the head's coordination.
After computing the head's position, we need to reverse the process to retrieve head's original coordination in the original frame.
- Clone this git repository.
git clone https://github.com/FarzamTP/TRACO-HexBug.git- Enter the directory
cd TRACO-HexBug- [Pre-processing frames]
jupyter notebook frame_prepration.ipynbjupyter notebook hexbug_object_detection_YOLO_torch.ipynb- [Training]
jupyter notebook hexbug_head_detection_resnet.ipynb- [Load Model]
jupyter notebook load_model.ipynbTo download the annotated dataset for Yolo, you need an api_key. Please use your Roboflow account to export the dataset using your key.