IconMatch

Easily select icons on the screen in any environment.

This is part of the Hands Free Computing project. Built with OpenCV 3.12 and Python 3.8.

Table of Contents

• Installation
• Usage
• API
• Roadmap
• Contributing
• License
• Contact

Installation

1. Clone the repo and open it locally:

$ git clone https://github.com/luiszugasti/IconMatch/
$ cd IconMatch

2. Install the requirements:

$ pip install -r requirements.txt

Usage

You can use the functions as shown in demo.py as a default entry point.

In the below example, the main set of functions is called within a callback function, as this allows the threshold value to be controlled from a GUI in OpenCV.

import cv2 as cv

from icondetection.demo.demo import render_rectangles, candidate_rectangle_demo
from icondetection.box import grayscale_blur, canny_detection, group_rects

src = cv.imread("source to your image file")

def threshold_callback(val):
    """
    Takes a value of threshold for the canny edge detector and finds the
    bounding rectangles of appropriate edges within an image.
    """

    # accept an input image and convert it to grayscale, and blur it
    gray_scale_image = grayscale_blur(src)

    # determine the bounding rectangles from canny detection
    _, bound_rect = canny_detection(gray_scale_image, min_threshold=val)

    # group the rectangles from this step - variable is global for demo purposes
    global grouped_rects
    grouped_rects = group_rects(bound_rect, 0, src.shape[1])

    # (for display purposes) use the provided rectangles to display in your program
    render_rectangles(grouped_rects, src.copy(), "Grouped Rectangles", desired_color=(36, 9, 14))
    render_rectangles(bound_rect, src.copy(), "Original Rectangles", desired_color=(96, 9, 104))
    candidate_rectangle_demo()

Key Features

• Detection of areas with a high likelihood of being clickable icons.
• Detection of closest rectangle to point of interest (be it gaze, or mouse as in the examples)

API

The current available APIs encompass what your image processing pipeline should contain. Both APIs are currently still experimental as I learn more about OpenCV and optimize code.

canny_detection(gray_scale_image, min_threshold)

Performs canny detection when given a gray scale image and a minimum threshold for hysteresis. Returns bounding rectangles of points of interest.

group_rects(bound_rectangles, initial_scanning_range, final_scanning_range)

Groups rectangles that are overlapping in two-dimensional space and returns their conglomerate components.

Roadmap

• Detect regions of interest with moderate accuracy
• Detect candidate region based on proximity
• Detect icon-like objects on the screen
• Context provision into regions of interest

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are genuinely appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Luis Zugasti - @luis__zugasti

Project Link: https://github.com/luiszugasti/IconMatch