TUI-interface. An IR-frame software for marker tracking

The product:
Algorithm and its realization in C# programming language for platform service operation. The platform consists of an interactive table, which serves for the realization of tangible user interface (TUI). The multi-touch system uses the infrared frame installed on top of the display. The users interact with the table by placing objects (markers) on it. They can move the markers around, rotate them, and their location serves as a base for a visualization. The algorithm and the software can be employed for any sorts of tasks, in which marker tracking of unique objects by three touching points is required.

The task of the project:
The aim of the algorithm is to identify visualized objects as points detected by the infrared frame and then to identify their location and rotation.

Specifics of the project:
At the development stage we needed to take into consideration such peculiarities as the frame’s measurement error and human errors: irregularly set markers, an accidental loss of a cursor while moving a marker around, touching the tabletop with fingers or other objects.

What we did and how we did it:
We based our project on our adaptation of a pattern-matching algorithm for two-dimensional coordinate lists written by Groth E.J.[1] We took into account that when it comes to marker tracking, the algorithm should be perceptive of the growth and reflection of patterns (similar triangles should not coincide).
The solution caused the work process to be broken into the following stages:
Marker initialization or registration;
Detecting registered markers:
-Point clustering;
-Marker identification;
Update on the markers:
-Detecting markers’ location;
-Detecting the degree of rotation;
-Updating the markers’ status.
Algorithm functions correctly with 9 markers with 3 touch points (legs): the location and rotation are indicated and detected exactly. The cases of a sudden tear-off of a leg are properly processed, preserving the markers.

[1] Groth E. J. A pattern-matching algorithm for two-dimensional coordinate lists //The astronomical journal. – 1986. – Т. 91. – С. 1244-1248.

Year of the project implementation: 2020

Usage:
The algorithm and the software we realized while working on the Phygital Platform exhibition stand. In the project we used a real PLC to demonstrate several scenarios of cyber attacks on an automotive industrial production. Scenario launch control took place on the screen with the IR-frame with the help of 3D printed markers, which were detected by the algorithm.

Project team
DEV: Sergey Polishchuk, Sergey Kamyn, Alexandra Yakimenko
PM: Valeria Fimina

Time-frame: 3 months