Ear-Landmark Labeling for Dataset Generation

Overview

This is a research project which I did in close collaboration with Astrivis. One goal of this project was to create an internal tool for quickly labeling landmarks in photos of ears in order to create a labeled ear landmark dataset. The second goal was to use the created dataset in order to train a Convolutional Neural Network (CNN) model to predict said ear landmarks.

Problem Setting

Data labeling is a tedious and time consuming process. However, depending on the labeling task it can make sense to build a dedicated application which uses heuristics and domain knowledge to speed up this process.

In the context of this project, a labeling tool to quickly annotate landmarks in an unlabeled collection of ear photos was needed. This labeled dataset would then be used for training a Convolutional Neural Network (CNN) model in order to predict said landmarks.

Once, such learned models are bootstrapped, it's possible to integrate them into the labeling process, thereby further improving labeling speed.

Approach

In the specific case of ears, one can make use of the fact that the landmarks are usually ordered along contours of the ear. To this end, a simple spline-fitting approach was introduced which would help align not-yet-labeled landmarks based on labeled landmarks.

After the creation of an inital dataset, a small CNN model was trained in order to predict ear landmarks and subsequently integrated into the labeling software. This further led to an increase in labeling speed, reducing the time to label each ear significantly.

Used Technologies

This project was implemented in Python with the use of NiceGUI for creating the internal application UI.