Template-based Foot Reconstruction from 3D Scans

Teaser Image of a triangle mesh of a reconstructed foot

Overview

This is a research project which I did in close collaboration with Astrivis. The goal was to fit a template 3D model of a human foot to a 3D scan, obtained with Astrivis' 3D reconstruction pipeline on the smartphone.

Problem Setting

Accurate 3D scans have the potential to allow one to take accurate measurement of body parts. Being able to extract and obtain those accurate measurements is especially interesting for the custom-fitted clothing industry, as it potentially makes the process of obtaining such measurements easier and streamlines the online purchasing process of customers.

As an example, let's assume that users were able to quickly scan their face or their feet with their smartphone to obtain a 3D model thereof. This 3D scan can then be used for the manufacturing of customized face masks (in the case of face scans) or for custom-fitted shoes or insoles (in the case of feet scans). This kind of scanning software for the smartphone is something that Astrivis provides.

However, in order to be able to take accurate measurements of 3D scans, one needs to obtain clean, noise-free and complete triangle meshes (as shown in the GIF above). This is challenging, since during the scanning process, various conditions can introduce noise, clutter and imperfections into the 3D model (in practice, we usually obtain a point cloud from which a triangle mesh needs to be generated).

Approach

One way to tackle the problem of generating a complete triangle mesh from a noisy point cloud, is to use a template mesh and gradually "deform" it until it matches perfectly the point cloud. This process is usually called registration and since the template mesh is allowed to deform arbitrarily (meaning it's not just scaling, rotating or translating it), we call it a non-rigid registration.

The below video demonstrates this registration process for the case of reconstructing feet. A template model is initialized close to the scanned foot at the beginning of the process. Afterwards the model is gradually deformed in a non-rigid way and iteratively fitted to the points of the scan through some variant of the iterative closest point algorithm (ICP).

Used Technologies

This project was implemented in C++ with the use of the Ceres library for the optimization part.