Estimating the position of vertexes in a 3D model

Question

Introduction to my project:

We are machine learning scientists and working on a biomedical system for tracking tongue for speech pathology using a technology called EMA (Electromagnetic articulography). In EMA a coil is attached to tongue surface using a glue and its 3D pos and orientation (roll and pitch) are read by EMA measurement system. So, you can track the coil and thus a point on the surface of tongue while speech is made. Also let's assume we have a low density 3D model of tongue for the patient is obtained by 3D MRI and we know exactly that which vertex in 3D model corresponds to the point that coil is attached.

It would be very valuable for our research if we can visualize the tongue for the patient. But the issue is that we just track 1 or 2 points on the tongue but our 3D model has more than 100 vertices which are not tracked. Now, we think it might be possible to estimate the position of those point given these two points.

Here are some of the facts that we think it should be possible:

1. Although tongue is not a solid object, it has a degree of rigidity thus the position of these 100 vertexes are correlated by physics For example how much they can be stretched or bent (Something like spring and mass model in physics).
2. by looking at a training set of 3D models for tongue during production of different sounds, we might say that the position of vertexes are not pure random and there is a distribution for them which might be estimated using the training dataset (sequence of MRI 3D models).
3. we might assume that in our presentation the tongue can have just a bent toward the palate to simplify the modeling problem. or a 2D profile of tongue (like the line at the center) is enough for modeling. or any simplification that we can make.

Now my question:

What do you know in computer graphics literature which resembles to our problem (any model, method, tool, software, library,....)? What is your suggestion for us to do this task in more efficient way?

Answer 1

If you can construct a mesh that is representative of the tongue, like a spring mass system, then you can attach 'hooks' at your two control points and get a close approximation of the deformation of the tongue.

Answer 2

If you have a good understanding of how the tongue deforms while speaking (presumably some part of it remains relatively fixed and other parts get a curvature), you might design a simple empirical model with a few degrees of freedom that maps any point in 3D space to the corresponding point after deformation.

For instance, you might map a line segment on a symmetry axis along the resting tongue to a parabola, so the horizontal plane by this segment to a parabolic cylindre, and all parallel planes to "parallel" parabolic cylinders. (This is just a simple example, you know better than me.)

I guess that you can focus on a median section, so that the problem is 2D and easier. From the 2D modelling, the shape of the transverse sections might be estimated from the position of the section along the tongue and the thickness after deformation.

Once you have this deformation model, you can use the tracked point(s) to determine the free parameters of the model (the number of degrees of freedom needs to match the number of independent measurements that you make).

Then when you have the parameters, you can compute the deformed positions of any vertex of the mesh.