One of my current projects involves visualizing people’s facial expressions as detected through Kinect data. I’m using Kinect SDK 1.7 and the corresponding toolkit for face tracking (Microsoft.Kinect.Toolkit.FaceTracking). The documentation for face tracking is somewhat light and lacks full information about indices of the 3D mesh received by calling Get3DShape() in a FaceTrackFrame instance (in C#). I iterated through the lines on the face—over 400!—and identified which corresponded to the mouth, eyes, and eyebrows. I also identified which made horror-movie-like lines across the opening of the mouth or over the eyes. The end result is shown below, with the scary lines removed and the desired lines in blue. (The big whitish spheres and their connections are from skeletal data.)

 

In case anyone else is interested in which lines I identified, here they are. Each line is represented as a pair of indices, where an index corresponds to the FeaturePoint value in each item of the EnumIndexableCollection returned by Get3DShape().

 

leftEyeConnections = {{21, 95}, {24, 101}, {22, 101}, {23, 103}, {23, 109}, {19, 103}, {21, 103}, {21, 105}, {107, 22}, {22, 109}, {24, 109}, {71, 23}, {71, 23}, {68, 20}, {67, 20}, {67, 97}, {67, 71}, {67, 21}, {71, 21}, {71, 105}, {23, 105}, {99, 68}, {99, 22}, {72, 22}, {68, 22}, {23, 72}, {23, 107}, {72, 107}, {68, 22}, {72, 68}, {97, 21}, {19, 95}, {99, 20}, {101, 20}, {97, 20}, {95, 20}};

rightEyeConnections = {{106, 56}, {110, 56}, {55, 110}, {102, 53}, {100, 53}, {96, 53}, {53, 96}, {102, 53}, {52, 96}, {54, 98}, {54, 96}, {100, 55}, {55, 102}, {74, 70}, {69, 73}, {74, 56}, {69, 53}, {53, 98}, {69, 54}, {106, 54}, {73, 54}, {56, 73}, {108, 56}, {73, 106}, {74, 108}, {108, 55}, {70, 55}, {100, 70}, {53, 70}, {54, 104}, {104, 56}, {52, 104}, {110, 57}, {102, 57}, {55,74}, {69,98}};

leftEyeBrowConnections = {{18, 17}, {16, 15}, {15, 18}, {16, 17}};

rightEyeBrowConnections =  {{50, 49}, {50, 51}, {49, 48}, {51, 48}};

mouthConnections = {{86, 8}, {89, 86}, {84, 40}, {85, 8}, {88, 83}, {88, 85}, {88, 81}, {84, 89}, {87, 82}, {83, 40}, {82, 89}, {81, 87}, {89, 80}, {88, 79}, {79, 33}, {80, 66}, {7, 66}, {7, 33}};

 

scaryMouthLines= {{81, 83}, {40, 87}, {87, 84}, {81, 40}, {84, 82}};

scaryEyeLines= {{72, 67}, {68, 67}, {71, 72}, {70, 69}, {69, 73}, {73, 70}, {74, 73}};