ThreeGear.UserMessage Class Reference

Message that exposes the user name and skinning information for the user's calibrated hands. More...

Inheritance diagram for ThreeGear.UserMessage:

Public Member Functions
	UserMessage (string userProfileName, Point3D[][] restPositions, int[][][] triangles, int[][][] skinningIndices, float[][][] skinningWeights, Quaternion[][] restJointRotations, Vector3D[][] restJointTranslations)
override string	Serialize ()

Static Public Member Functions
static UserMessage	Parse (TokenStream tokenStream, MessageType type)
Static Public Member Functions inherited from ThreeGear.HandTrackingMessage
static HandTrackingMessage	Deserialize (string data)
	Factory method that parses a HandTrackingMessage from a string More...

Properties
string	UserProfileName [get]
	The name of the user's profile. More...
Point3D[][]	RestPositions [get]
	The positions of the hand in its rest pose (with the bones given by RestJointFrames). More...
Matrix3D[][]	RestJointFrames [get]
	The joint frames of the hand in its rest pose. More...
Quaternion[][]	RestJointRotations [get]
	The joint frames of the hand in its rest pose. More...
Vector3D[][]	RestJointTranslations [get]
	The joint frames of the hand in its rest pose. More...
int[][][]	Triangles [get]
	A triangle is just 3 ints (one for each vertex). More...
int[][][]	SkinningIndices [get]
	Skinning indices for the hand. More...
float[][][]	SkinningWeights [get]
	Skinning weights for the hand. More...
Properties inherited from ThreeGear.HandTrackingMessage
MessageType	Type [get]
	The message type is useful for distinguishing between similar messages, e.g., between PRESSED and RELEASED, which are both PinchMessages. More...

Additional Inherited Members
Public Attributes inherited from ThreeGear.HandTrackingMessage
const int	N_JOINTS = 17
	The number of joints in the hand model, currently 17 (arm, wrist, and 3 each for the 5 fingers). More...
const int	N_FINGERS = 5
	The number of fingers is five (thumb, index, middle, ring, pinky). More...
const int	N_HANDS = 2
	Two hands (left and right). More...
const int	N_POSES = 7
	The number of specifically recognized poses. More...
const int	N_FINGER_DOFS_PER_HAND = 16
	The number of finger degrees of freedom per hand. More...
Protected Member Functions inherited from ThreeGear.HandTrackingMessage
	HandTrackingMessage (MessageType type)
	HandTrackingMessage (HandTrackingMessage msg)
string	ToString (double x)
string	ToString (int i)
string	ToString (Vector3D v)
string	ToString (Quaternion q)

Detailed Description

Message that exposes the user name and skinning information for the user's calibrated hands.

This message will be sent every time the user profile changes. This will happen every time a new client connects, but also if the scale of the user's hands appears to change.

You can use this data to display an active cursor of the user's hand. For client applications, however, we have moved away from displaying the whole skinned hand (which can be distracting) in favor of more abstract cursors.

The skinning used by our system is "Linear blend skinning" also known as "Smooth skinning." http://graphics.ucsd.edu/courses/cse169_w05/3-Skin.htm

Property Documentation

Matrix3D [][] ThreeGear.UserMessage.RestJointFrames

get

The joint frames of the hand in its rest pose.

Quaternion [][] ThreeGear.UserMessage.RestJointRotations

get

The joint frames of the hand in its rest pose.

Vector3D [][] ThreeGear.UserMessage.RestJointTranslations

get

The joint frames of the hand in its rest pose.

Point3D [][] ThreeGear.UserMessage.RestPositions

get

The positions of the hand in its rest pose (with the bones given by RestJointFrames).

int [][][] ThreeGear.UserMessage.SkinningIndices

get

Skinning indices for the hand.

See SkinningWeights for an example of how this is used.

float [][][] ThreeGear.UserMessage.SkinningWeights

get

Skinning weights for the hand.

This example shows how to use the skinning weights to generate a hand mesh.

for (int iHand = 0; iHand < HandTrackingMessage.N_HANDS; ++iHand)
{
    Point3D[] restPositions = userMessage.RestPositions[iHand];
    Matrix3D[] currentPoseTransforms = poseMessage.JointFrames[iHand];
    Matrix3D[] restPoseTransforms = userMessage.RestJointFrames[iHand];

    handVertices[iHand] = new VertexPositionNormalTexture[restPositions.Length];

    // The rest points are relative to the rest transforms,
    // so to get the final points we need to _invert_ the
    // the rest transforms before applying the current pose.
    Matrix3D[] relativeTransforms = new Matrix3D[restPoseTransforms.Length];
    for (int j = 0; j < restPoseTransforms.Length; ++j)
    {
        // A little clumsy due to limitations of the MS Matrix class, but what we're
        // computing here is 
        //     relativeTransform = currentPose * inv(restPose)
        relativeTransforms[j] = new Matrix3D();
        relativeTransforms[j].Append(restPoseTransforms[j]);
        relativeTransforms[j].Invert();
        relativeTransforms[j].Append(currentPoseTransforms[j]);
    }

    for (int jVertex = 0; jVertex < handVertices[iHand].Length; ++jVertex)
    {
        Point3D restPos = restPositions[jVertex];

        int[] influenceBones = userMessage.SkinningIndices[iHand][jVertex];
        float[] influenceWeights = userMessage.SkinningWeights[iHand][jVertex];
        int nInfluences = influenceBones.Length;

        // Sum up all the bone influences:
        Point3D sumPoint = new Point3D(0, 0, 0);
        for (int kInfluence = 0; kInfluence != nInfluences; ++kInfluence)
        {
            float w = influenceWeights[kInfluence];
            int b = influenceBones[kInfluence];
            Point3D xfPoint = relativeTransforms[b].Transform(restPositions[jVertex]);
            sumPoint.X += w * xfPoint.X;
            sumPoint.Y += w * xfPoint.Y;
            sumPoint.Z += w * xfPoint.Z;
        }

        handVertices[iHand][jVertex].Position = new Vector3((float) sumPoint.X, (float) sumPoint.Y, (float) sumPoint.Z);
    }
}

int [][][] ThreeGear.UserMessage.Triangles

get

A triangle is just 3 ints (one for each vertex).

string ThreeGear.UserMessage.UserProfileName

get

The name of the user's profile.

In older versions of the software, we required each user to set up a unique profile that captured various parameters of his/her hand. Although this is officially still supported, we're moving away from this model, meaning that more often than not the profile will be "GenericProfile" more often than not.

---

The documentation for this class was generated from the following file:

• UserMessage.cs