HandTrackingClient::UserMessage Class Reference

Message that exposes the user name and skinning information for the user's calibrated hands. More...

#include <HandTrackingMessage.h>

Inheritance diagram for HandTrackingClient::UserMessage:

Public Types
typedef std::array< int, 3 >	Triangle
	A triangle is just 3 ints (one for each vertex).
typedef std::vector< int >	IndicesVector
typedef std::vector< float >	WeightsVector
Public Types inherited from HandTrackingClient::HandTrackingMessage
enum	MessageType {   WELCOME = 0, USER = 100, CALIBRATION = 101, POSE = 200,   PRESSED = 300, DRAGGED = 301, RELEASED = 302, MOVED = 303,   SIMULTANEOUSLY_PRESSED = 400, INDIVIDUALLY_PRESSED = 401, SIMULTANEOUSLY_RELEASED = 402, INDIVIDUALLY_RELEASED = 403,   DRAGGED_BIMANUAL = 404, POINT = 500, INVALID_DATA = 10000 }

Public Member Functions
const std::string &	getUserProfileName () const
	The name of the user's profile. More...
const std::vector< Vector3f > &	getRestPositions (int hand) const
	The positions of the hand in its rest pose (with the bones given by getRestJointFrames). Indexed by JointFrameIndex.
const std::vector< Triangle > &	getTriangles (int hand) const
	The topology of the skinned hand mesh. Indexed by JointFrameIndex.
const std::vector < IndicesVector > &	getSkinningIndices (int hand) const
	Skinning indices for the hand. See getSkinningWeights for an example of how this is used.
const std::vector < WeightsVector > &	getSkinningWeights (int hand) const
	Skinning weights for the hand. More...
std::array< Matrix4f, N_JOINTS >	getRestJointFrames (int hand) const
	The joint frames of the hand in its rest pose (as Matrix4).
std::array< Transformf, N_JOINTS >	getRestJointTransforms (int hand) const
	The joint frames of the hand in its rest pose (as Transform).
	UserMessage (const std::string &userProfileName, const std::array< std::vector< Vector3f >, N_HANDS > &restPositions, const std::array< std::vector< Triangle >, N_HANDS > &triangles, const std::array< std::vector< IndicesVector >, N_HANDS > &skinningIndices, const std::array< std::vector< WeightsVector >, N_HANDS > &skinningWeights, const std::array< std::array< Quaternionf, N_JOINTS >, N_HANDS > &restJointRotations, const std::array< std::array< Vector3f, N_JOINTS >, N_HANDS > &restJointTranslations)
	Constructs a UserMessage. Should not generally be called by users of the API.
MessageType	getType () const
std::string	serialize () const
	Serializes the message to send over the network. More...

Additional Inherited Members
Static Public Member Functions inherited from HandTrackingClient::HandTrackingMessage
static const char *	messageTypeToString (MessageType m)
	Convert from a MessageType to its string representation; used for serialization.
static MessageType	stringToMessageType (const std::string &str)
	Convert from a string to a MessageType; used in deserialization.
static HandTrackingMessage *	deserialize (const std::string &data)
	Given a line of text sent over the network connection, converts it to a valid hand message. More...
Protected Member Functions inherited from HandTrackingClient::HandTrackingMessage
	HandTrackingMessage ()

Detailed Description

Message that exposes the user name and skinning information for the user's calibrated hands.

This message will be sent every time the user profile changes. This will happen every time a new client connects, but also if the scale of the user's hands appears to change.

You can use this data to display an active cursor of the user's hand. For client applications, however, we have moved away from displaying the whole skinned hand (which can be distracting) in favor of more abstract cursors.

The skinning used by our system is "Linear blend skinning" also known as "Smooth skinning."

http://graphics.ucsd.edu/courses/cse169_w05/3-Skin.htm

Member Function Documentation

const std::vector<WeightsVector>& HandTrackingClient::UserMessage::getSkinningWeights ( int  hand ) const

inline

Skinning weights for the hand.

Given a UserMessage and a JointMessage, here is how to generate a skinned hand model (using linear blend skinning):

UserMessage* userMessage = ...;   // Received once when the profile is set; needs to be cached.
PoseMessage* poseMessage = ...;   // Received once per tracking frame.
for (size_t iHand = 0; iHand < NUMBER_OF_HANDS; ++iHand)
{
    const std::vector<IndicesVector>& skinningIndices = 
        userMessage->getSkinningIndices(iHand);
    const std::vector<WeightsVector>& skinningWeights = 
        userMessage->getWeightsVector(iHand);
    const std::vector<Vector3f>& restPositions = 
        userMessage->getRestPositions(iHand);
    const std::array<Matrix4f, N_JOINTS> currentPoseTransforms = 
        poseMessage->getJointFrames(iHand);
    const std::array<Matrix4f, N_JOINTS> restPoseTransforms = 
        userMessage->getRestJointFrames(iHand);
    
    // The rest points are relative to the rest transforms,
    // so to get the final points we need to _invert_ the
    // the rest transforms before applying the current pose.
    std::array<Matrix4f, N_JOINTS> relativeTransforms;
    for (size_t j = 0; j < N_JOINTS; ++j)
        relativeTransforms[i] = currentPose[j] * restPoseTransforms[j].inverse();

    std::vector<Vector3f> skinnedPoints = restPositions;
    const size_t nVertices = skinningIndices.size();
    for (size_t jVertex = 0; jVertex != nVertices; ++jVertex)
    {
        // Influence bones/weights for _this_ vertex:
        const IndicesVector& influenceBones = skinningIndices[jVertex];
        const WeightsVector& influenceWeights = skinningWeights[jVertex];
        const size_t nInfluences = influenceBones.size();
        assert (influenceWeights.size() == nInfluences);

        Vector3f skinnedPosition (0, 0, 0);
        for (size_t kInfluence = 0; kInfluence != nInfluences; ++kInfluence)
        {
            const float w = influenceWeights[kInfluence];
            const int b = influenceBones[kInfluence];
            skinnedPosition += w * (relativeTransforms[b] * restPositions[kVertex]);
        }
        skinnedPoints[jVertex] = skinnedPosition;
    }
}

MessageType HandTrackingClient::UserMessage::getType ( ) const

inlinevirtual

Returns the message type (e.g. PRESSED or DRAGGED). The message type is useful for distinguishing between similar messages, e.g., between PRESSED and RELEASED, which are both PinchMessages.

Implements HandTrackingClient::HandTrackingMessage.

const std::string& HandTrackingClient::UserMessage::getUserProfileName ( ) const

inline

The name of the user's profile.

In older versions of the software, we required each user to set up a unique profile that captured various parameters of his/her hand. Although this is officially still supported, we're moving away from this model, meaning that more often than not the profile will be "GenericProfile" more often than not.

std::string HandTrackingClient::UserMessage::serialize ( ) const

virtual

Serializes the message to send over the network.

Developers should not need to worry about this function unless they write their own networking code.

Implements HandTrackingClient::HandTrackingMessage.

---

The documentation for this class was generated from the following files:

• HandTrackingMessage.h
• HandTrackingMessage.cpp