Hand Skeleton Pose Provider
Script that serves the purpose of providing poses (positions and rotations) of the hand bones. Poses are most likely to come from a hand tracking source, but they can in theory come from any backing source - it is up to the behaviours that inherit from HandSkeletonPoseProvider to calculate or read hand poses however and from what ever source they choose to
This is an abstract class, it is mostly implemented by more specific input data providing scripts, but there are other uses derived from it
In addition to hand pose it provides the hand scale as well. This scale is usually read from a hand tracking backend and can be used to scale the user's hand in virtual environment to best approximate the size of the user's virtual hand to match with the size of the real one. Again, this isn't something implemented in the HandSkeletonPoseProvider, but rather it is up to the derived behaviours to determine scale
HandSkeletonPoseProvider is meant to provide poses for one hand - one pose provider provides poses of hand bones that belong to one particular hand, left or right
Logic contained in this script mostly revolves around transforming hand bone poses from local space to world space (relative poses to absolute poses) and/or vice-versa, inheriting behaviours can use certain members detailed below to set absolute or relative bone poses and the hand scale as they see fit
Public properties
float Scale { get; }
Scale of the target hand. This property is also exposed in the inspector of the pose provider inside Unity editor and it can be assigned there, but this is mostly for trying out different scales and observe the effect of changing it in the editor as the value of scale is usually set by scripts derived from pose provider
UnityEvent OnPoseDataUpdated { get; }
Event sent whenever hand poses get updated using one of the methods described bellow. Visible in the inspector window as well
Public methods
abstract HandType GetHandType()
This method is used to determine for which hand does the pose provider provide poses, left or right
HandBoneKeyedReadOnlyCollection<Pose> GetBoneRelativePoses()
Returns a read-only collection indexed by zero-based integer or a HandBoneId that contains relative poses for bones at the corresponding indices. Every bone pose in this list is relative to the bone's parent bone pose. In case of the Wrist bone (index 0), pose is specified relative to world, i.e. it is an absolute bone pose
HandBoneKeyedReadOnlyCollection<Pose> GetBoneAbsolutePoses()
Returns a read-only collection indexed by zero-based integer or a HandBoneId that contains absolute poses for bones at the corresponding indices - every bone pose in this list is specified in world space
HandBoneKeyedReadOnlyCollection<Pose> GetBoneAbsoluteScaledPoses()
This is the same as GetBoneAbsolutePoses, but bone positions are scaled by the Scale of the pose provider. Bone positions are scaled with the position of the root bone (Wrist) used as the origin of the scale, therefore Wrist has the same position in this list as it does in both lists returned from GetBoneRelativePoses and GetBoneAbsolutePoses methods
Protected properties
float Scale { set; }
Property setter of the previously described Scale property, derived scripts can set the scale to provide using this
Protected methods
void SetBoneRelativePoses(IReadOnlyList<Pose> poses)
Method that should be called by the derived classes in order to set the relative bone poses. The list of bone poses passed as argument should contain at least 26 Pose objects, one for every bone in a hand skeleton, an exception is thrown if otherwise. This method, when completed successfully, sends out the OnPoseDataUpdated event
void SetBoneAbsolutePoses(IReadOnlyList<Pose> poses)
Has completely the same effect as SetBoneRelativePoses, with the obvious exception that this method sets the absolute bone poses directly and relative poses are calculated based on these
virtual void OnGetBonePoses()
This method serves as callback for derived behaviours - by overriding it, the derived behaviours can perform their own custom logic and changes on poses just before the HandSkeletonPoseProvider calculates and returns the requested bone poses. This method gets called from GetBoneRelativePoses, GetBoneAbsolutePoses and GetBoneAbsoluteScaledPoses methods, i.e. whenever some code wants to obtain current hand bone poses
Additional notes
It should be noted that the HandSkeletonPoseProvider calculates poses in a deferred fashion. To better explain - calling SetBoneRelativePoses/SetBoneAbsolutePoses will not immediately recalculate bone absolute/scaled/relative poses. These methods just copy the provided poses to the corresponding lists. Only when one of the GetBone(Relative/Absolute/Scaled)Poses methods get called are the corresponding bone poses calculated. This way pose provider avoids redundant pose calculation in case poses get set multiple times before any of the methods for getting them are called, as all the calculations are deferred until poses are actually needed. This also has the side-effect in that the SetBoneRelativePoses and SetBoneAbsolutePoses mutually override each other: calling the latter will overwrite relative bone poses set by the former and vice-versa.
Behaviours that derive from HandSkeletonPoseProvider can for example override OnGetBonePoses method in order to obtain pose data from their sources, process it as they see fit and then set those poses and scale to the base pose provider using either SetBoneRelativePoses or SetBoneAbsolutePoses method. HandSkeletonPoseFilter and HandSkeletonSourcedPoseProvider scripts provide good examples of this kind of pattern for implementing HandSkeletonPoseProvider behaviour
Hand Skeleton Sourced Pose Provider
Special type of HandSkeletonPoseProvider that works with a HandSkeleton which is required to be attached to the same GameObject that the pose provider is attached to
It reads the poses from the associated HandSkeleton and provides those poses to other behaviours that use it as the source for hand poses - other HandSkeletons mostly
Since the HandSkeleton does not have to have all the possible unique HandBones attached, it has certain requirements - the associated HandSkeleton must have its own pose provider assigned, otherwise the HandSkeleton must be complete, i.e. have all 26 bones attached. If this is not met then the HandSkeletonSourcedPoseProvider will basically act as if it was disabled, since it won't be able to extrapolate poses for the bones which are not present in the associated HandSkeleton
Using this pose provider enables the possibility of chaining HandSkeletons, one serving as the source of poses for the next one
To provide the concrete example for this, you can have one HandSkeleton which uses certain InputDataProvider that provides it with hand poses which are read from some input source. Then you can have another HandSkeleton, for example one with hand mesh to visualize the first HandSkeleton with. In that case you would probably want to attach HandSkeletonSourcedPoseProvider to the first HandSkeleton and then assign that pose provider to the other HandSkeleton. That way HandSkeleton with mesh will always have its pose exactly matched with the source HandSkeleton
Hand Skeleton Pose Filter
Another special type of HandSkeletonPoseProvider, it can be used as a filter for bone poses obtained from one pose provider that the HandSkeletonPoseFilter processes/filters in certain way and then those filtered poses can be provided to other behaviours that need to read hand poses
It defines SourcePoseProvider property that accepts a HandSkeletonPoseProvider reference. This pose provider will act as a source of hand poses that get read by the pose filter and at later point processed by it. Since the pose filter is itself a HandSkeletonPoseProvider, it can be assigned to any other behaviour that needs to read hand poses
If no SourcePoseProvider is assigned then nothing is performed and poses that will be provided are the same as those provided by the pose filter the last time it was requested to provide. The same behaviour also occurs if the HandSkeletonPoseFilter is inactive (its GameObject is inactive)
If the HandSkeletonPoseFilter component is disabled then the source bone poses will not be processed and they will simply be provided as they are provided by the source pose provider, i.e. they will just pass through the filter unchanged
This is an abstract class and it requires that the method for processing source poses be implemented by derived classes - GetFilteredBoneRelativePoses. This method is called by the base pose filter whenever it gets requested to provide bone poses. Implementing methods are passed collection of bone poses to which they can write the filtered poses which are expected to be specified relative to their parent bone poses (except for the Wrist pose which is assumed to be in world space)
HandSkeletonPoseFilters can be chained - one can be the source pose provider for the next one, so multiple independent pose filters can all process an original bone poses obtained from an input source one after the other before they are finally provided to a HandSkeleton, for example
Last updated