Hand Skeleton

• Script in OctoXR that serves as logical representation of a left or right hand. Representing the whole hand from the wrist appropriately requires multiple GameObjects, so the HandSkeleton script comes in tandem with HandBone script

• HandBone script is attached to a GameObject that represents a particular hand bone and then it is added to a HandSkeleton where with other HandBones it makes up the HandSkeleton

• HandSkeleton requires a HandBone attached to the same GameObject it gets attached to and that particular HandBone always represents a Wrist bone of the HandSkeleton. Consequentially, HandSkeleton always contains the root hand bone and is itself also part of the root of a hand. Another consequence of this is that the HandSkeleton cannot be added to a GameObject that already has HandBone component which is added to an existing HandSkeleton, if that happens HandSkeleton gets destroyed and an appropriate exception is thrown

• It has a HandSkeletonPoseProvider associated that can be assigned either from a script or using the HandSkeleton's inspector panel

• What the HandSkeleton essentially does is it reads the hand poses from the assigned pose provider and makes sure all of its HandBones have poses that match those obtained from its pose provider. The simplest and most straightforward way to achieve this is to simply set the HandBone's Transform position, rotation and scale - this is what most types of HandSkeletons defined in OctoXR do

• HandSkeleton is abstract and needs to be implemented by a more specific HandSkeleton script that can drive the bone poses in whatever exact way it wants to

• It does provide a number of protected methods that can be used for HandSkeleton pose calculations and updating in order to make it more easy for the inheriting behaviours

• Improtant to note is that the HandSkeleton script is executed in edit mode too by default. There are a number of overridable callbacks that can be implemented by the derived scripts and they get executed in edit mode as well so it should be kept in mind if implementing a custom HandSkeleton. All of the public properties and methods are safe to use in both edit- and run-time. This goes for all input and physics related scripts as well as all the specialized HandSkeletons and HandBones

• Similar to some of the input related scripts, HandSkeleton does not update its HandBone Transforms on its own, it is up to derived behaviours to do so. HandSkeleton just provides the properties and methods that can be used in order to make that easier as well as the common set of properties and options for all hand skeletons which should be applied when updating HandBones

• Some other more specific HandSkeleton and HandBone derived scripts are described in more detail at the bottom sections of this page, with the exception of physics related ones, those are described on their own dedicated pages

Public properties

HandSkeletonPoseProvider PoseProvider { get; set; }

• Pose provider to read the target hand poses from. All of the HandSkeletons defined in OctoXR at the moment will not perform any action regarding their HandBones' poses if this is not assigned, so it probably makes sense to always assign this property, in most cases this will be assigned to a InputDataProvider

HandType HandType { get; set; }

• Defines for which hand the hand skeleton updates bone poses, left or right. This property can be changed only if there is no pose provider assigned, otherwise the value of the property will be equal to that of the pose provider's hand type

bool ApplyPoseProviderScale { get; set; }

• Specifies whether to scale the HandSkeleton with the scale provided by the pose provider (Scale property of the HandSkeletonPoseProvider)

float Scale { get; set; }

• Scale of the HandSkeleton. This scale is multiplicative with the scale provided by the pose provider, e. g. if the HandSkeleton's ApplyPoseProviderScale is set to true and the assigned pose provider's scale is 2 and the HandSkeleton's Scale is set to 3, then total scale to apply to the HandSkeleton is 2 x 3 = 6

UnityEvent OnPoseUpdated { get; }

• Event sent after HandSkeleton's pose (this includes all HandBones' poses attached to the HandSkeleton too) has been updated

Transform Transform { get; }

• Cached reference to the Transform component of the HandSkeleton. Naturally this property is not exposed in the HandSkeleton's inspector panel as the HandSkeleton is the one that takes care of assigning this property to the correct value

HandBoneKeyedSparseReadOnlyCollection<HandBone> Bones

• Collection of HandBones attached to the HandSkeleton

• Important thing to mention is that the HandSkeleton does not need to contain all the possible hand bones, meaning this collection can contain anywhere between 1 and 26 elements. As mentioned earlier, Wrist bone is always contained in a HandSkeleton, so this collection always contains at least one bone

• This list can also be indexed into by a HandBoneId or a zero-based integer, however there is one improtant distinction between this type of collection and the ones used in some of the input related scripts described on previous pages. Because the HandSkeleton does not necessarily contain all of the unique hand bones, it is not safe to index into this collection using any integer value corresponding to a HandBoneId. To clarify with an example, if the HandSkeleton contains only one bone along with the Wrist bone, e. g. MiddleIntermediate, the index at which that bone is located in this collection is 1, despite its identity corresponding to integer value of 13 - trying to index into it with 13 would in this case result in an out of range exception being thrown. It is completely safe to index into it with any HandBoneId, but in that case null will be returned for bones not contained in the HandSkeleton

• This is an ordered collection - HandBones are always ordered by their HandBoneId (numerical value of it), so it does not matter in which order HandBones are added, they may not remain at their initial indices. Again, because of the requirement for Wrist bone's constant presence, this does not apply to it - it is always at index zero

• There are many methods on this collection to easily check for the presence of a particular hand bone or for trying to retrieve the reference to one or for retrieving all the bones from it that belong to a particular hand finger etc.

• Adding and removing bones from it is possible, but methods for those operations are defined on the HandSkeleton, not the collection. Methods are detailed in text further below

• In the inspector panel this collection is represented as the fixed length list with bone identities as keys and GameObjects representing bones (GameObjects with a HandBone scripts attached). GameObjects that should act as HandSkeleton's bones can be assigned there and they can be removed as well - doing so in the Unity editor will cause all of the appropriate callbacks and methods to be called along with any properties on the HandSkeleton and HandBones involved to be initialized/uninitialized as they would be during run-time

HandSkeletonBoneAddedEvent OnBoneAdded { get; }

• Event sent when a new hand bone is added to the HandSkeleton. Identity of the newly added bone is sent as the event argument

HandSkeletonBoneRemovedEvent OnBoneRemoved { get; }

• Event sent when a hand bone gets removed from the HandSkeleton. Identity of the removed bone is sent as the event argument

bool IsComplete { get; }

• Indicates whether the HandSkeleton has all the hand bones that it can possibly have. All hand bones, their identities and other relevant information is listed in a table on this page. This property is not visible in the HandSkeleton's inspector

Public methods

void AddBone(HandBoneId boneId, GameObject boneObject)

• Adds the specified GameObject to act as a hand bone with the specified identity to the HandSkeleton

• There are a number of validations performed by this method and exceptions thrown if any validation check fails - if the specified GameObject already acts as a hand bone in another HandSkeleton or it is already a bone in the same HandSkeleton among others

• There is also the requirement that the specified GameObject must be in the same scene as the HandSkeleton. This means you cannot add a prefab asset object as a bone to a HandSkeleton in any scene you may be currently editing. This may reduce some flexibility, but it is required in order to keep the state of the HandSkeleton as well as the HandBones added to it valid

• GameObject that is successfully added gets a HandBone script attached to it. Depending on the exact type of the HandSkeleton it may be some more specific HandBone derived script

• Also note that the GameObject can have a HandBone attached already, it will get reused (with its identity possibly changed), but as mentioned above it must not be already added to another HandSkeleton, only not used HandBones are valid. Exact behaviour depends on the concrete implementation of the HandSkeleton, some HandSkeletons that require a specific HandBone derived objects may in turn destroy the existing HandBone on the specified GameObject and then add the required specific one to it

bool RemoveBone(HandBoneId bone)

• Removes the bone with the specified identity contained in the HandSkeleton from it. Return value indicates whether the HandSkeleton did or did not have a HandBone with the specified identity in it

• This operation does not destroy HandBone component as it may be reused later by the same or some other HandSkeleton

• Wrist bone cannot be removed, an exception is thrown if trying to do so

void RemoveBoneAt(int index)

• This method is the same as the previous one, it just takes a zero-based index as argument and removes the HandBone located at that index in the HandSkeleton's collection of bones, though it will generate error if the index specified is out of bounds for the HandSkeleton's bone collection and also if the specified index is 0 (root bone)

void ClearBones()

• Removes all bones except the root bone from the HandSkeleton

HandBoneId? GetParentBoneId(HandBoneId bone)

• Returns the identity of the bone that is the parent bone of the specified one and also contained in the HandSkeleton - what this means is that the returned bone may not be the specified bone's natural parent (closest possible ancestor) as is defined in the HandSkeletonConfiguration, but rather it is an ancestor bone of the specified bone. Null is returned if there is no such bone in the HandSkeleton

HandBone GetParentBone(HandBoneId bone)

• Same as above, it returns a reference to the resulting parent bone instead of just its identity. Null is returned if there was no parent bone found in the HandSkeleton

bool TryGetParentBone(HandBoneId boneId, out HandBone bone)

• Same as above, but it assigns the reference to the resulting HandBone and indicates whether there was a parent bone found, null is assigned to the output reference if no parent bone of the specified one is found in the HandSkeleton

Protected methods

Vector3 GetBoneScale(Transform boneTransform, float handScale)

• Returns the scale for the hand bone whose Transform is specified by the first parameter that should be applied to it in order to have it be matched with the specified total scale of the hand. The result can then be set on the specified Transform via its localScale property

Vector3 GetBoneScaledPosition(Vector3 boneUnscaledPosition, Vector3 rootPosition, float handScale)

• Another helper method - calculates and returns absolute position for a bone with the specified hand scale applied based on the input unscaled bone position and the absolute position of the root bone which is assumed to be the origin of the scaling

Vector3 GetBoneScaledPosition(Transform boneTransform, float handScale)

• This method is similar to the previous one, the difference is that this one tries to calculate scaled position of the bone whose Transform is specified by the first method parameter based on the bone Transform's current absolute position and HandSkeleton's current Transform (which is also the root bone's Transform) and then applies the specified total hand scale to the result

void SetBoneScalesAndScaledPositions(float handScale)

• Sets positions and scales of bones in the hand skeleton based on their current absolute scales and positions. These are applied to bone Transforms directly. This method can help in keeping the whole hand scaled appropriately in situations when no pose provider is assigned to the HandSkeleton

void SetBoneBindPoses()

• Updates bind poses of all HandBones in the HandSkeleton (accessed via BindPose property on HandBone script), but just for the ones that have bind pose capturing enabled (CaptureBindPose property on HandBone). This method works only in edit mode inside Unity editor, it does nothing if otherwise since bind pose capturing works only in edit-time as well

void UpdatePose()

• Updates hand skeleton pose based on pose provider poses and scale and HandSkeleton's scale and other options. Poses are updated by assigning them to HandBone Transform components directly. This method will essentially update HandBone poses the way it is intended by the HandSkeleton

• It sets world space positions and rotations on HandSkeleton's bones' Transforms, therefore it does not matter whether the HandBones are children of the HandSkeleton or any other object, but it would make sense not to have their Transforms under control of some other source, like physics by having a Rigidbody attached for example because in that case the resulting HandBone poses will almost certainly not be what one would expect

void FinalizePoseUpdate()

• Performs some post-pose update steps. This method sends out the OnPoseUpdated event

• This method is not part of the UpdatePose method because derived HandSkeletons may want to perform post-pose update step at some later point in the frame from the pose update

virtual HandBone CreateBone(GameObject boneObject)

• This method adds a new HandBone component to the specified GameObject and it gets called by the HandSkeleton in some cases when adding new bones to it. Derived HandSkeletons can override this in order to add their own specific types of HandBones to the GameObjects being specified by this method

virtual void BoneAdded(HandBone bone, int index)

• Callback method called after the specified bone has been added to the hand skeleton, before the hand bone has been notified about it via OnAddedToHandSkeleton method defined in HandBone script and before the OnBoneAdded event is sent. Index parameter specifies an index into the HandSkeleton's bone collection where the new bone is inserted

virtual void BoneRemoved(HandBone bone, int removedAtIndex)

• Callback method called after the specified bone has been removed from the HandSkeleton, before the hand bone has been notified about it via OnRemovedFromHandSkeleton method defined in HandBone script and before the OnBoneRemoved event is sent. Index parameter specifies an index into the HandSkeleton's bone collection where the bone was located before its removal

virtual void Reset()

• Unity built-in callback. Reset method is called after the HandSkeleton is reset using the Reset functionality in the editor or when the HandSkeleton is attached to a GameObject. This functionality resets all the values in the HandSkeleton to the initial ones. Because the HandSkeleton makes all the HandBone's state to be initialized when it is added to the HandSkeleton, HandBones which were in the HandSkeleton before reset was used will remain in an invalid state - from their perspective they are still attached to the HandSkeleton. This method is used to handle such HandBones and to properly remove them from the HandSkeleton, calling and sending all the appropriate callbacks and events that should be sent when a HandBone gets removed from the HandSkeleton

• Always call base.Reset from Reset overrides

virtual void OnValidate()

• Unity built-in callback. Initializes and ensures the state of the HandSkeleton is valid and corresponds with the state of its HandBones

• Always call base.OnValidate from OnValidate overrides

virtual void Awake

• Unity built-in callback. Initializes and ensures the state of the HandSkeleton is valid and corresponds with the state of its HandBones

• Always call base.Awake from Awake overrides

Building a HandSkeleton

In most cases manual creation of a hand skeleton will not be necessary as there are prefabs included in OctoXR package that have pre-built skeletons set up and ready, such as LeftHand/RightHand.

However, there may be a need to manually add a HandSkeleton derived script to a GameObject, such as when you have your own custom hand model that you wish to use in a scene. In that case the process is as follows:

• Add a specific HandSkeleton script to a GameObject that would correspond to the hand wrist. Which exact HandSkeleton derived script you should add depends on what you need it to do. Chances are you will need it just for the visual representation of a hand - BasicHandSkeleton should be sufficient in that case. If however you want physics-enabled one then PhysicsHandSkeleton should be considered, although you will probably want to set up bone colliders for this one if you haven't already

• To add bones for the HandSkeleton you can just assign them in the HandSkeleton's inspector by drag-and-dropping GameObjects you wish to act as HandSkeleton's bones to the desired hand bone identity keyed object fields under the Bones foldout menu

• There is Auto Add Bones button just under the Bones list in the inspector. You can try and use that to automatically detect GameObjects that best fit the role of being bones for the HandSkeleton. Note that this detection method is based solely on GameObjects' names, if you are working with a custom hand model it is questionable whether this functionality will manage to add any bones, but you can always try - there is no need to worry about messing up something, this operation is not destructive in any way and it will try to add only bones which are missing from the HandSkeleton, the ones that are already added are left alone

• You can manually add HandBone or HandBone derived script to a GameObject before adding it as a bone to a HandSkeleton if you wish, although this is not necessary. However, this way you can mix different types of HandBones for a HandSkeleton since the HandSkeleton, depending on the concrete type, will almost certainly attach the same specific HandBone script to the GameObjects which are added as hand bones to it. Depending on the type of HandSkeleton and the exact type of HandBones the HandSkeleton supports this may not always be possible, some HandSkeletons will require specific types of HandBones, but you can freely try any combination and see the results

• If any errors are generated while trying any of the previous actions, HandSkeleton(s) and HandBone(s) involved are always properly handled and they are not left in some undefined or invalid state so you don't need to worry about it

• All the things described above, with the exception of Auto Add Bones functionality, can be done from a script using a combination of HandSkeleton's methods described in previous section on this page and Unity engine's public API if you prefer to do things via code, the end effect is the same no matter the way you choose to create a HandSkeleton

• When you are done adding the bones to the HandSkeleton, you will probably want to assign pose provider for it in order to make it move with the target poses that are provided and that can come from different sources

Basic Hand Skeleton

• HandSkeleton derived script that defines, as its name would suggest, a basic hand skeleton

• This HandSkeleton is suited for most common needs

• It moves itself and its HandBones to target poses by directly setting their Transforms' world space positions, rotations and scales

• It exposes one property - UpdateRun which is of a certain enumeration type that defines which Unity's update callback should be used to set the HandBone Transforms; it can be Update, FixedUpdate, LateUpdate or any combination of these

Visualized Hand Skeleton

• Script that derives from HandSkeleton and is intended to be inherited by hand skeletons that serve the purpose of visualizing a hand in some special way

• Usual use for this type of HandSkeleton is to visualize another HandSkeleton, and the most common way to achieve this is to use a HandSkeletonSourcedPoseProvider on the HandSkeleton that needs to be visualized and having that pose provider assigned to the VisualizedHandSkeleton as its pose provider (via PoseProvider property), though there may be other ways to achieve this defined by the specific VisualizedHandSkeleton

• It uses the LateUpdate built-in callback to set the target pose which is set in the same way as it is set by the BasicHandSkeleton, LateUpdate is overridable if more custom behaviour is needed

• The way VisualizedHandSkeleton is defined and set up, it can serve as a visualization of another HandSkeleton or it can be a visualization in and of itself, you can attach any InputDataProvider to it just as with any other HandSkeleton

Armature Visualized Hand Skeleton

• A type of VisualizedHandSkeleton, it visualizes the hand by using small spheres for hand bone joints and cylindrical segments that end up resembling sticks to represent connection of a bone with its child bones (meaning 6 segments for Wrist bone is used and 1 for every other, except the bones that have no child bones, which are only visualized by their joints). All these visualized components combined result in something that resembles a construction armature that is shaped like a hand

• For every bone visualization component, a separate GameObject is used with MeshRenderer component attached to it. This results in many separate meshes being used for visualizing a hand and these renderer components can are used for manipulating all the usual rendering options provided by them

• ArmatureVisualizedHandSkeleton uses its own special type of HandBone - ArmatureVisualizedHandBone, which manages its corresponding set of objects used as visualization components for the bone (one joint and possibly one or more segments). It exposes certain properties to provide simple and fast way of accessing each individual visualization component as well as some other rendering related properties

• Removing a bone from the ArmatureVisualizedHandSkeleton results in the bone's visualization components being disabled so that they no longer get rendered

• If you want to create a script for your own custom bone type to be used with a ArmatureVisualizedHandSkeleton, then have that script be derived from ArmatureVisualizedHandBone instead of HandBone

• Important to mention is that the ArmatureVisualizedHandSkeleton provides one additional way for making it act as a visualization of another HandSkeleton, in addition to using a HandSkeletonSourcePoseProvider. It exposes the certain HandSkeleton-typed property - VisualizationOf. Any HandSkeleton can be assigned to this property in order to make the ArmatureVisualizedHandSkeleton act as a visualization for the assigned one. This has one difference from using HandSkeletonSourcedPoseProvider - HandBones which are missing from the source HandSkeleton are removed from the ArmatureVisualizedHandSkeleton as well and if a HandBone is added to the source HandSkeleton, a corresponding bone gets added to the ArmatureVisualizedHandSkeleton. This behaviour does not occur when using a HandSkeletonSourcedPoseProvider method for visualizing other HandSkeletons