Background

For any avatar model (skeletal mesh), VirtualGrasp identifies hand bones among the entire skeleton hierarchy, and creates the internal hand representation to control the hand movement and interaction with objects.

The VG SDK includes an avatar model called GleechiRig (in VirtualGrasp/Resources/GleechiHands/) which is also used in the onboarding scene. In this avatar, both a left and a right hand are included in a single skeleton hierarchy.

However, avatars can have non-hand bones such as elbows, shoulders, or even full body bones. VirtualGrasp automatically identifies the hand bones and only controls the hand movement, but does not influence any optional other parts of the skeleton.

Custom Hand Models

Hand Model Standard

The image below shows GleechiRig’s left hand model. The labeled wrist and finger bones represent the hand bones whose movement VirtualGrasp controls.

If you want to use a custom avatar (enabled in the Pro version of the VirtualGrasp SDK), the recommended minimum conditions to be met related to the skeletal hand structure are:

  • A hand’s wrist bone should include 5 fingers as children (no extra bones),
  • with 3 bones in each finger whose movement VG controls, and
  • for thumb, the 3 bones correspond to the first metacarpal and the 2 phalanges, and
  • for non-thumb fingers, the 3 bones correspond to the 3 phalanges in human hand anatomy.

Additionally,

  • The last transform (4th bone) at finger tip (thumb3_L for example) is recommended to be provided to give an accurate representation of the last bone’s dimension. If a fingertip is missing, VirtualGrasp will estimate its position, but this may result in finger tip penetration when pushing or grasping objects. Note it is very easy in Unity to add missing fingertip bones manually.
  • VirtualGrasp also supports custom hands with the 4 metacarpals connecting the non-thumb fingers, however it does not control the movement of them.
VG hand model.
VG Hand model example with 4 bones (including finger tip) each finger.

Separate Hand Models

If you have a skeletal mesh for one hand side and are creating the opposite side by duplicating and mirroring this hand, you can setup two separate avatars with same sensors to control the two hands. In such cases VirtualGrasp considers the left and right hand skeleton meshes as two separate Avatars. You can set it up using same Hand Profile and sensor setup.

Please watch the video below on how to setup your Custom Hand in Unity.

Hand Profiles

VirtualGrasp provides VG_HandProfiles (in Unity as ScriptableObjects) to configure a number of hand model-related settings and thereby allows you to quickly switch between different custom hands.

VG_HandProfile in Unity.
VG_HandProfile as scriptable object in Unity.

Please watch the video below for a tutorial that shows an example of how to configure a VG_HandProfile for your Custom Hand in Unity.

If this video shows "This video is unavailable," please try to refresh the page.

Avatar Types

There are three avatar types in VirtualGrasp:

  • By default an avatar is a sensor avatar, meaning that avatar’s hands are directly controlled by the sensors for movement and object interaction. This is also the avatar used for recording sensor data (feature available in Pro version).
  • If the Replay option is ticked, then the avatar will be registered as a replay avatar. The hands of this avatar will be controlled by pre-recorded sensor data. Note this sensor record and replay feature is only available in VirtualGrasp Pro version.
  • Both sensor avatar and replay avatar can be created as a physical avatar if Physical option is ticked.
Unity hand model.
Hand model references need to be provided in MyVirtualgrasp → Avatars → Skeletal Mesh.
Note that "Replay" only appears in Pro-versions of VG.
The physical avatar currently is only semi-physical in that only colliders are added to hand bones, no rigid bodies or articulation bodies are used.