Background

For any avatar model (skeletal mesh), VirtualGrasp identifies the left and right hand bones among the entire skeleton hierarchy, and creates the internal hand representation to control the hand movement and interaction with objects.

The VG SDK includes an avatar model called GleechiRig (in VirtualGrasp/Resources/GleechiHands/) which is also used in the onboarding scene. In this avatar, both hands are included in a single skeleton hierarchy, and there are no bones other than hand bones included.

However, avatars can have non-hand bones such as elbows, shoulders, or even full body bones. VirtualGrasp automatically identifies the hand bones and only controls the hand movement, but does not influence any other parts of the skeleton.

Custom Hand Models

Hand Model Standard

Image below shows GleechiRig’s left hand model. The labeled wrist and finger bones represent the hand bones whose movement VirtualGrasp controls.

If you want to use a custom avatar (enabled in the Pro version of the VirtualGrasp SDK), the minimum conditions to be met related to the skeletal hand structure are:

  • A hand’s wrist bone should include 5 fingers as children (no extra bones),
  • with 3 bones in each finger whose movement VG controls, and
  • for thumb, the 3 bones correspond to the first metacarpal and the 2 phanlanges, and
  • for non-thumb fingers, the 3 bones correspond to the 3 phanlanges in human hand anatomy.

Additionally,

  • The last transform (4th bone) at finger tip (thumb3_L for example) is recommended to be provided to give accurate representation of last bone’s dimention. If a fingertip is missing, VirtualGrasp will estimate its position, but may result in finger tip penetration when pushing or grasping objects. Note it is very easy in Unity to add missing fingertip bones manually.
  • VirtualGrasp also supports custom hands with the 4 metacarpals connecting the non-thumb fingers, however it does not control the movement of them.
VG hand model.
VG Hand model example with 4 bones (including finger tip) each finger.

Separate Hand Models

If you have a skeletal mesh for one hand side and are creating the opposite side by duplicating and mirroring this hand, you can setup two separate avatars with same sensors to control the two hands. In such cases VirtualGrasp considers the left and right hand skeleton meshes as two separate Avatars. You can set it up using same Hand Profile and sensor setup.

The image below shows an example of how to set it up in Unity, but it applies to any VG integrated game engines.

Unity Avatar Sensor setup for two separate hand models.
Unity Avatar Sensor setup for two separate hand models.

Hand Profile

VirtualGrasp provides VG_HandProfiles to configure a number of hand model-related settings and thereby allows you to quickly switch between different custom hands.

VG_HandProfile in Unity.
VG_HandProfile as scriptable object in Unity.

(Editing of this section is in progress.)

Avatar Types

There are three avatar types in VirtualGrasp:

  • By default an avatar is a sensor avatar, meaning that avatar’s hands are directly controlled by the sensors for movement and object interaction. This is also the avatar used for recording sensor data (feature available in Pro version).
  • If the Replay option is ticked, then the avatar will be registered as a replay avatar. The hands of this avatar will be controlled by pre-recorded sensor data. Note this sensor record and replay feature is only available in VirtualGrasp Pro version.
  • Both sensor avatar and replay avatar can be created as a physical avatar if Physical option is ticked.
Unity hand model.
Hand model references need to be provided in MyVirtualgrasp → Avatars → Skeletal Mesh.
Note that "Replay" only appears in Pro-versions of VG.