Background

VirtualGrasp not only provides solutions to grasp interaction, but also for push interactions.

Button push as an action in real world is very simple process due to the very physical laws existing in the nature. When the same problem is ported into VR, a realistic button push behavior requires some tedius steps that often needs complex setup of physical properties on both button and the hand.

VirtualGrasp provides a solution to make push interaction setup easier and intuitive through object articulation, with and without relying on physical simulations.

Most important concepts relevant to push interaction are explained in object articulation page.

Below we will explain:

  • first how to setup push interaction without adding any physical properties on an object – push without physics,
  • and then how to easily extend this setup to use physical simulation provided by a game engine – push with physics.

Push Without Physics

From Object Selection to Push Without Physics

When push is done with physics, the selection of object is straightforward – by contact and apply force. When push is done without physics, there need to be a way to decide which object is to be pushed. VirtualGrasp does this through checking the geometric relation between a push agent and the object to be pushed. To allow VR developers to be able to specify for example “This button can not be pushed from below”, push pivot is added to the object to specify the preferred approach direction by the push agent for pushing this object.

Given that

  • the push agent has been assigned (through set interaction affordance as INDEX_PUSHABLE), and
  • push pivot is added to the object to be pushed,

the decision on which of the pushable objects is selected for pushing is done by considering:

  • how close is the push agent to the pushable object and
  • how aligned is the push agent’s approach direction with the preferred approach direction.
A Unity button.
A Unity button

For example image above shows the push pivot transform added to the red button. The blue arrow (zaxis of this transform) specifies the approach direction. So when index finger is approaching from bottom to the button, opposite to the blue arrow, the object will not be selected for push.

Once an object is selected, you can turn on the visual hint to show which object is selected for push without physics. See VG_HintVisualizer to learn how to use it.

How to Setup Push Without Physics

Through VirtualGrasp’s object articulation feature, a VR developer can setup a pushable object with various behaviors simulating real world’s button object in any game engine. And this setup does not require the developer to add any physical components (such as RigidBody or Colliders) to either objects or hands in the game engine.

Using Unity game engine as an example, to create a pushable button, you add the component VG_Articulation to the object, and the parameters shown in below image create a button that can be pushed by index finger, and will bounce alternatively to the two discrete states when push is released (see affordances):

VG Articulation
VG_Articulation Component.
  • Since button is moving along a single axis linearly, we select the joint type to be PRISMATIC.
  • For PRISMATIC joint, we need to provide pivot which specifies the joint axis along which object moves.
  • For PRISMATIC joint, we also need to provide joint limit, i.e. a linear movement range between 0 and 0.014 meter.
  • Since the pushable button’s movement direction (by pivot) is same as the direction we want hand to approach for pushing, so we don’t need to specify a different push pivot.
  • For push without physics, you need to select interaction affordance as INDEX_PUSHABLE.
  • To let object bounce back alternatively to two stages when push is release, you need to set the state affordance to be TWO_STAGE.
  • To let object bounce back NOT to the lower boundary of the joint limit (min = 0), but to slightly lower position, we set two discrete states with first value 0.004 (>min).
When TWO_STAGE affordance is chosen, object’ joint state will bounce back to the smallest value of the discrete states after the 2nd push. Since by default discrete states will take the two range values from the joint limits [min, max], so if discrete states is not provided, the joint state will bounce back to the min value. If you want it to bounce to a different value from min, you should define the discrete states with first (smallest) value larger than joint limit’s min value. As shown in above image, the smallest discrete value 0.004 is bigger than the min value 0.

Push With Physics

VirtualGrasp also supports push through physics, by seamlessly integrating VG’s object articulation feature with the physical simulations already existing in the client engines (Unity or Unreal).

To create a pushable button with physics, besides all the setup in VG_Articulation component as shown in push without physics section, you only need to:

  • set up the physical properties for the object in the game engine. For example in Unity, you should add RigidBody and Collider to this object; and
  • set up the physical properties for the hand. In Unity, VirtualGrasp automatically setup the hand physical properties (RigidBody and Colliders) if you enable Physical avatars in Sensors (see image below).
VG Avatars
MyVirtualGrasp script - Avatars.

Push Without Physics vs. With Physics

To help you set up the push object with physics, most of the object articulation parameters that applies to without physics situation also will take effect, with very few exceptions:

  • For push with physics, an object can be pushed by any part of the hands or any object that have physical properties. Therefore setting interaction affordance to be INDEX_PUSHABLE is not relevant because it will not limit the push by only the index finger.
  • For push with physics, push pivot won’t take effect to specify a preferred approach direction, which can be good or bad depending on the game logic and the desired user experiences.

The table below give some hints and considerations to help you choose which push option to use:

Push without Physics Push with Physics
Push without Physics allows easy, though limited setup of push interaction. Push with Physics allows more realistic physics-based push interaction.
Easy to setup since no need to specify dynamic properties of a rigid body like mass, drag etc. + - Need careful selection of dynamic properties since they influence how object react to push
Limited in terms of “who” can push the object (now only INDEX_PUSHABLE). - + Anything with RigidBody and Collider can push since it is physics-based.
Can specify preferred approach direction through push pivot + - Can not specify preferred approach direction.
From some direction the object cannot be pushed - + Object can be pushed from any direction since it is physics-based.