2.1 - Audio and Haptics

1. Add haptic feedback on hover and select enter

You can add haptic on hover enter and select enter to provide physical feedback to the player’s hands.

Note: If you are using the OpenXR plug-in for development on devices like the HTC Vive or Valve index, this feature may not work.

1. Edit both controller objects at the same time:

• In the Hierarchy, expand XR Rig > Camera Offset.

• Ctrl/Cmd+select both the RightHand Controller and LeftHand Controller so they are both selected.

2. Locate the haptic events:

• At the bottom of the Controller objects’ XR Ray Interactor components, expand the Haptic Events fold-out.

3. Create subtle haptic feedback when the user hovers over an object:

• Click the On Hover Entered check box to enable it.

• Set the Intensity to a low value (e.g. 0.1-0.5).

• Set the Duration to a very low value (e.g. 0.1 seconds).

4. Create more noticeable feedback when the user grabs an object:

• Select the On Select Entered check box.

• Set the Intensity and Duration values to slightly higher values (e.g. 0.25 seconds).

The controller should now vibrate subtly when you hover over an object and slightly more so when you select an object.

2. Add audio feedback on either hover or select enter

Now that you have haptic feedback, you can enhance the User Experience (UX) with audio feedback.

Note: If you are using the OpenXR plug-in for development on devices like the HTC Vive or Valve index, this feature may not work.

1. Select the sound you want to use:

• From the Course Library > Audio > FX, determine which subtle UI sound effect you want for the hover or select event.

• To preview the sound, you may need to drag up the Preview panel from the bottom of the Inspector window.

2. Apply this sound effect to both controllers:

• From the Hierarchy, select both the RightHand Controller and LeftHand Controller objects.

• In the XR Ray Interactor component, expand the Audio Events fold-out.

• Click to enable either the On Select Entered or the On Hover Entered setting.

• and then assign your chosen sound effect.

You should now hear a sound effect either when you hover over an object, or when you select an object.

3. Add 3D audio from the fireplace

In addition to audio feedback on events, you can add 3D spatial audio into the scene emanating from the fireplace to increase immersion.

1. Give the fire a sound effect:

• Add an Audio Source component to the particle object.

• For the AudioClip property, assign the SFX_Loop_Fire sound effect from the Course Library > Audio > FX folder.

2. Make the sound play automatically and loop:

• In the Audio Source component, make sure both the Play on Awake and Loop settings are enabled.

3. Configure this component as a 3D sound:

• Adjust the Spatial Blend property by dragging the slider all the way to the right 3D setting (or by setting the slider to 1).

4. Edit the overall volume and volume roll-off of the fireplace:

• Edit the Volume property.

• Expand the 3D Sound Settings fold-out, then adjust the Min Distance and Max Distance values.

The fireplace should now make a crackling sound, which should change in volume and left/right balance depending on where you are located and oriented in the scene.

Related resources:

• Audio Source Settings

4. Add a Reverb Zone

To maintain a sense of presence in VR, it’s especially important to make sure audio behaves the way it would in the real world. This includes considering how sounds might reverberate around different kinds of rooms.

1. Add a new Audio Reverb Zone as a child of the Room object:

• In the Hierarchy, right-click the Room_[style] object.

• Click Audio > Audio Reverb Zone.

2. Make sure the reverb zone encompasses your whole room:

• In the Audio Reverb Zone component, make sure the Min Distance property is set to at least the width of your room (10).

3. Make your reverb zone match your room:

• Use the Reverb Presets drop-down to experiment with different styles of reverb.

• Select a setting that matches your room (Room or Living Room).

You should now have a reverb zone set up in your room to make the 3D sound even more realistic.

Related resources:

• Audio Reverb Zones

5. Experiment with spatializer plugins [optional]

In addition to Unity’s built-in 3D sound, which changes the volume in the left and right ears depending on player distance and orientation, there are additional spatializer plugins that you can install to make the sound even more realistic.

1. Download and import a plugin:

• From the links below, download the relevant spatializer plugin for your platform.

• Import the package into your project.

2. Enable the plugin:

• Click Edit > Project Settings.

• Select the Audio section from the left panel.

• Use the drop-down next to Spatializer Plugin to select your plugin.

3. Implement the plug-in on your fire particle:

• Follow the instructions relevant to your device to implement spatialized audio.

You should now have even more realistic sounding audio in your scene, taking into account the user’s Head Related Transfer Function (HRTF).

Related resources:

• Unity Audio Spatializer

• Oculus Spatializer Download

• Oculus 3D Audio Spatialization Guide

• Oculus Spatializer Features

• Apply Spatialization in Unity

• Microsoft HRTF Spatializer Download
  (supports UWP and PC running Windows 10)

6. Recap

New Functionality:

• Haptic feedback on controllers

• Audio feedback on controllers

• Realistic 3D Audio from fireplace

New Concepts & Skills:

• Importance of haptics and audio for immersion

• 3d vs 2d audio

• Reverb zones

• Spatialized audio with head-related transfer function (HRTF)

Technical Skills:

• Audio & Haptic events

7. Extension Activities

If you want to further develop your skills, explore new concepts, and improve your project, check out some of the optional extension activities below.

Each one is tagged as [Easy], [Medium], [Difficult], or [Expert] and will also include a [Requires Programming] tag if some coding is required.

1. Add a speaker [Easy]

Add a speaker to the room that plays music - or even a record player that actually rotates:

• Find a speaker from the Course Library > Prefabs > Objects > Speakers folder.

• Find music options from the Course Library > Audio > Music folder.

• Bonus - If you use the Record Player, you’ll need a simple script to rotate the platter the record spins on.

2. Add realistic bouncing sounds [Medium] [Requires programming]

Add a bouncing sound for your ball so that you hear it bounce every time it hits the floor:

• You may want to download and import a new sound effect

• Add a 3d audio source to your ball just like you did for your fireplace.

• To make it play a sound when it hits the ground, you will probably want to use the OnCollisionEnter function and PlayOneShot method.

• Bonus - if you want to make the volume change based on the speed the ball is moving, try using rigidbody.velocity.magnitude.