7afc438a-bba9-5dda-a13e-893d0e867685:IJ50VM7ryBj2kIwUvpGgtc

Best Practices for Immersive Audio in VR

Installing the Microsoft Audio Spatializer

Installing the Oculus Audio Spatializer

Assigning an Audio Spatializer to Your Project

Adding Spatial Abilities to Audio Sources

Using the Microsoft Audio Spatializers

Using the Oculus Audio Spatializer

Conclusion

 In physical reality, the shape, layout, and physical materials of your environment affect how sound reaches your ears. In this tutorial, we'll explore how to use audio spatializer plug-ins to match your VR audio to the visual experience. 

Microsoft.SpatialAudio.Spatializer.Unity 1.0.36 (NuGet package)

In physical reality, the shape, layout, and physical materials of your environment affect how sound reaches your ears. In this tutorial, we'll explore how to use audio spatializer plug-ins to match your VR audio to the visual experience. 

Both Oculus and Microsoft provide spatializer plug-ins for VR in Unity. Using a spatializer plug-in will allow your physical environment (for Oculus) and the position and angle of Audio Sources, relative to the user, to affect how they sound, as in real life. We'll look at how to use these in combination with 3D Sound Settings on Audio Sources to provide high-fidelity VR audio.

There are two Microsoft Audio Spatializers. MS HRTF Spatializer is for projects that need to be compatible with earlier HoloLens and Windows XR headsets and projects. This is included in the Windows XR Plugin, installed via the Package Manager. 

1. From the Window drop-down, select Package Manager.

2. In the left column, click Windows XR Plugin. Click Install in the lower-right corner (Figure 01). 

Microsoft Spatial Audio is downloaded via NuGet, an add-on Package Manager for Unity, and should be used for new projects.

1. Save the latest version of the NuGet package to your computer.

2. From the Assets menu, select Import Package > Custom Package (Figure 02). 

3. Navigate to where you saved NuGet.unitypackage (the actual file will be named to reflect its version number). Click to select, and click Open.

4. You can also double-click on the downloaded NuGet.unitypackage with Unity open to install. 

5. Leave all files selected and click Import (Figure 03). 

6. Once installed, click the NuGet drop-down and select Manage NuGet Packages (Figure 04). 

7. In the Search bar, type Spatializer. Click Install in the upper-right corner of the entry for Microsoft.SpatialAudio.Spatializer.Unity (Figure 05). 

Figure 01: Installing the Windows XR Plugin via the Package Manager

Figure 02: Importing a custom package

Figure 03: Selecting components of NuGet for import

Figure 04: Menu entry for managing NuGet Packages

Figure 05: Installing Microsoft Spatial Audio using NuGet

The Oculus Audio Spatializer for Unity is available as a standalone .unitypackage, on the Oculus Developer website, or as part of the Oculus Integration Unity Asset.

1. Select Asset Store from the Window drop-down in the Unity Editor.

2. Search for Oculus Integration (Figure 06). 

3. Click Oculus Integration. Click the button to download, then again to import to your project (Figure 07).
 

4. Click Import in the lower-right corner to import the Oculus Integration Package (Figure 08). 

5. If prompted to update to a newer version of the OVRPlugin, click Yes (Figure 09). 

6. After updating OVRPlugin, restart the Unity Editor (Figure 10a). 

7. When prompted, click Upgrade to use the new spatializer plug-in (Figure 10b). 

8. Finally, click to restart the Unity Editor to use the new spatializer plug-in (Figure 10c). 

9. The Unity Editor should now restart, using the newest version of the Oculus Audio Spatializer. 

Figure 06: Searching for Oculus Integration in the Asset Store

Figure 07: Downloading and importing the Oculus Integration Asset via the Asset Store

Figure 08: Importing the downloaded Oculus Integration Asset Package

Figure 09: Updating the Oculus Utilities Plugin

Figure 10a: Restarting the Unity Editor after updating OVRPlugin

Figure 10b: Upgrading to the new audio spatializer plug-in.

Figure 10c: Restarting the Unity Editor to use the new spatializer plug-in.

1. From the Edit drop-down, select Project Settings.

3. Click Spatializer Plugin and make your selection (Figure 11). 

Figure 11: Assigning a Spatializer Plugin

1. If using the Oculus Audio Spatializer, set Output to use the Spatializer mixer.

2. Click Spatialize to enable spatialization and, optionally, Spatialize Post Effects.

3. Finally, set the Spatial Blend from 2D to 3D (Figure 12). 

Figure 12: Configuring an Audio Source for VR sound.

The Microsoft Audio Spatializer automatically handles spatialization for Audio Sources that have spatialization enabled. 

When targeting Oculus hardware, you can assign physical material properties to any GameObject with a Mesh Filter or Terrain component in your Scene. The GameObject needs to be static.

For each piece of level geometry that should affect audio:

1. Click to select the GameObject in the Hierarchy view. 

2. Add the ONSP Propagation Geometry component.

3. Check the box labeled File Enabled. If applicable, also check Include Child Meshes (Figure 13). 

4. When you’ve attached the ONSP Propagation Geometry component to all participating Meshes and Terrains, you’re ready to build the audio geometry for your Scene(s) by using the Oculus drop-down to select either option from the Spatializer submenu (Figures 14-15). 

5. This attaches an ONSP Propagation Material component to each GameObject with the ONSP Propagation Geometry component, allowing you to assign a physical material preset, or define a custom material to alter sound in the virtual environment (Figure 16). 

6. Each time the menu (from Step 4) is used to bake audio geometry, another ONSP Propagation Material component is added to all applicable GameObjects. Instead of baking all the Scene geometry again, you can bake newly-added geometry by clicking Bake Mesh to File in the Inspector (Figure 17). 

Figure 13: Enabling baking for ONSP Propagation Geometry

Figure 14: Oculus Spatializer menu entry when using it standalone

Figure 15: Oculus Spatializer menu entry when using the Oculus Integration Asset

Figure 16: Selecting an ONSP Propagation Material Preset.

Figure 17: Baking ONSP Propagation Geometry

Audio spatialization can take your VR project’s aural immersion to the next level.

Best Practices for Immersive Audio in VR

1. Best Practices for Immersive Audio in VR

2. Installing the Microsoft Audio Spatializer

3. Installing the Oculus Audio Spatializer

4. Assigning an Audio Spatializer to Your Project

5. Adding Spatial Abilities to Audio Sources

6. Using the Microsoft Audio Spatializers

7. Using the Oculus Audio Spatializer

8. Conclusion

Complete this tutorial