All content in this course uses:

MR Interaction

This tutorial will teach you the basics of Mixed Reality interaction, specifically for the HoloLens headset.

Tutorial21_MRInteraction.zip

 This tutorial will teach you the basics of Mixed Reality interaction, specifically for the HoloLens headset.

To complete the tutorial, you’ll need to:

Set up your Scene for Mixed Reality development

Simulate HoloLens interaction in the Unity Editor

Configure Move, Scale, and Rotate commands

Overview

Open the Mixed Reality Scene Settings window (

Mixed Reality Toolkit > Configure > Apply Mixed Reality Scene Settings

Check the boxes shown above and leave the others unchecked.

 automatically set up your Scene to include the following: 

Set Up your Scene for Mixed Reality Development

Gaze at the Cube using right click + mouse.

Use the arrow keys on your keyboard to move around.

Use the spacebar + mouse to simulate an air tap.

, right-click and move your mouse to gaze around the Scene.

Use the arrow keys to move around the Scene.

While gazing at the Cube, hold the spacebar and use your mouse to move the virtual hand. Click to simulate an air tap.

Changing an object’s color is a simple way to ensure the core interaction mechanic is working and can save you lots of time in your development process. The color-changing functionality is already done for you on this Cube, but you will learn how to make these changes in the next section.

Simulate HoloLens Interaction in the Unity Editor

Add a Cylinder to the Scene and position it in front of the Camera and off to one side (opposite the Cube).

Configure the OnPointerEnter event to change the Material using the Event Trigger component.

Use the Selection window to select the Cylinder

Inside the Cube’s inspector, look at the Event Trigger component and note the specific events highlighted here:

 is triggered when a certain action happens in Unity. You can think of an event like a broadcast system. When the event is triggered, it's broadcasted to all GameObjects in your Scene. Any GameObject that's "interested" in the event can add actions (change Material, start animation, turn off, play audio, etc.)  to the event that get executed upon the event’s broadcast. 

 can be used to specify functions you wish to be called for each Event System event. You can assign multiple functions to a single event and whenever the Event Trigger receives that event, it will call those functions.

Implement Pointer Interactions

Create an empty GameObject and name it VoiceManager.

Add a Speech Input Source component and add the “red” keyword to the keywords list.

Add the Speech Input Source component by hitting 

 manages speech recognition on the HoloLens. This component is where you add keywords that will trigger an event when the system recognizes the word through the mic.

 are commonly used in Unity to delegate tasks and "manage" other GameObjects in your Scene. Common examples of managers include: GameManager, StateManager, VoiceManager, AudioManager, and AnimationManager.

 By naming a GameObject "GameManager," you're not giving it special power. Naming is purely semantic.)

Add a Keyword to the Voice Manager

Create a Capsule GameObject and name it VoiceControlledObject.

Add a Speech Input Handler component and add the “red” keyword to the keywords list.

Add a Material change to the Response event and make it turn red.

, and position it so that it's in front of the Camera (viewable in the Game view).

, searching for the Speech Input Handler, and double-clicking to select it.

 keyword that was declared in the Speech Input Source by hitting the 

 button and using the drop-down to select 

Add the Material change to the Response event by using the drop-down to select Material (

) and selecting the red highlight Material. 

 is used to add responses to the voice events declared in the Speech Input Source script.

Create a Voice Controlled Object

Create another GameObject and name it RotateScaleObject. Position it in front of the Camera to illustrate rotate and scale.

Add the Bounding Box Rig to the GameObject and configure the Bounding Box Prefab.

, and position it in front of the Camera and to the opposite side of the 

Holotoolkit > UX > Prefabs > BoundingBoxes > BoundingBoxBasic.prefab

 into the Bounding Box Prefab slot on the Bounding Box Rig component. It should look like this:

Add the Ability to Rotate and Scale

You now have a working knowledge of the core interaction techniques for the HoloLens and have successfully set up a Scene to work with those interactions. 

By completing this tutorial, you've learned to:

Unity Version 2018.4