Welcome to AR Development

Welcome to the world of AR development!

In this tutorial, you'll learn about the field of AR, what to expect in this learning experience, and what you'll need to get started.

Hear from established creators

Before you begin your journey in AR development with Unity, it will be helpful to hear from people who are further along in their journeys.

In the video below, a few established creators discuss what it’s like to work in the world of AR.

If you're going through this tutorial as the first step in the AR Development pathway, congratulations! You have a lot of exciting AR experiences ahead of you. Below is an overview of what you’ll be doing in this pathway.

Mission 1: Get started with AR

In this first mission, you'll build your first AR app and install it on your mobile device to make sure everything is working properly. You're on this mission right now.

Mission 2: AR face filters

In this mission, you'll learn to create your own AR face filter apps. First, you’ll make a simple filter that applies materials and 3D models to the user’s face.

Then, you’ll create a more interactive app that allows the user to select, customize, and visualize different pairs of glasses.

Mission 3: AR markers and planes

In this mission, you'll create AR apps that respond to the user’s physical environment. First you’ll make a marker-based app; the app will recognize a specific image and project an interactive 3D model on top of it.

Then you’ll make an app that detects flat surfaces and allows the user to spawn a portal on their desk, floor, or wall.

Mission 4: AR experience design

In this final mission, you’ll put your AR experience design skills to the test. You'll take an existing desktop game about aliens abducting sheep and adapt it for mobile AR.

This content is designed for anyone who wants to create mobile AR experiences with Unity. More specifically, it was designed with the following learners in mind:

• People who have created AR filters with other programs but want to expand their current AR skillset and develop more customized AR experiences with Unity.
• People who have already created other types of non-AR Unity experiences but are interested in developing for AR.
• People who enjoy using AR apps and want to create their own.

This course does assume a basic level of Unity knowledge, such as navigating the Editor and 3D space. If you’re not at all familiar with Unity, we recommend you check out the Unity Essentials pathway first. Familiarity with Visual Scripting is not required, but would be helpful.

If you’re reading this, you’re probably familiar with AR; you may have even developed your own AR apps already. But just in case, here’s a simple definition: augmented reality (AR) overlays digitally created content onto the user’s real-world environment.

To help visualize this, check out some of the AR applications made with Unity:

AR and other types of XR

AR is part of the broader category of extended reality (XR), an umbrella term that encompasses AR, VR (virtual reality), and MR (mixed reality):

• VR replaces the user’s entire visual field with a virtual one. This is distinct from AR, where you can still see your surroundings but with virtual content on top. Check out some of these examples of VR projects made with Unity.
• MR is the broadest term of the three, since it can include elements of AR and VR. It is most helpful to think about AR, MR, and VR along a reality-virtuality continuum, with the real physical environment on one end, and completely virtual environments on the other end.

In this framework, VR is closer to the virtuality end of the spectrum, AR is closer to the reality end of the spectrum, and MR would include most of the spectrum between the two.

Hear from some established creators

You’ve now learned about what AR is, but what makes it so special? In the video below, a few established creators discuss what separates AR from other forms of digital media.

Before you begin developing new experiences, you should familiarize yourself with the range of AR experiences that are already out there. Awareness of existing AR applications will give you helpful context for developing something unique yourself.

Below is a list of AR experiences you can try across different categories. Try to explore one or more from each category.

1. AR face filters

These apps recognize human faces and overlay digital content on top of specific facial features:

• Social media with filters (Snapchat, Instagram, TikTok)
• Fun face filters (FaceLab iOS | A​​ndroid, Banuba iOS | Android)
• Makeup and hair filters (YouCam Makeup iOS | Android)

2. Marker-based AR

These apps recognize a particular image, pattern, or QR code, then overlay content on top of that image:

• Marketing and games (Pizza Hut Pac Man)
• Provide additional information (business cards, medical demos)
• Art (ArtVive demo | iOS | Android)
• Utility (Inkhunter demo | iOS | Android)

3. Markerless AR

Unlike marker-based AR, markerless AR does not require any particular image in order to orient and hold content appropriately in the user’s 3D environment:

• Utility (Google Translate AR demo | iOS | Android)
• Design (Houzz demo | iOS | Android)
• Fun (Angry BIrds: AR iOS | Android)

4. Location-based AR

Unlike any of the above, these apps take into consideration your GPS location, often combined with scanning the environment, to overlay content in the scene:

• Tourism (World Around Me demo | iOS | Android)
• Education (Stellarium demo | iOS | Android)

Entertainment (Pokemon Go demo | iOS | Android)

Evaluate the AR experiences

With the examples above in mind, take a moment to consider the following questions:

• What are some recurring features in these apps?
• For what types of experiences is AR the most appropriate solution?
• What are the use cases (if any) where AR would not work at all? If not, why?
• What are the affordances and limitations of AR as a medium?

Hear from some established creators

In the video below, a few experienced AR creators discuss how they keep up with the constantly changing AR landscape.

There is a wide range of hardware that you can use to experience and develop AR. In this course, you'll focus on handheld AR, where a mobile device’s camera and screen are used to experience AR.

Even though this learning experience focuses on handheld AR, there is a very important distinction between handheld AR and head-mounted AR that you should understand.

Head-mounted AR

With head-mounted AR, the user actually wears a device on their head and experiences AR through lenses covering their eyes.

Photo by My name is Yanick on Unsplash.

Many companies offer head-mounted AR devices, each with its own distinct technology. Here are a few prominent head-mounted AR devices:

• Magic Leap
• Microsoft Hololens
• Google Glass

These devices are currently prohibitively expensive for most people and are therefore more targeted towards businesses rather than individual users.

You'll not be learning to develop for head-mounted devices in this learning experience.

Handheld AR

With handheld AR, users hold their own handheld device (usually a mobile phone) in their hands, point the camera at something, and view the AR content on the device’s screen.

Photo by UNIBOA on Unsplash.

Nearly all handheld devices run on either Google’s Android operating system or Apple’s iOS operating system. In the United States, about 50% of the population has Android devices and 50% has iOS devices. Worldwide, the split is very different, with closer to 85% using Android and only 15% using iOS. Developing apps for either of these platforms is easy with Unity, but the process for configuring for and deploying to the devices is different.

For these tutorials, you'll need one of the following devices with a cable allowing you to connect it to your computer:

• An iPhone X or later running iOS 11.0 or later (here’s the process to confirm your iOS version).
• An Android device running Android 7.0 Nougat or later (here’s the process to confirm your Android version).

Just as most phones run on either iOS or Android, most people’s computers run on either MacOS or Windows.

If your computer runs MacOS (any Mac computer), then you can deploy apps to both Android and iOS devices. However, if your computer runs Windows (most non-Mac computers), you can only deploy apps to Android devices. You cannot deploy apps to iOS devices with a Windows computer because the process requires a program called XCode, which is only compatible with MacOS.

Important: If you only have access to a computer running Windows and a mobile device running iOS, you'll not be able to test your AR projects. Any other combination will work.

You now have a better understanding of the world of AR and what to expect in the rest of this learning experience. In the next tutorial, you'll prepare for AR development by downloading all the required software and opening a new Unity project.