Instructor Introduction & ML-Agents Overview

Traditional Game AI vs. ML-Agents & Hummingbird Course Preview

Observations

Challenge 1: Observations

Challenge 2: Rewards

In this project, we will start with an overview of ML-Agents. Then we will talk about the hummingbird project and what kinds of observations and rewards we need for successful training.

Course Overview

Initial Setup, Asset Import & Default Contact Offset

Asset Overview & Lighting

Install ML-Agents & Script Creation

Explanation

Variables and Accessors

Feed()

ResetFlower()

Awake()

ResetFlower(), GetFlowerFromNectar(), Awake() & Start()

FindChildFlowers()

Variables & Accessors

Initialize() & OnEpisodeBegin()

MoveToSafeRandomPosition()

UpdateNearestFlower()

OnActionReceived()

CollectObservations()

Heuristic()

Freeze() & Unfreeze()

OnTriggerEnterOrStay()

OnCollisionEnter()

Update() & FixedUpdate()

Tags, Floating Island, Flower Plant, & Flower Bud

Hummingbird Agent

Ray Perception

Heuristic Testing and Challenge

Overview

Anaconda Download & Install

Anaconda Environment Setup

Trainer Config YAML

Training Preparation

Training

Training Progress & Time Scale

Tensorboard

Importing .nn File & Inference

Re-training

Training Wrap-up & Challenge

New Scene & UI Import

UIController & GameManager

More GameManager

Finish Scene & Play

Your Next ML-Agents Project

Live Session - Getting Started with Machine Learning in Unity

In this project, we will set up Unity, import assets for the course, and install ML-Agents. After that we'll create new C# scripts that we'll be using in the next project.

Project Setup

In this project, we will script the functionality of an individual flower. The hummingbirds interact directly with these flowers, so they need to respond appropriately.

Flower.cs

In this project, we will script the Flower Area, which finds and manages all flowers in the hummingbird's environment. This script is also responsible for resetting the flowers when a new training run or game is started.

FlowerArea.cs

In this project, we will script the Hummingbird Agent. This code contains most of the critical functionality for our hummingbird to successfully train, observe the world, take actions, and interact with other objects.

HummingbirdAgent.cs

In this project, we will take the C# scripts we've written and hook them up to objects in our scene. Then we will take control of a hummingbird and fly around the scene, attempting to drink nectar from flowers with keyboard controls.

Scene Setup

In this project, we will set up Anaconda for Python management and train our agents. We'll talk about how to configure training, as well as how to monitor it. Once we have a successfully trained neural network, we'll import it to our project for inference.

In this project, we will add a simple UI and Game Manager so that we can make a small mini game. By the end, you'll be able to compete with an ML-Agent hummingbird.

Minigame

In this project, we wrap up the course. After this, it's your turn to use your imagination and come up with a new ML-Agents project!

Course Wrap Up

HummingbirdScene_1.0.zip

HummingbirdUI_1.0.zip

HummingbirdCompleteProject-1.0-release.zip

MachineLearning_LiveSession_Assets.zip

HummingbirdSourceCode.zip

Reinforcement Learning is one of the most exciting types of Artificial Intelligence and the Unity ML-Agents project is one of the easiest and most fun ways to get started. The instructor for the course is Adam Kelly, an experienced Unity developer who has been creating tutorials and courses on ML-Agents for Immersive Limit since the early beta days in 2018.

In this course, you will learn how to create intelligent flying hummingbirds that can navigate to flowers, dip their beaks in, and drink nectar. These hummingbirds have six degrees of freedom, meaning they can fly and turn in any direction to find targets. They have more complicated controls and their flight paths cannot be solved with traditional navigation systems. You will learn how to craft a training environment and train neural networks to perform this challenging task, then compete against the birds yourself in a simple mini-game.

There's no question that Reinforcement Learning is a complicated subject, but ML-Agents leaves the hard parts to the researchers so that you can focus on solving problems. By the end of the course, you'll have a good understanding of how you can create intelligent agents and integrate them with your own Unity games or simulation projects.
 

2019.3

ML-Agents: Hummingbirds

ee1274e4-bc78-5450-8687-deb8c708dd4d:IJ50VM7ryBj2kIwUvpE7ap

Challenge 1: Solution & Rewards

courseOverview_04.mp4

In this tutorial you will be challenged to think of observations your agent can make about the world around it in an effort to learn about that world.

Games

Take some time to think about what our hummingbird agent might need to observe in order to successfully drink from a flower. Anything that can be represented or calculated as a number can be observed, including Vector3s, Quaternions, booleans, and raycasts. ML-Agents has built-in support for camera vision and raycasts (which act kind of like LIDAR).

The brains of these hummingbirds will not have any memory. That means if they don't observe something at a given moment, they have no knowledge of it — but you can help them out by providing extra observations.

This is just a thought exercise. Write down your ideas, but don't start coding yet.

Challenge 1: Observations

1. Challenge 1: Observations

2. Challenge 1: Solution & Rewards

Complete this tutorial