Instructor Introduction & ML-Agents Overview

2019.3

Traditional Game AI vs. ML-Agents & Hummingbird Course Preview

Observations

Challenge 1: Observations

Challenge 2: Rewards

Initial Setup, Asset Import & Default Contact Offset

Asset Overview & Lighting

Install ML-Agents & Script Creation

Explanation

Variables and Accessors

Feed()

ResetFlower()

Awake()

ResetFlower(), GetFlowerFromNectar(), Awake() & Start()

FindChildFlowers()

Variables & Accessors

Initialize() & OnEpisodeBegin()

MoveToSafeRandomPosition()

UpdateNearestFlower()

OnActionReceived()

CollectObservations()

Heuristic()

Freeze() & Unfreeze()

OnTriggerEnterOrStay()

OnCollisionEnter()

Update() & FixedUpdate()

Tags, Floating Island, Flower Plant, & Flower Bud

Hummingbird Agent

Ray Perception

Heuristic Testing and Challenge

Overview

Anaconda Download & Install

Anaconda Environment Setup

Trainer Config YAML

Training Preparation

Training

Training Progress & Time Scale

Tensorboard

Importing .nn File & Inference

Re-training

Training Wrap-up & Challenge

New Scene & UI Import

UIController & GameManager

More GameManager

Finish Scene & Play

Your Next ML-Agents Project

Live Session - Getting Started with Machine Learning in Unity

2019.4

Get introduced to Adam Kelly, the instructor who will guide you through this course.

Compare traditional game AI workflows with machine learning approaches and see how ML can change game behavior design.

Explore how an agent observes its environment as a basis for learning about the world.

Identify meaningful observations an agent can make about its environment to support learning.

Explore strategies for defining reward signals that encourage effective agent behavior.

In this project, we will start with an overview of ML-Agents. Then we will talk about the hummingbird project and what kinds of observations and rewards we need for successful training.

Course Overview

Set up a new Unity project and import assets to prepare your development environment.

Review the key GameObjects and layout of the scene used throughout the course.

Learn to install the ML-Agents package in Unity and configure the required scripts.

In this project, we will set up Unity, import assets for the course, and install ML-Agents. After that we'll create new C# scripts that we'll be using in the next project.

Project Setup

Learn how to initialize and run the flower script inside the Unity project.

Start defining your flower class in C# by adding and using public variables.

Learn to write the first function for your flower class in C#.

Start building a reset function to restore your game state or scene to its initial setup.

Complete the flower classification logic by implementing and using a final function in your class.

In this project, we will script the functionality of an individual flower. The hummingbirds interact directly with these flowers, so they need to respond appropriately.

Flower.cs

Learn to build and iterate on a FlowerArea C# script to support your project’s behavior.

Learn how to define and set up the variables used in the FlowerArea class.

Create three core functions for the FlowerArea class to support its behavior.

Learn how to implement the FinChildFlowers function step by step in your script.

In this project, we will script the Flower Area, which finds and manages all flowers in the hummingbird's environment. This script is also responsible for resetting the flowers when a new training run or game is started.

FlowerArea.cs

Begin implementing the HummingBirdAgent script to control the agent’s behavior.

Practice adding variables, starting with public variables, at the top of your C# class in Unity.

Create the first two C# override methods for your behavior logic and understand how they customize base functionality.

Learn how to implement the MoveToSafeRandomPosition function step by step in your project.

Learn to implement the UpdateNearestFlower C# function to locate the closest flower in your Scene.

Learn how to create and use the OnActionReceived method in your Unity script.

Learn to implement the CollectObservations public override so your agent can gather data for training.

Create a Heuristic function to control decision-making behavior in your system or agent.

Learn how to freeze and unfreeze an agent to control its behavior during a simulation.

Write the collision-handling code that responds when an agent hits objects in its environment.

Practice handling an additional collision scenario to make your gameplay more robust.

Add a final function to the HummingbirdAgent script to handle an additional collision scenario.

In this project, we will script the Hummingbird Agent. This code contains most of the critical functionality for our hummingbird to successfully train, observe the world, take actions, and interact with other objects.

HummingbirdAgent.cs

Learn to configure prefabs in your scene so they are ready for use in your project.

Set up the Hummingbird Agent in Unity and prepare it for training.

Learn how ray perception gathers information from the game world for use in your Unity project.

Practice interacting with and evaluating the hummingbird and flower setup to prepare for further experimentation.

In this project, we will take the C# scripts we've written and hook them up to objects in our scene. Then we will take control of a hummingbird and fly around the scene, attempting to drink nectar from flowers with keyboard controls.

Scene Setup

Get an introductory overview of how training works when using ML-Agents in Unity.

Learn how to download and install Anaconda to prepare your development environment.

Learn to configure an Anaconda environment to install and manage ML-Agents.

Get an overview of the Trainer Config YAML file and how it defines training settings.

Set up a training scene and run the ML-Agents Python command from a virtual environment.

Discover how to run the training command to start a new training session.

Learn how to monitor and understand training progress directly within the scene.

Get an introduction to Tensorboard and how it supports visualizing training data and metrics.

Learn how to run inference in Unity using a fully trained agent.

Learn the core ideas behind continued training when working with ML-Agents in Unity.

Experiment with changing parameters and predict how those adjustments will affect the results in your scene.

In this project, we will set up Anaconda for Python management and train our agents. We'll talk about how to configure training, as well as how to monitor it. Once we have a successfully trained neural network, we'll import it to our project for inference.

Set up a new Scene for your minigame and add basic UI elements to support gameplay.

See how the minigame’s user interface interacts with the rest of its systems.

Examine the GameManager script to understand how it controls and coordinates your game.

Learn to finalize your scene so the minigame is ready to play.

In this project, we will add a simple UI and Game Manager so that we can make a small mini game. By the end, you'll be able to compete with an ML-Agent hummingbird.

Minigame

Review the ML-Agents concepts covered and get guidance on how to continue developing your projects.

In this project, we wrap up the course. After this, it's your turn to use your imagination and come up with a new ML-Agents project!

Course Wrap Up

ML Agents Hummingbirds 2019.3 Common Cartridge File

Below you can download a CSV outline and a Common Cartridge file for this course.


Open the CSV outline in spreadsheet software (Excel, Google Sheets) for a summary of all content in the course, with links to each tutorial, exercise, and quiz.


Use the Common Cartridge file to import the course content into your Learning Management System (such as Canvas, Moodle, Blackboard, etc.). See instructions from your LMS provider for importing Common Cartridge files.

HummingbirdScene_1.0.zip

HummingbirdUI_1.0.zip

HummingbirdCompleteProject-1.0-release.zip

MachineLearning_LiveSession_Assets.zip

HummingbirdSourceCode.zip

Reinforcement Learning is one of the most exciting types of Artificial Intelligence and the Unity ML-Agents project is one of the easiest and most fun ways to get started. The instructor for the course is Adam Kelly, an experienced Unity developer who has been creating tutorials and courses on ML-Agents for Immersive Limit since the early beta days in 2018.

In this course, you will learn how to create intelligent flying hummingbirds that can navigate to flowers, dip their beaks in, and drink nectar. These hummingbirds have six degrees of freedom, meaning they can fly and turn in any direction to find targets. They have more complicated controls and their flight paths cannot be solved with traditional navigation systems. You will learn how to craft a training environment and train neural networks to perform this challenging task, then compete against the birds yourself in a simple mini-game.

There's no question that Reinforcement Learning is a complicated subject, but ML-Agents leaves the hard parts to the researchers so that you can focus on solving problems. By the end of the course, you'll have a good understanding of how you can create intelligent agents and integrate them with your own Unity games or simulation projects.
 

ML-Agents: Hummingbirds

0e6ad105-3f5f-5980-9a27-fcd8120b0b00:5wZ4XsXcAZy0HxzXD44xDa

MLAgentsLiveSessionAssets.zip

MachineLearning_LiveSession_TrainTheAgent.JPG

Live Session Recording - October 20, 2020

Share your project!

A gameplay walkthrough of the humming bird

Unity ML Agents Project: Humming Bird

Humming bird 

Madhur_98

My first reinforcement learning project!  Trained a Hummingbird Agent to suck nectar from flowers and turned it into a minigame!   Play Link:    https://play.unity.com/en/games/79fcf10f-5e1c-44f9-a535-47da102fc09f/hummingbirds

RL_Hummingbirds

iswangshaoxuan

https://github.com/TitoBotelho/ml-agents-library-unity

Simple RL Problem

titob854

Obstacle_Dodger.mp4

Obstacle Dodger

furqan06

Kart Agent

princenizroh7

It is a drone that uses ray sensors to locate and identify its target, and once it has identified it, it tries to fly as close as possible and ram into the target.

modelDemo.mp4

ML Drone Agent

AlexUnderCover_

GMT20201020-164529_Getting-St_1920x1080.mp4

In this recording of the live session, you'll learn how to create a machine learning experience that has a basic input (one or two dimensional vector observation), an obvious reward, and simple reset state. Throughout the session, learners will be able to:

Discuss how ML differs from and works with Artificial Intelligence

Identify observations for an ML agent to make in a given Scene

Identify ways to reward the ML agent that will reinforce desired behavior

Script the ML Agent to collect observations about conditions in the scene and calculate rewards accordingly

Script the ML Agent to change behavior based on rewards received

Games

Download the screenshot of challenge #4 from the Slides 

We would love to see what you were able to create after learning about Machine Learning! Please take a screenshot of your project or do a screen-recording walking us through it, then post it here to share your creation. 

Live Session - Getting Started with Machine Learning in Unity

1. Live Session Recording - October 20, 2020

2. Share your project!

Please login to submit

Complete this Tutorial