Scene Construction

In this section, you’ll create a Prefab for the baby penguin.

1. Double-click on the BabyPenguin Prefab in the Prefabs folder to open it.

2. Add a Rigidbody component.

3. Set the Rigidbody Constraints to lock position in X and Z and to lock rotation in X, Y, and Z so that the baby doesn't accidentally get knocked around, but can still be affected by gravity

4. Add a Sphere Collider.

5. Set the Sphere Collider Center to (0, 0.24, 0).

6. Set the Sphere Collider Radius to 0.25.

7. Change the tag to "baby" (you will need to create a new tag first).

In this section, you’ll create a Prefab for the fish that the penguin agent needs to catch.

1. Double-click on the Fish Prefab in the Prefabs folder to open it.

2. Add a Rigidbody component.

3. Set the Rigidbody Constraints to lock rotation in X and Z so that it doesn't flip over

4. Add a Capsule Collider that encapsulates the fish.

5. Set the Capsule Collider Center to (0, 0, -0.08).

6. Set the Capsule Collider Radius to 0.15.

7. Set the Capsule Collider Height to 0.66.

8. Set the Capsule Collider Direction to Z-axis.

9. Change the tag to "fish" (you will need to create a new tag first).

10. Attach the Fish script.

In this section, you’ll create a penguin agent that can observe its environment and take actions using deep learning. We won’t train it yet. We’re just setting it up for now.

1. Double-click on the Penguin Prefab in the Prefabs folder to open it.

2. Add a Rigidbody component.

3. Set the Rigidbody Constraints to lock rotation in X and Z so that it doesn't flip over.

4. Add a Capsule Collider.

5. Set the Capsule Collider Center to (0, 0.22, 0.13).

6. Set the Capsule Collider Radius to 0.24.

7. Set the Capsule Collider Height to 1.41.

8. Set the Capsule Collider Direction to Z-axis.

9. Attach the PenguinAgent script.

10. If the BehaviorParameters script is not added automatically, add that as well.

11. Set up Behavior Parameters (Figure 04):
a. Behavior Name: PenguinLearning
b. Space Size: 8
c. Branches Size: 2
d. Branch 0 Size: 2
e. Branch 1 Size: 3

12. Set up Penguin Agent (Figure 04):
a. Max Step: 5000
b. Decision Interval: 4
c. Drag the Heart Prefab from the Project tab into the appropriate box.
d. Drag the Regurgitated Fish Prefab from the Project tab into the appropriate box.

The Behavior Name will need to match configuration files that you will make in the Training and Inference tutorial. Make sure that it’s spelled exactly the same or else training will not work.

The Vector Observation Space Size corresponds to the AddVectorObs() calls we made in CollectObservations() in PenguinAgent.cs. We have eight total values.

The Vector Action Branches Size indicates how many actions are possible. In our case, these are Move Forward and Turn, so we have two. Branch 0 is Move Forward, and we have two options: don't move and move forward. Branch 1 is Turn, and we have three options: turn left, don't turn, and turn right.

Max Steps means the agent will reset automatically after 5,000 steps. Decision Interval of 4 means that the agent will be asked what to do every four steps and will carry out that action between each request.

13. Add a RayPerceptionSensorComponent3D component.

14. Set up the RayPerceptionSensorComponent3D (Figure 05):
a. Detectable tags size: 3
b. Element 0: baby
c. Element 1: fish
d. Element 2: Untagged

Detectable Tags tells the sensor which tags to report collisions for. Rays Per Direction tells the sensor to cast two rays to either side of center, as well as straight ahead. Max Ray Degrees tells the sensor to spread the two rays out 60 degrees in either direction. The total spread is 120 degrees, 30 degrees between each ray. Sphere Cast Radius projects a sphere of radius 0.5 along each ray to test collisions, not just a single point in space.

The screenshot below (Figure 06) shows spherecasts, two of which have hit fish and two of which have hit the walls of the environment. The white line seems to have somehow passed through the wall. This is a reminder that Unity Physics is not flawless, but fortunately our ML-Agents are robust enough to work despite occasional misleading or incomplete information.

In this section, you’ll create an enclosed space for the penguin agent to catch fish and feed its baby. ML-Agents can work in environments of any shape and size, but this tutorial uses a small, confined space to keep things simple.

1. Double-click on the PenguinArea Prefab in the Prefabs folder to open it.

2. Hold the Ctrl key on your keyboard and click on InvCylinder_Collider, RockCollider_01 and RockCollider_02 to select all three (Figure 07). These 3d Meshes are intended to be simplified Mesh Colliders and will not be visible.

3. Remove the Mesh Renderer component using the drop-down menu in the Inspector as shown (Figure 08). This will make the Colliders invisible to the camera.

4. Click the Add Component button and add a Mesh Collider component to all three objects (Figure 09).

5. Create a new folder called Materials inside Assets\Penguin.

6. Create a new Material called Snow in the Materials folder.

7. Set the Snow Albedo/Color to 255, 255, 255, 255.

8. Set the Snow Smoothness to 0.

9. Create a new Material called Water in the Materials folder.

10. Set the Water Rendering Mode to Transparent.

11. Set the Water Albedo/Color to 0, 200, 255, 165.

12. Set the Water Smoothness to 0.

13. Apply the Snow Material to the iceberg and rocks in the area. (You can apply Materials to multiple objects at once by Ctrl + clicking each object and dragging the Material into the Inspector tab).

14. Apply the Water Material to the water.

15. Create a new Text Mesh Pro object by right-clicking on PenguinArea, and choosing 3D Object > Text - TextMeshPro (Figure 11).

16. Click the Import TMP Essentials button if prompted.

You should now have a 2D text object floating in 3D space.

17. Rename the object to Cumulative Reward (TMP).

18. Move the text so that you can see it:
a. PosX: 7
b. PosY: 2
c. PosZ: 11
d. Width: 20
e. Height: 5
f. Rotation (0, 30, 0)

19. Set the Text settings:
a. Default text: 0.00
b. Font Size: 30
c. Vertex Color: Black
d. Alignment: Center Horizontal and Center Vertical

20. Add a BabyPenguin Prefab to the area.

21. Add a Penguin (agent) Prefab to the area.

22. Add a PenguinArea script component to the PenguinArea.

23. Drag the Penguin from the Hierarchy to the Penguin Agent field in the PenguinArea.

24. Drag the BabyPenguin from the Hierarchy to the Penguin Baby field in the PenguinArea.

25. Drag the Cumulative Reward (TMP) to the Cumulative Reward Text field in the PenguinArea.

26. Drag the Fish Prefab from the Project tab to the Fish Prefab field in the PenguinArea.

27. Exit the Prefab editor by clicking on the Back (<) arrow in the Hierarchy tab.

The Academy object manages training or inference for multiple agents and areas in a Scene. There will be multiple areas in your Scene, but only one PenguinAcademy.

1. Create a new empty GameObject and rename it PenguinAcademy.

2. Add the PenguinAcademy script to the object.

3. Expand the PenguinArea in the Scene view so that you can see its children objects.

4. Select the Penguin in the PenguinArea.

5. Change the Behavior Type (Figure 19) of the Behavior Parameters component to Heuristic Only (you will revert this change after you test it).

This will not modify the Prefab, but it will allow you to test the Scene by playing as the penguin.

6. Move the Main Camera so that it’s pointing at the area:
a. Position: (-8, 9, -20)
b. Rotation: (30, 22, 0)

7. Press the Play button at the top center of the Unity Editor.

You should be able to control the movement of the penguin with the W, A, and D keys on your keyboard. Try to pick up fish and deliver them to the baby. Once you are within the six meter feed_radius, the penguin should feed the baby and the Cumulative Reward text should increase. When you deliver all four fish, the area and reward should reset.

8. Change the Behavior Type back to Default after you’ve successfully tested the penguin.

Before moving on to training, you should create several copies of the area so that several penguins can train simultaneously. This step is optional, but should help training go faster.

9. Duplicate the PenguinArea seven times so that you have a total of eight areas.

10. Space out the areas by about 40 meters in X and Z so that they do not overlap.

You should now have Prefabs for the penguin agent, baby penguin, fish, and area as well as a Scene set up and ready for training.