Generate sprites with Muse

1. Overview

The Muse Sprite Generator is a tool that uses machine learning to generate sprites based on the prompt you enter. You can set the number of sprites to generate. You can also apply the generated sprites directly to objects in your Scene view..

In this tutorial, you will learn to do the following:

• Generate a sprite image using prompts in Muse.

• Add a sprite from Muse to your Unity project.

2. Before you begin

Complete the previous tutorials in this course, and have that project open in the Unity Editor.

In the Unity Editor, go to the main menu and open Muse > New Sprite Generator. If you don't see that option, then you need to install the Muse Sprite package, following one of these methods:

• Go to the Muse Explore page in a web browser, and select Sprite. For detailed steps, refer to Add Muse packages to your project.

• Follow the instructions to Install a Unity package by name, using the name com.unity.muse.sprite.

3. What are sprites?

In Unity, sprites refer to 2D images or animations that are used for rendering graphics. They are typically utilized for characters, objects, and backgrounds in 2D games or for UI elements in 2D interfaces.

In this project, you're going to create a sprite of autumn leaves on the ground to be used as a decal in the scene. Decals add details to surfaces in a realistic way. They are typically used to represent things like signs, graffiti, dirt splatters, worn-out paint, bullet holes, and more. You can read more about decals in the Unity Documentation.

4. Generate a Sprite

Next, you'll generate a sprite to use as a decal in the scene.

Currently, decals are used to show cherry blossom leaves on the ground in the scene.

You will generate a different leaf decal and replace the cherry tree leaves.

We've provided an example, but use your own ideas to generate any sprite you like as a new covering for the ground.

1. From the Unity Editor's main menu, select Muse > New Sprite Generator to open the Muse Generator window. Resize or dock the window as you see fit.

2. Set the number of sprite variations you wish to generate at a time. Each of the generated images will be a slightly different version of the sprite based on the prompt.

3. Add a prompt for your sprite.

In our example, we'll use: "Collection of autumn leaves on the ground."

4. Add negative prompts if you specifically want to remove or avoid elements in a sprite.

We'll enter "no soil" so that only leaves are generated.

5. Enable the Remove Background option to ensure that your sprites have an alpha map applied, which will make any empty areas transparent.

6. Select Generate to start the sprite generation process

7. If you are happy with the generated images, you can export them as sprites to your project, or run the generation again for more options.

To export, right-click on the image you wish to use, the select Export.

8. Give the sprite a name and save it in the Assets folder.

9. In the Assets folder, select the sprite you just generated. Notice in the Inspector that the Texture type is already set to Sprite (2D and UI). This means it is ready to use as a sprite in your project.

5. Override the existing leaves

Let's apply the new sprite as the texture in the material that is already in use on the ground.

1. In the Assets folder, go to the Assets > Scenes > Garden > Art > Terrain > Materials folder.

2. Select the CherryPetalDecal_Mat material.

3. From the Project window, drag the sprite you created into the Inspector window, to the Base Map property, to replace the current cherry blossoms sprite.

4. Disable the WorldSpaceUVs option and adjust the scale until you are happy with the result.

6. Next steps

Now you know the basic features of Muse's sprite generator, and you can easily create your own sprites for UI, decals, or 2D characters. In the next tutorial, you'll use Muse to add animations to a character.