Get started with Muse Chat

This tutorial will introduce you to Muse’s chat capability – a generative and conversational AI interface and in-Editor Chat-based tool that has been created to answer your questions about Unity's technology and products. You can chat with Muse to solve challenging problems, get ideas for your project, and generate usable code. The large language models employed by Muse Chat are specifically designed to provide you with the most accurate and up-to-date information about Unity.

What are large language models?

Large language models (LLMs) are advanced computer programs that can understand and generate human language. They work by processing vast amounts of text data to learn patterns and meanings, which allows them to answer questions, write essays, translate languages, and even have conversations. These models use complex algorithms and huge computing power to perform these tasks, which makes them valuable tools for various applications, from chatbots to content generation. Muse Chat is powered by third-party LLMs combined with our in-house built scaffolding system.

New to Unity?

If you’re new to Unity, welcome! If you like, you can begin your Unity learning journey with Muse without any previous experience with Unity. The Chat feature can even help you install and set up the Unity Hub and Unity Editor, and answer any questions you have along the way.

As you learn Unity, Muse Chat will recommend the Unity Essentials learning pathway. You can even use Muse Chat and Unity Essentials side-by-side!

Important: You’ll need a Unity ID to use Muse Chat. If you don't already have a Unity ID, please refer to Install the Unity Hub and Editor to get started.

Subscribe to Muse

To subscribe or begin your free trial of Muse, go to the Muse explore page. Make sure you’re signed in with your Unity ID.

To add Muse Chat to your project, follow these instructions:

1. Visit the Muse Explore page.

If you aren't already logged in with your Unity ID, you'll be prompted to log in.

2. Select Get started on the Chat tile, then select Install the Chat package to add Muse Chat to your project.

3. Once the package is installed, select Muse from the main menu, select the Chat submenu item, then click and drag the Muse Chat window beside the Inspector window to dock it.

Now it's time to try it out; just start typing In the Ask Muse box! Here are some examples:

• I want to build my first game in Unity. What types of games are the simplest to build for a beginner?
• What is Unity? Simplify this concept.
• How do I add physics simulation to my game object?
• How do I make a peanut butter and jelly sandwich?

Tip: This is a trick question!

• How do I make a peanut butter and jelly sandwich in Unity?

You'll see that you can ask Muse anything…as long as it's about Unity. For other questions, use general purpose conversational AIs.

As you enter prompts, check out the Additional Resources section of the Muse Chat window. It updates dynamically to provide you with links to more information that are relevant to your chat.

To ask questions specific to particular objects in your project, you can use Muse Chat’s attach items function.

Here’s how you can attach different elements:

• GameObjects: Drag and drop items from your project’s Hierarchy window into the Muse Chat window to get more precise responses to your queries.
• Assets: Attach items from your Project folder to provide targeted advice, then drag them into the Muse Chat window. For example: “How can I adjust this material to achieve a more realistic metal look?”
• Code: Attach a script from your assets, then ask Muse to analyze the script or describe its purpose.
• Console logs: Select a console message, attach it in the Muse Chat window, and ask Muse for potential solutions.
• Multi-select: Drag and drop to attach items and reference them across both your Hierarchy window and Asset folder.

To utilize attachments, simply drag and drop, or select Attach Item next to the chat input field and select the item(s) you want to include with your prompt. Then, to further draw focus to the attachment, you can refer to it in your question; “How do I fix this [console error]?” or “Why are the two attached objects not bouncing off each other. What should I check?”

Muse's chat capability is built upon LLMs that allow you – even encourage you – to use natural language to conduct a conversation. Prompt engineering describes techniques you can use to increase your chances of getting an appropriate answer to your question.

In the examples above, "Simplify this concept" is an example of prompt engineering. The conversational AI has the power and versatility to express information in many different ways, or dig down to the level of detail you need, depending on the context and directives you give it.

Tips for prompt engineering in Muse

• Keep it concise: When you're interacting with Muse, remember that it’s not a human but a machine, so your prompts don't have to be grammatically correct. Instead, focus on providing as much context as possible without adding too many unnecessary words.
• Ask for the information sources you want: For example, "Recommend a tutorial that will help me learn about post-processing." Or, "Find documentation about the 2D animation rigging package"
• Tell Muse what you already know: To get a response with the appropriate level of detail, make sure to give Muse as much information as possible. For example, if you have some experience in lighting and you ask Muse "How do I design lighting for an outdoor scene?" you're likely to get basic information you don't need. Instead, try: "Advanced lighting user: What considerations when lighting outdoor scene?”. You could also say "Entry-level technical artist: What do I need to know about shaders?" or, "I'm a brand new user. I want to make a variation of a Sudoku game."
• Let Muse know what you've already tried: Provide Muse with details on what is working or not working. For example, you can tell Muse how your code is behaving, paste your script into the chat, and ask Muse how to accomplish your next step. You can also paste snippets of code asking Muse why it’s not working, then ask to adjust the code by implementing the identified faults. For example, “Troubleshoot: [your code]” and then if applicable “Implement changes.”.
• Use Muse as your personal tutor to help you create a plan to learn about the Unity Editor or a specific Unity feature or product. For example, tell Muse to “Act like an experienced power user: create a 10-day plan to learn the most important information about quaternions, emphasizing practical implementation”. Remember that you can pick up this conversation again in your chat history.
• Use Muse as your coding buddy: If you would like to generate a script rather than writing it yourself, you can ask Muse to do that. For example, “Write a script for drag and drop functionality using my mouse” and it will return a fully functioning script to plug and play into a component and use within seconds.
• Work with Muse in steps: If you're looking for instructions in multiple steps, you can frame your message in multiple steps, rather than as one long complicated question. Working in steps will improve the quality and the structure of the responses. You can also reply with prompts such as, "Regarding #5 above, how do I start?"

Overall, you'll get the best results if you state your intent and your frame of reference clearly up front and use additional prompts to explore the details. Remember that each chat session is a conversation!

For more tips on prompt engineering for Muse Chat, check out our blog post on this topic.

Muse's chat capability is very good at helping you find Unity resources — but it can do so much more! You can ask Muse Chat to do the following:

• Explain how to get a specific effect that isn't covered in any documentation or tutorials. For example: "How can I write a shader for brown muddy water?"
• Write scripts that you can attach to your GameObjects. (Later in this project, you'll use scripts from Muse Chat when you create your own game.)
• Add context to previously already written code. You can ask Muse to document, comment, or improve code.
• Curious about the Universal Render Pipeline (URP)? We have recently ingested URP documentation. Now you can get not only written descriptions but also images. Just ask: “Show me how the frame rendering loop works in URP.”

If you want more best practices on how to use Muse Chat, see the prompt guide in the Muse menu.