Develop your critical evaluation skills

Creating real-time experiences is all about making decisions. These choices range from exciting to mundane, and can impact the project as a whole or concern only the tiniest of details. When you’re working on your own project many of these choices, if not all of them, will fall to you, whether you’re working solo or with collaborators.

Making the best possible decisions in a moment can sometimes feel like a mysterious quality that some people have and others just don’t. That’s not actually the case — anyone can work to improve the set of skills that decision-making relies on! Critical evaluation is at the heart of this.

By the end of this tutorial, you'll be able to:

• Explain the importance of critical evaluation in a creative project.
• Identify approaches to obtain and evaluate information required to make a decision in a creative project.
• Consider the role of critical evaluation in your journey as a creator.

Expert creator insights

The tutorial also includes insights on critical evaluation from a group of established creators:

• Sutu Ai Campbell, AR/VR Artist, Director of Sutu Eats Flies and Co-Founder of Eye Jack
• Nora Shramek, Senior Lighting and Rendering Artist at Unity
• Krystel Theuvenin, Faculty Member at the Urban Arts Institute

One of the key competencies you can develop to support your decision-making is critical evaluation.

This is your ability to:

• Identify useful information or resources related to a question, problem, or decision that you need to make.
• Evaluate how significant that information is and what it means in the context of your situation.
• Make decisions that are supported by your evaluation — and understand why you are doing so.

Critical evaluation is very important for creators across every discipline. When you’re working on real-time experiences, there can be complex technical requirements and puzzles you need to solve. You might need to work out the best way to implement a particular feature for a project, identify the best of two approaches to achieve an artistic goal, or even choose the right project to focus on next. You’ll need to work out the best way forward in any situations like these in order to achieve your goals as a creator.

In this tutorial, we’ll share some approaches that you can use to help you develop the skills that support critical evaluation. You can apply these to complex situations that you encounter as a creator or in your wider professional or personal life. It won’t happen overnight; but like a muscle, you can strengthen these skills with practice over time.

Bring your own experience to this tutorial

You can work through the exercises in this tutorial using any kind of complex decision as an example: personal or creative. If you have a creative decision that you’re trying to work out the best approach to already, keep it in mind as you work through the steps — you might find it helpful!

There are many different factors (or variables) that can influence complex decisions for real-time creators. Some of these factors are grounded in facts and others are grounded more in thoughts and feelings. Neither of these broad groups is inherently more important than the other, though only one may be appropriate in a particular scenario. Many variables will actually consist of a combination of facts and thoughts or feelings.

Variables that are strongly fact-influenced include:

• Non-negotiable project requirements: If you’re working on a professional project, there may be set requirements that are out of your control. For example, a fixed deadline may mean that your only choice is to cut a non-critical feature that is not performing correctly close to the launch of your application.
• Resources available: If you only have particular resources available to you, this may restrict your options. For example, if you only have 3D artists in your game jam group then making a 2D game may be very difficult.
• Dependencies: If one system is dependent on another, then your choices may be influenced or dictated entirely by that dependency. For example, the particular render pipeline you select will make different features and possibilities available to you, as you discovered in What are render pipelines?.

Variables that are strongly influenced by thoughts and feelings include:

• Creative vision: Intuition and aesthetic taste can influence your creative vision as much as the understanding you have developed of your craft. This could mean that you start a project with a very clear idea about the outcome that you want to achieve, which would impact many if not all decisions.
• Risk and expediency: Approaches to risk and balancing pragmatic and rigorous approaches are often strongly influenced by personal preferences as well as awareness of key facts that relate to the decision. For example, imagine that you are close to completing a project. You love a particular feature, but there are a number of user experience issues you are struggling to address. What you choose to do in that situation will be strongly influenced by your personal approach to risk and expediency.

Bias and decision-making

We all have biases that can impact our decisions. By becoming more aware of these biases and working to mitigate them, you can improve the quality of your decision-making.

Exploring bias in more detail is beyond the scope of this tutorial, but you may find it very useful to do some research on how bias can impact decision-making in general and for creators in particular.

There’s lots of interesting research and information about unconscious bias if you want to explore this topic further. You might like to start with Outsmart your own biases (Harvard Business Review), an article with a practical focus.

Before you can make a decision, you need to try and gather relevant information to help you form your opinion. For relatively simple decisions, this could be quite a quick process (although sometimes decisions that seem simple are actually quite complex!). For decisions that are wider in scope or with outcomes that are high stakes, this process can become much more involved.

There are lots of different ways to gather the information that you need to make a more complicated decision, but finding or devising a framework to help you find the right questions to ask or research is an excellent way to start.

Depending on your role you may need to do this to a different extent, but it’s important for anyone who makes decisions. Our expert creators shared the importance of asking questions in their own professional contexts:

Note: This tutorial assumes that you’re working as a solo creator, but you can also work through this process as a team of collaborators or with the support of peers.

There are lots of different frameworks that you can research, and it can take time to find or adapt the right one for you. Let’s keep it simple here with four key questions:

• Why is this decision important? This question can be used to clarify your objective in making the decision.
• What do you need to know to properly understand the situation? This question helps you dig down to identify the questions that you need to answer before you can decide.
• Who is or will be impacted by this decision? This question reminds you to consider others — even if you’re making the decision alone and you will bear the impact, it’s an important point to consider.
• What are the risks and how can you mitigate them? This question encourages you to think proactively about addressing risks, either before or after making the decision.

How can I improve at asking questions?

To work out the answers to the four key questions in the framework you just reviewed, you’re probably going to need to identify and get answers to a range of questions that are specific to your particular project.

Here our expert creators share some advice on developing your confidence and asking better questions:

Choose a decision that you made recently to work through this framework and see if it suits you. This can be a creative, professional, or personal decision — it’s exploring and practicing the approach that’s important.

Here’s a basic example to guide you through the process. Imagine that you are trying to decide whether to include a particular feature that you really like in a real-time application prototype:

• Why is it important? Perhaps the feature is something that appeals to you creatively, but is it integral to your target user experience? Perhaps it doesn’t seem to be critical at this stage, but it does enhance the worldbuilding or the narrative design of the experience in a particular way — how important is that to you?
• What do you need to know? Exploring the issue further might include working out: the complexity of implementing the feature to a basic level; whether you can do it yourself or need external support; any dependencies between the feature and other systems in your experience; the current timeline for the prototype and whether including the feature is feasible; the viewpoint(s) of your target users.
• Who is impacted? This could include identifying the people who will: complete the work; test and evaluate the work; support you if there are technical or design issues; be impacted if the feature cannot be delivered or the prototype is delayed.
• How can you mitigate potential risks? Here this could include: working through a low fidelity (lofi) paper prototyping approach to test the feature further before commiting; identifying a cut-off point to decide whether or not the feature is in scope for the prototype, to allow a little further exploration; minimally scoping the feature for the prototype; finding additional support to help you implement the feature.

Each of these sections could be expanded out in more detail. You may find yourself considering your own scenario much more deeply, especially if you chose a more complex creative decision to work through.

Find what works for you

If this particular framework isn’t for you, that’s okay! There are lots of other frameworks out there. Take some time to find or devise something that suits you better.

Just gathering all the information that supports a decision isn’t enough, you also need to evaluate and potentially interpret it in the context of that evaluation.

In the previous steps you asked yourself a very important question: Why is this decision important? The answer to this is a key guiding statement that you can return to throughout your evaluation as you work through the decision that you need to make. It’s useful to keep this in mind throughout the evaluation stage.

Check in with yourself

Before you begin analysing the information that you’ve gathered, it’s often worthwhile checking in with yourself about the decision you need to make or the problem you need to solve.

The following questions can be a helpful place to start:

• Are there any questions related to the decision where you’re working from assumptions? If so — are there more questions you need to find answers to so that you can break through those assumptions?
• Are there any strong thoughts or feelings that are impacting your perspective on the decision? If so — can you identify why you think or feel that particular way?

By being aware of these things, you can engage with them consciously in the decision-making process.

Evaluate your information

It can be helpful to work through a series of questions to help you analyse the information that you’ve gathered and determine how significant it is.

As you develop your skills in evaluation this process can become much quicker, but it can be helpful even for very experienced creators to step back and consciously work through the following questions:

• Who or what is the source of this information?
• How relevant is the information to the decision you’re trying to make?
• How reliable is the information?
• What is the wider context of the information?
• Can you identify any limitations or shortfalls to the information?

You previously worked through the question framework using a decision that you had to make recently. Use that same scenario to work through the process of evaluating information to help you make a decision.

Before you do it yourself, let’s return to the previous example of deciding whether a feature you really like is in scope for a prototype. Here’s an example of evaluating information related to working out how difficult it will be to implement the feature.

The first information source you find is a collection of forum posts from Stack Overflow and the Unity forums. They aren’t tutorials, but they relate broadly to the feature you want to implement — some are general discussion and some are troubleshooting.

Let’s work through the questions:

• Who or what is the source of this information? It’s not always possible to know the exact experience of contributors in forums. In this case they seem to be primarily newer creators rather than those with significant experience.
• How relevant is the information to the decision you’re trying to make? The posts are all about similar features, which is useful. However, some go into troubleshooting aspects in depth and don’t really give you a sense of the overall complexity of implementing the feature for someone with your prior experience.
• How reliable is the information? The opinions and ideas shared seem to come from practical experience with specific projects. In that sense it’s reliable, but it is also limited in scope and not specific to your own situation.
• What is the wider context of the information? Most of the information comes from troubleshooting efforts when solo creators encounter challenges. A little of the content is also a general discussion of different approaches to the general feature type.
• Can you identify any limitations or shortfalls to the information? As the main contributors seem to be newer creators, they may not have comprehensive enough experience to see all potential issues and complexities. They have more experience than you, but perhaps not enough to be reliable enough sources of information to clearly guide you in this. The information also isn’t specific enough to your situation.

After completing this evaluation, you might decide that you need to take one or more of the following actions:

• Ask your own query in the forum to get more specific answers.
• Speak to more experienced creators in your creative network to get advice.
• Find a mentor who can take the time to deeply understand your current competencies and experience and give more detailed guidance on both the feature and what you would need to make it.

Now you’ve read through the example evaluation, apply this approach to information gathered to support the decision you’re working through in this tutorial.

Once you’ve made a decision, your journey as a creator continues! However, the work that you put into working out the best path to take can remain useful as you move forward.

Sharing your decision with others

When you’ve gone through a detailed process of evaluation to support a decision, it can be much easier to communicate both the decision itself and your reasoning behind it to other people with clarity.

This clarity can be incredibly useful when working with collaborators or returning to a previously-made decision at a later stage.

Reflecting on your decision

It’s hard (if not impossible!) to make a perfect decision. As a creator, reflecting on the choices that you made and considering what you could do better next time is an important part of the development of your craft. As with sharing your decision, the evaluation work that you did will help you in this reflection process.

Taking this knowledge from reflection into your next major decision-making process helps create a positive cycle of improvement and knowledge developed through experience.

This tutorial was a first taste of critical evaluation — it’s an ongoing journey, and there’s plenty more for you to explore!

The next time you have to make a complex decision as a creator, try out the frameworks in this tutorial or others that you’ve found to help you in the process.