Part 2: Data design

Mike Acton gave a particularly clear description of core DOD principles in his 2019 WeAreDevelopers talk, “Building A Data-Oriented Future” which is well worth watching. The principles start at about 22:17, and can be summarized as follows:

• The (global) energy required to transform some data should be proportional to the amount of surprise. This is what we mean when we say “design for the common case”.
• The purpose of all programs, and all parts of all programs, is to transform data from one form to another. Even OOP programs exist to transform data, although the abstractions they employ can make them fundamentally inefficient at doing so. DOD means working out what format is most appropriate for your data, and designing a pipeline to transform that data as required.
• If you don’t understand the data, you don’t understand the problem. How would you sort a deck of cards? What if there are only 10 cards? What if there are a million? What if 99% of the cards have the same value? These are fundamentally different problems. Following on from this point…
• Different problems require different solutions. Don’t try to build abstractions that can solve any theoretical problem, because such abstractions will by their nature be inefficient. Solve the problems you actually have.
• If you have different data, you have a different problem. This is the reason why this guide doesn’t contain lots of high-level advice on gameplay systems. There’s no “here’s the best way to make an inventory system with DOTS”, or “here’s the best way to build an AI state machine with DOTS”, or even “here’s the best way to perform data denormalization”, because the answer to all of these sorts of questions is that it depends on the data you need to process.
• If you don’t understand the cost of solving a problem, you don’t understand the problem. Define your constraints. Set performance targets for solving a problem. Design your data and your transformation pipeline to work within those constraints.
• If you don’t understand the hardware, you can’t reason about the cost of solving the problem. How many CPU cores are available to you? How much memory? How many CPU cycles does a memory cache miss cost, compared to the cost of actually executing an instruction? How large are the CPU cache lines and how much of them are being wasted by inefficient memory layouts of your data? The answers to these questions will inform you of the constraints your data design must adhere to.

These principles can help to shift your thinking away from OOP’s focus on abstractions and generalizations, and towards a DOD mindset. The main takeaways are to solve for the specific problem you have within the constraints of your target hardware and performance requirements; then work out what data you need to transform to solve your specific problem, and how to arrange that data to make the transformation process as efficient as possible.

Once you understand the fundamentals of data-oriented design, you are ready to design some data and systems for your application. Because you’re now thinking in terms of data (in components) and code to transform that data (in jobs scheduled from systems), the best place to start is to figure out what data you actually need, and what transformations you need your code to perform on the data to achieve the behavior you want.

Additionally, the transformations should give you an idea of the expected read/write access patterns for the data, which then determine how best to structure the data into entities and components for efficient processing.

The Unite Copenhagen 2019 presentation Options for Entity Interaction is an excellent introduction to data design, and demonstrates how considering the specific details of your use-case can help you to select the best tools and data structures. Some of the specific APIs referenced in the talk are now obsolete: for example, ComponentLookup replaces ComponentDataForEntity, and IJobEntity replaces IJobForEach. However, the overall advice on how to approach entity interactions in a data-oriented way remains invaluable.

Here’s a Breakout Data Design Worksheet which shows an example of what the data design might look like for a data-oriented version of the classic Atari game Breakout. Use similar worksheets for your own applications. This saves a lot of time during the implementation of your application, because it helps you to catch many issues with the data design early, and the design should make it quick and easy to figure out which components and systems you need to implement.

Here are some things to consider when you design data.

The goal of DOD is high performance. You should arrange your data in a way that enables the systems that operate on that data to access it in the most efficient way.

The single most important principle in designing data for performance is to minimize CPU cache misses and maximize CPU cache hits. As described in many of the learning resources in Part 1 of this guide, when a CPU needs to process data and finds it already in a cache line, accessing the data is very fast for the CPU. This scenario is called a cache hit. When the data is not in a cache line, that’s called a cache miss, and it causes the CPU to have to wait a long time to fetch the data from main memory.

In DOD, you should forget about the higher-level generic containers that you might be used to: for example Dictionaries, or linked lists, which involve a huge amount of random memory access, resulting in many cache misses. Instead, think in terms of simple, predictable, linear data structures: arrays and lists (or rather, NativeArray and NativeList).

ECS stores component data in arrays packed inside chunks. It’s efficient to process a whole chunk (or several chunks) of components at once because it maximizes CPU cache hits. Conversely, looking up a component of a specific Entity is a random access, which makes it just as inefficient as a Dictionary lookup. Iterating through a list of Entities and accessing their component data one at a time is an anti-pattern that you should avoid wherever possible. To properly leverage the linear memory organization, code should process components through an EntityQuery.

Often, the best way to achieve fine-grained EntityQueries is to construct entities from a large number of small components to get fine-grained control over entity archetypes, rather than to construct a smaller number of large components. Additionally, smaller components may result in much more efficient use of the CPU cache.

Given that a typical cache line is only 64 bytes, once the CPU has done the slow work to fill that 64 bytes with the data your application asked for, it makes sense to use that data as efficiently as possible.

When you design a component for the ball in the Breakout example, it might be tempting to keep all of the data in one component like you would in OOP if you were creating a “ball” class:

public struct Ball : IComponentData  
{  
    public float2 Position;  
    public float2 Direction;  
    public float Speed;  
    public float2 Size;  
    // ... etc...  
}  

This is fine if all of the systems that process Ball components each need to access every piece of this data. But different systems operate on different subsets of this data. For example, the Rendering system requires the Ball’s Position and Size but doesn’t need the Direction and Speed. But if all of this data is packed into the same component, the CPU will load all of it into the cache whenever any of it is needed, filling the cache with data which isn’t being used in the current data transformation, and reducing the overall efficiency of that system.

If your data design keeps all of the data in separate components, the Rendering system can have a cache line that accesses packed Position components, and another cache line that accesses packed Size components, and then it does useful work on all of the content of those caches.

Traditional OOP techniques (such as working with GameObjects and MonoBehaviours) confine the majority of user code to the main thread, which means you only get to use a fraction of the potential processing power of most modern CPUs. However, multithreaded code typically takes longer to write, because of the amount of time you need to spend debugging subtle problems like race conditions.

The Unity job system makes it much easier to write multithreaded code without race conditions because it contains a safety system which effectively bans code that would allow race conditions to occur. To be able to check for potential race conditions efficiently, the job system introduces a constraint: you can only create jobs that operate on blittable data types.

Blittable data is data that can be copied into memory from disk or from another part of memory, in a single block copy operation, with no need to fix broken data references once it has been copied. Blittable data types include int, float, bool, and structs that contain only those types. Data types which live on the managed heap and involve references to memory locations, such as classes, strings and managed generic containers are not blittable. You should design your data to use only blittable types.

Another major advantage of using blittable data is that the code that transforms it can be compiled by Burst, which results in highly-optimized native code. You can apply the Burst compiler to all jobs and methods that you write in High-Performance C# (HPC#), a subset of C# .NET which excludes non-blittable managed data types. For more information about what data types, keywords, and language features are compatible with HPC#, see the C# Language Support pages of the Burst User Guide.

Familiarize yourself with what constitutes Burst compatible code so that you can Burst compile as many of your jobs as possible.

On the worksheet, each of the data transforms specifies an input and an output. For any given transformation, data that’s used as an input and not modified as an output is read-only, and you should declare it as such (as we’ll discuss below) when implementing your systems.

It’s important to declare read-only data because it allows the job scheduler to safely parallelize the jobs that process that data. This in turn gives the job scheduler more options to figure out how to arrange scheduled jobs, which gets the most efficient usage of the available CPU threads. Correctly declaring data as read-only is also important in projects which contain reactive systems (that is, ones that only update when the data changes). Accessing data as read/write causes those reactive systems to run even if the data hasn’t actually changed. For these reasons, you should separate data that is read-only (in some transformations) into different components from data which is read/write.

If your data design worksheet shows data that is read-only in every transformation, the data should perhaps instead be stored in a BlobAsset. A blob asset is an immutable data structure stored in unmanaged memory which can contain blittable data, as well as structs and arrays of blittable data, and can also contain strings in the form of BlobString. Because blob assets are stored on disk in binary format, they can be deserialized much more quickly than OOP assets such as ScriptableObjects, and because they’re not stored in chunks, they don’t contribute to chunk fragmentation, so they don’t get in the way of processing your mutable component data.

As is mentioned in the Key Principles section, when you consider how to design your application’s data so that it can be processed efficiently, it’s important to consider how often your code will need to access each piece of data. The worksheet specifies the quantity of each piece of data, and a space to calculate the expected read/write frequency based on the inputs and outputs of the data transformations. This should give you a clear picture of what the most common data access patterns are likely to be.

Structure the data to make the most common operations have the most efficient access (for example, you should prioritize the efficiency of the operations that happen 100,000 times a frame over those that happen once per frame). Perhaps this means that a transformation that only happens once per frame involves a lot of random memory access and cache misses as a side effect. Or perhaps it means that you have to do some additional housekeeping once per second to update the data structures that make the common transforms more efficient. Or it might mean that you have to perform some heavy data processing to instantiate and initialize entities. All of those things are okay so long as they’re helping your most frequent operations to be as fast as possible. If you optimize the common case first, you can always check the Profiler to see if the other cases are unacceptably slow and revisit those later.

What this means in practice is that you should plan EntityQueries well. Ideally, you want the right EntityQuery so you can write a job that does useful work on every entity it iterates over.

The job system and Burst support a number of primitive types, including various sizes of integer and floating point types, and bool. However, the job system and Burst don’t support the C# string type except in very limited circumstances, because strings are managed types. Additionally, strings are very inefficient data types for most purposes: they are generally only useful for UI displays, file names and paths for loading or saving, and for debugging. UI and file I/O are generally outside the remit of DOTS code in Unity, and you should debug your code with string literals (as discussed below).

For most other internal purposes, convert human-readable string identifiers to a blittable, runtime-friendly format for faster processing. Depending on your use case, this could be an enum, a straightforward integer index, or perhaps a hash value calculated from the string. The Entities package provides the XXHash class which can generate 32-bit or 64-bit hashes for this purpose.

String exceptions

If you need to use strings, there are a few DOTS-friendly options. The Collections package contains a number of string types of various sizes, from FixedString32Bytes up to FixedString4096Bytes.

You can use FixedStrings in ECS components and in jobs, although depending on the length of the strings you want to represent, it might end up wasting memory. If you want to store strings inside blob assets, use BlobString. You can use some limited forms of string handling for the purposes of debug logging (see Burst user guide: String Support).

Familiarize yourself with how data conversion is performed for Entities. The key takeaways from a data design standpoint are the following:

• Authoring data is optimized for flexibility
  • Human understandability and editability
  • Version control (mergeability, no duplication)
  • Teamwork organization
• Runtime data is optimized for performance
  • Cache efficiency
  • Loading time and streaming
  • Distribution size

In other words, while this Best Practice Guide focuses on creating and processing the optimal runtime data structures, you should also consider the data format that you want to use to author this runtime data, and how you’re going to convert it from the authoring data format to runtime data.