The examples in this workflow will be in Maya. This workflow assumes you have advanced rigging skills in Maya and already know how to export your rig and animations to Autodesk’s FBX format.
Edge Case 1: Preventing gimbal lock
When exporting an animation from Maya using the FBX format, you may notice Unity shows an animation that rotates in a different way than it did in Maya, commonly presenting as jumping or flipping.
This is not a result of importing into Unity, but rather a side effect of exporting to the FBX format. FBX adds a joint’s Joint Orient and Rotation values together instead of keeping them as separate values, like Maya does, resulting in gimbal lock once the file is imported to Unity.
Gimbal lock happens when a rotation is resolved by a jump or flip (from just under 180 degrees to just over -180 degrees). Wikipedia has an informative page about gimbal lock.
In Maya, you need to set up your rig in a way that separates the Joint Orient on the parent node of the node that has the isolated rotation animation. For example, in the image below, we put the Joint Orient of the final animation control rig on the rotation (X, Y, Z) of the nodes ending in ZER (standing for zero). This will zero out the rotation of the child node. Next, we put the rotation of the final animation control rig joint on the child ending in JNT. (Figure 01)
This will zero out the rotation by default and ideally prevent the values from going over 180 degrees or under -180 degrees. This way, the values in the FBX will be the same as in Maya and will appear correctly in Unity. It’s also imperative to assign the correct rotate order to the different joints.
Remember that similar joints in separate hierarchies, such as FK, IK, Ctrl, and Rig, should have the same rotate order.
Edge Case 2: Scaling issues
Upon import into Unity, you notice the squash and stretch on your character doesn’t look the same as it was in Maya, or your scaled assets look sheared and deformed.
FBX’s method of calculating scaling is different than Maya (or most DCC). It doesn’t apply a segment scale compensation according to its parent node, like Maya does by default. And, non-uniform scaling (different values for scaleX, scaleY, and/or scaleZ) cause shearing issues due to the axis not necessarily being in the same position and rotation.
There are a few ways to solve this issue; the least complicated is to isolate the scaling on a separate node, which here ends in ENV. (Figure 02)
There are some important things to note about the ENV nodes. They never have any children (apart from constraints) because their scaling would affect the scaling of their children. They never move or rotate. They only scale and they get their scaling value by constraints or connections to the final rig.
Edge Case 3: Stepped keyframing causes issues
The animation looks different in Unity than in Maya when you’ve stepped your keyframes.
Animations played back from FBX don’t function with frames. They’re resampled into curves that are played back at the players’ current refresh rate. The animation doesn’t play at a standardized rate like a movie, but rather at the maximum allowed by your graphics card and monitor. For example, if your graphics card and monitor can sustain it, the imported animation will play at 144fps. One second of animation will still take one second, but what was 24 frames of animation in Maya has now been resampled to 144 frames. This makes it difficult for Unity to manage the step interpolation in the animation curve.
In Maya, use a spline animation curve instead of a stepped animation curve and save a very quick keyframe from one frame prior to the next to mimic the step, and make an exponential shape in your curve. (Figure 03)
Edge Case 4: Bone weighting issues
The imported rig doesn’t appear to have the same weights applied to the mesh and, as a result, doesn’t deform in the same way as it had in Maya. The FBX may preview the weights as they appear in Maya, but this is not reflected in Unity pre-2019.1 versions.
Unity has a four-joints-per-vertex influence limit on meshes. If you bind your mesh without enforcing a max influence of four or less, you’ll have issues. It’s also important to keep in mind that Unity only understands the Classic Linear skinning method from Maya (and other DCC).
Limit your rig binding to the Classic Linear skinning method and set the Max Influences to 4. Checking the Maintain max influences box will ensure you don’t unintentionally go over the max influences while painting skin weights. (Figure 04)
Edge Case 5: Normals are affected by some deformers
The vertex normals seem to change orientation when animated and exported as Alembic into Unity. This may happen in Maya as well. This happens when you use some deformers, mainly lattices and wraps.
If you come from the VFX industry, you might not be used to paying attention to normals, as normals are rarely used in rendering. But for assets going into a real-time rendering engine, normals are important, as the engine will take them into account when rendering the final image onto the screen. Some normals are affected by more specialized deformers, such as lattices and wraps. Once a deformer is applied, the normals aim towards what appears to be their default, non-deformed position in Maya. Were you to export the mesh via FBX, you might not notice any visual differences; however, when exporting as Alembic, you’ll find that the normals don’t look correct on the mesh.
In Maya, avoid deformers that aren’t envelopes and blendshapes. It’s more limiting, but you’ll learn to get crafty about it (think of non-uniform scaling, for example).
As you can see from these edge cases, there are several things to keep in mind when rigging in Maya and exporting those rigs and their animations to Unity. This workflow doesn’t discuss all possible edge cases, but does delve into some of the more common occurrences. Hopefully this will give you an idea of what to look out for and what to ask if, in the future, you stumble on a new case.