Arcane University:Implementation of Custom Animations

This article describes how to implement fully custom animations into Skyrim with no external runtime dependencies. All credit to Jonahex for figuring out the workflow and drafting the article. (subsequently edited by Thingy Person, caveat emptor)

Tools

 * 1. ck-cmd. An up to date build (May 2022 or later) is required for custom creature implementation. Can be downloaded from github. Either download the latest build from appveyor artifacts or build it from source. ck-cmd is primarily a command line tool. The following commands are useful for animation implementation:
 * 1) convert. Used for conversion between binary and xml representation of Havok (.hkx) files.
 * convert -o -v WIN32 -f SAVE_DEFAULT — to convert xml to Skyrim LE hkx.
 * convert -o -v AMD64 -f SAVE_DEFAULT — to convert xml to Skyrim SE hkx.
 * convert -o -f SAVE_TEXT_FORMAT — to convert Skyrim LE hkx to xml.
 * Fun fact #1: SE hkx can’t be converted back to xml. So, save your xmls!
 * 2) importrig. Used to import a skeleton from fbx to nif and hkx.
 * importrig -a -e .


 * Fun fact #2: the skeleton hkx will be saved to the folder where ck-cmd.exe is located, no matter which output paths you set.
 * Fun fact #3: you usually don’t need animation output, but you need to set the animation output folder anyway, as the command doesn’t work without it.
 * 3) exportfbx. Used to export nifs back to fbx.
 * exportfbx -e .
 * 4) importskin. Used to import creature or armor skin from fbx to nif.
 * importskin -e .
 * 5) retargetcreature. This command copies a creature Havok project, replacing the creature name in all instances where it’s needed and creating an esp with idles, sound descriptors and movement types for the new creature.
 * retargetcreature   -s .


 * Fun fact #4: there is no output folder argument. The esp will be placed in your Skyrim LE Data folder. All other output is stored in the ck-cmd folder. It includes animationdata and animationsetdata folders, animationdatasinglefile.txt, animationsetdatasinglefile.txt and a folder with the new Havok project.
 * 6) importanimation. Used to export fbx animations to hkx.
 * importanimation   --b= --c=project.txt> --e= .
 * Fun fact #5: but the animationdata and animationsetdata folders, animationdatasinglefile.txt, and animationsetdatasinglefile.txt will all be stored in the ck-cmd folder!
 * Fun fact #6: though they won’t be generated if the folder that is passed to the --b argument contains a project.txt with upper case letters in the file name.
 * And the final ck-cmd fun fact: if any folder path in a command ends in a (back)slash, ck-cmd will crash.
 * 2. Skyrim Behavior Tool. GUI tool for behavior editing. Download it from github.

Preparation
To do anything with animations, you need Skyrim LE Havok projects and animation cache. So, extract Skyrim – Animations.bsa. It contains all Havok projects in the game, as well as animationdata and animationsetdata folders and animationdatasinglefile.txt and animationsetdatasinglefile.txt files. For manual editing of behaviors, you can further convert Havok (hkx) files to xml with ck-cmd.

Custom animated creature implementation
Let’s say you have a custom creature modeled and animated, and its animations follow those of an existing creatures which we will call donor. The donor's behavior graph will be used as a baseline for the new creature. The custom creature may have animations that differ from those of the donor creature (both in number and characteristics) and also have a different skeleton (both in number and placement of bones), but the more differences there are, the more the behavior has to be edited in later steps.

Export the creature to fbx format, ideally where Y is forward and Z is up, and with the root bone positioned at the origin; also disable "add leaf bones" (if using blender) and export animations one-by-one. This results in the following files:
 * 1. Skinned mesh in fbx.
 * 2. Skeleton with collision in fbx.
 * 3. Animations in separate fbx files.

This is what you need to do to implement it:
 * 1. Use ck-cmd importskin command to import skin fbx to nif. Rename root bone in imported nif to NPC Root [Root].
 * 2. Use ck-cmd importrig command to import skeleton fbx to nif and hkx.
 * 3. Use ck-cmd retargetcreature command to copy donor Havok project. This will result in a havok project with the name Project.hkx and 4 folders:
 * animations, with animation hkx files
 * character, with character.hkx, where various misc info about the creature is stored
 * character assets, with skin and skeleton
 * behaviors, with behavior hkx files
 * Note that editing something in a hkx file means first converting it to xml, editing xml and then converting back to hkx.


 * 4. Adjust the bone pair map, which is needed for mirrored animations to work. It is located inside Characters\ Character.hkx in the hkobject hkbMirroredSkeletonInfo under the parameter bonePairMap. It is a list of bone indices (starting from 0) and the order in which each one is listed represents the bone it is paired to (also starting from 0). If the bone index is the same number as its rank order, the bone is "paired to itself" and thus not mirrored. For example, if your skeleton includes the following bones: root, pelvis, leg_left, leg_right, your bone pair map is 0 1 3 2.
 * In order to figure out which bone index refers to which part of the skeleton, you can find the list of bones in the skeleton.hkx file and count their rank order numbers yourself, or find the same list of bones automatically enumerated in Skyrim Behavior Tool (tab Character Data, panel NPC Root [Root]); take care because this list is numbered starting from 1 rather than from 0, so subtract 1 from each number before putting it in the bone pair map.
 * 5. Use ck-cmd importanimation command to import fbx animations to hkx. Make sure to set the --b argument to the donor behavior folder, and the --c argument to the retargeted animation cache file!
 * 6. Now collect all the files and put them in the relevant locations inside the Skyrim SE Data folder:
 * animationdata and animationsetdata folders and animationdatasinglefile.txt, animationsetdatasinglefile.txt files from importanimation command to Meshes folder.
 * Havok project from retargetcreature command to where you store your creatures, probably Meshes\ \Actors\.
 * Skin, skeleton.nif and skeleton.hkx to \Character Assets\.
 * 7. Now take the esp file generated by the retargetcreature command and load it in the CK. Create race, armor addon, armor and actor records for your creature. In the armor addon, set paths to the skin. In the race, set paths to the skeleton and the Havok project, and the generated movement types. Check the generated IDLE records – the paths there may need to be adjusted to where you actually stored your Havok project.

If everything was done right, your creature animations should now work in game! But there are a lot of things to tweak in behaviors…

Root motion
Your animations may include root transform, which describes the movement of the creature’s center of mass. Skyrim's engine handles it in a very special way. Hkx animations actually can contain only Z translations and X and Y rotations of the root. X and Y components of translations and Z and W components of rotation are stripped from animations by ck-cmd's importanimation command and are stored in a text cache in the animationdata\boundanims folder (and in animationsdatasinglefile.txt). Translation and rotation of the creature are instead determined by the engine. Movement speed is defined by the creature’s movement type. Most creatures have several animations for movement, which are blended depending on movement speed. In order to make a creature's movement animation synchronized with its motion, one needs to do two things:
 * 1. Set walking and running speed in movement types to actual speed values for which respective animations are made.


 * 2. Open your creature’s behavior and find the hkbBlenderGenerator of a given movement animation. Each of its children (a hkbBlenderGeneratorChild) represents an animation that can be blended, and has a parameter called Weight, which is the speed corresponding to the respective animation. When the current speed of the creature is between the weights of two such animations, the resulting animation is blended from those two animations proportionally. So, you need to set those weights to values for which the respective animations were made.

Creating creature collision
For creature physics to work, you need provide two things in the skeleton: a bounding box, and collision. You can either make them in NifSkope (a very long and painful process) or export from your 3d editor to fbx together with the skeleton. Let’s follow the second route using Blender. In this case, the .blend file you are exporting from needs to have three things:


 * 1. Armature representing your creature’s skeleton.
 * 2. Empty called BoundingBox with child box mesh called BoundingBox_Box.
 * 3. Hierarchy of empties replicating your skeleton. You don’t need to construct it manually: you can obtain it by exporting the skeleton nif with ck-cmd's exportfbx command.

Each empty corresponding to a bone can have mesh children representing bone colliders and empty children representing ragdoll constraints.

Colliders must be called either  _rb_capsule for a capsule shape,  _rb_sphere for a sphere shape,  _rb_box for a box shape or  _rb_convex for a convex shape. Note that collider pivot transform is ignored by ck-cmd, so transforms must be baked into the geometry. Mass and components of inertia tensor for colliders are calculated by ck-cmd, but can be too far off. In that case, you can set them as float custom properties named "mass", "inertiaX", "inertiaY", and "inertiaZ" on the collider mesh.

Constraints determine which motions are allowed in a ragdoll. A constraint is defined by an empty, which must be child of the constraint’s parent bone and must be called  _rb_con_ _rb_attach_point. The constraint's orientation is defined by the orientation of the respective empty. There are two kinds of constraints: full-blown ragdoll constraints and simpler hinge constraints. Constraint type is defined by the string custom property "constraint_type". Set Ragdoll for a ragdoll constraint and Hinge otherwise. For ragdoll constraints, you need to provide the following parameters in string custom properties: "coneMaxAngle", "maxFriction", "planeMaxAngle", "planeMinAngle", "twistMaxAngle", and "twistMinAngle". For ragdoll constraints, you need provide the following parameters in string custom properties: "maxFriction", "maxAngle", and "minAngle". Check vanilla skeletons for good values of those parameters.

Fun fact again: since there are in essence two instances of the skeleton (one in the armature and another in the hierarchy of empties), ck-cmd makes two skeletons in the nif on export. So you will need to delete one.

Custom animated object idles
Skyrim includes a lot of idle animations in which some visible object is used, such as musical instruments, a pickaxe, buckets or firewood being carried, etc. Such objects are described by AnimObject records. But it appears that replacing those objects is impossible without behavior edits: when an animation event is fired and the behavior enters the respective state, it sends an AnimObjLoad event which contains a payload string with the EditorID of the AnimObject to be used. So, in order to create a new idle using an existing animation but with a different object, you need to do the following:




 * 1. Open the defaultmale Havok project and navigate to the mt_behavior inside it. Find the state corresponding to the usage of the object of interest and copy it. Skyrim Behavior Tool doesn’t support copying, so you need to recreate the whole state hierarchy and add it to the same parent.
 * 2. Replace the payload string of the AnimObjLoad event of your newly created state with your object’s EditorID.
 * 3. Create new animation events for your object.
 * 4. Find transitions to the original object’s state and copy them for your new state and animation events.
 * 5. Create IDLE records for your new animation events as well as relevant idle markers.