Recently, for one of my projects, I had to learn about skeletal animation and GPU skinning. As always, I first went to google which gave me a wealth of information. As always, I try to get the real crux of things rather than reading a whole document only to find out in the end that the details are not there. For my case, I could not find a simple explanation of how the skeletal animation works including the whole series of transforms involved. This blog entry will try to answer this.
The transformation matrices
I will start with the different transforms that are used in this process. Typically in a simulation or game, a transform is represented as a 4x4 matrix. For skeletal animation, we have a collection of bones. Each bone has a local transform (also called relative transform) which tells how the bone is positioned and oriented with respect to its parent bone. If the bone's local transform is multiplied to its parent's global transform, we get the global transform (also called absolute transform) of the bone. Typically, the animation formats store the local transforms of the bones in the file. The user application uses this information to generate the global transforms. I define my bone structure as follows,
struct Bone {
glm::mat4 relativeXForm, absoluteXForm;
int parent; //index of parent
};
As you can see, I only include the fields that I need. You can populate whatever you want in this structure. Final word about the parent variable. This stores the index of the parent bone of the current bone. For the root bone, the parent is -1. For all of the other bones, it will be a number starting from 0 to N-1 where N is the total number of bones in the skeleton.
OK lets say I have a collection of bones stored in a vector called skeleton. We can get the absolute transforms for all of the bones in the skeleton using the following code.
for(size_t i=0;i < skeleton.size(); i++) {
Bone& b = skeleton[i];
if(b.parent==-1)
b.absoluteXForm = b.relativeXForm;
else
b.absoluteXForm = skeleton[b.parent].absoluteXForm * b.relativeXForm;
}
As you can see, it is very simple ans straight forward. Now there is another big word that I see scattered all over the place in the skeletal animation tutorials and codes available online. This word is bindpose. Simply put bind pose is the absolute transforms of the bones in the non animated state (i.e. when no animation is applied. This is usually when the skeleton is attached (skinned) to a geometry. We can also say, it is the default pose of the skeletal animated mesh. Typically, bones can be in any bindpose (usually for humanoid characters, the character may be in A pose, T pose etc. based on the convention used). Typically, the inverse bindpose is stored at the time of initialization. So continuing to the previous skeleton example, we can get the bindpose and inverse bindpose matrices as follows,
for(size_t i=0;i < skeleton.size(); i++) {
bindPose[i] = (skeleton[i].absoluteXForm);
invBindPose[i] = glm::inverse(bindPose[i]);
}
When we apply any new transformation (an animation sequence for example) to the skeleton, we have to first undo the bind pose transformation so that the bones can be first moved to their origin (so that the bones transform is identity and it transform is its parent's transform). This is similar to how we apply composite transformation in OpenGL and DirectX i.e. we first move the object to world origin by untranslating it and then do the transformation at origin. At the time of calculation of the bindpose matrices, we store the inverse of the bindpose matrix. Any new animated transformation on the bone has to be multiplied by the inverse of the bindpose matrix.The final matrix that we get from this process is called the skinning matrix (also called the final bone matrix). Continuing to the example given in the previous paragraph, lets say we have modified the bone's relative transforms using the animation sequence. We can then generate the skinning matrix as follows,
for(size_t i=0;i < skeleton.size(); i++) {
Bone& b = skeleton[i];
if(b.parent==-1)
b.absoluteXForm = b.relativeXForm;
else
b.absoluteXForm = skeleton[b.parent].absoluteXForm * b.relativeXForm;
skinnedXForm[i] = b.absoluteXForm*invBindPose[i];
}
So as you can see this is simple to do.
Skinning on the GPU
I think there are plenty of places where GPU skinning has been explained elsewhere so I wont touch up on it. I will detail my skinning vertex shader which is as follows,
attribute vec4 blendIndices;
attribute vec4 blendWeights;
varying vec3 out_normal;
varying vec3 es_vpos;
uniform mat4 Bones[62];
void main() {
vec4 newVertex;
vec3 newNormal;
int index = int(blendIndices.x);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.x;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.x;
index= int(blendIndices.y);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.y + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.y + newNormal;
index= int(blendIndices.z);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.z + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.z + newNormal;
index= int(blendIndices.w);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.w + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.w + newNormal;
out_normal = normalize(gl_NormalMatrix * newNormal);
es_vpos = (gl_ModelViewMatrix * vec4(newVertex.xyz,1)).xyz;
gl_Position = gl_ProjectionMatrix * vec4(es_vpos.xyz,1);
gl_TexCoord[0] = gl_MultiTexCoord0;
}
As you can see, I use a previous gen shader version since my focus is on detailing the crux. Basically, the skeletal animated mesh will store the skinning information (blend weights and blend indices). You need to pass these to the relevant attributes and finally, you need to pass the skinning matrices. I pass my skinning matrices in one shot as follows,
shader.Use();
glUniformMatrix4fv(shader("Bones"), skinnedXForm.size(), GL_FALSE, glm::value_ptr(skinnedXForm[0]));
shader.UnUse();
There are two things that you have to be careful about.
1) Make sure that the size of the bones array is correct. Often times, you will find that part of the mesh is skinned fine while the other parts are not skinned correctly. If so make sure the size of the bones array is correct.
2) Make sure that the proper vertex attrib array is enabled and the correct parameter types are passed. For my skinning vertex shader, I use the following code to pass this data to the shader.
glEnableVertexAttribArray(shader["blendWeights"]);
glBindAttribLocation(shader.GetProgram(), shader["blendWeights"], "blendWeights");
glVertexAttribPointer(shader["blendWeights"], 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), &vertices[0].blendWeights); glEnableVertexAttribArray(shader["blendIndices"]);
glBindAttribLocation(shader.GetProgram(), shader["blendIndices"], "blendIndices");
glVertexAttribPointer(shader["blendIndices"], 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), &vertices[0].blendIndices);
If you have done all of these steps fine, you should have a nice skinned skeletal mesh on screen. I hope that I have clarified the skeletal animation steps so that it is easier to grasp and understand. I don't know why this information is not detailed in nice simple manner anywhere.
Based on a couple of requests, here is the source code for an EZMesh animation viewer.
https://www.dropbox.com/s/u11f3ldf2nxeayi/EzMeshAnimationViewerSkinned.zip?dl=0
This is what the output looks like
You can get details about EZMesh format from here:
http://codesuppository.blogspot.sg/2009/11/test-application-for-meshimport-library.html
and the EZMesh project is hosted here: http://code.google.com/p/meshimport/
Controls for the program:
Press ',' to go to previous animation frame
Press '.' to go to next animation frame
Press 'l' to enable looping otherwise the animation stops at the last/first frame.
Enjoy !!!
Thanks,
Mobeen
The transformation matrices
I will start with the different transforms that are used in this process. Typically in a simulation or game, a transform is represented as a 4x4 matrix. For skeletal animation, we have a collection of bones. Each bone has a local transform (also called relative transform) which tells how the bone is positioned and oriented with respect to its parent bone. If the bone's local transform is multiplied to its parent's global transform, we get the global transform (also called absolute transform) of the bone. Typically, the animation formats store the local transforms of the bones in the file. The user application uses this information to generate the global transforms. I define my bone structure as follows,
struct Bone {
glm::mat4 relativeXForm, absoluteXForm;
int parent; //index of parent
};
As you can see, I only include the fields that I need. You can populate whatever you want in this structure. Final word about the parent variable. This stores the index of the parent bone of the current bone. For the root bone, the parent is -1. For all of the other bones, it will be a number starting from 0 to N-1 where N is the total number of bones in the skeleton.
OK lets say I have a collection of bones stored in a vector called skeleton. We can get the absolute transforms for all of the bones in the skeleton using the following code.
for(size_t i=0;i < skeleton.size(); i++) {
Bone& b = skeleton[i];
if(b.parent==-1)
b.absoluteXForm = b.relativeXForm;
else
b.absoluteXForm = skeleton[b.parent].absoluteXForm * b.relativeXForm;
}
As you can see, it is very simple ans straight forward. Now there is another big word that I see scattered all over the place in the skeletal animation tutorials and codes available online. This word is bindpose. Simply put bind pose is the absolute transforms of the bones in the non animated state (i.e. when no animation is applied. This is usually when the skeleton is attached (skinned) to a geometry. We can also say, it is the default pose of the skeletal animated mesh. Typically, bones can be in any bindpose (usually for humanoid characters, the character may be in A pose, T pose etc. based on the convention used). Typically, the inverse bindpose is stored at the time of initialization. So continuing to the previous skeleton example, we can get the bindpose and inverse bindpose matrices as follows,
for(size_t i=0;i < skeleton.size(); i++) {
bindPose[i] = (skeleton[i].absoluteXForm);
invBindPose[i] = glm::inverse(bindPose[i]);
}
When we apply any new transformation (an animation sequence for example) to the skeleton, we have to first undo the bind pose transformation so that the bones can be first moved to their origin (so that the bones transform is identity and it transform is its parent's transform). This is similar to how we apply composite transformation in OpenGL and DirectX i.e. we first move the object to world origin by untranslating it and then do the transformation at origin. At the time of calculation of the bindpose matrices, we store the inverse of the bindpose matrix. Any new animated transformation on the bone has to be multiplied by the inverse of the bindpose matrix.The final matrix that we get from this process is called the skinning matrix (also called the final bone matrix). Continuing to the example given in the previous paragraph, lets say we have modified the bone's relative transforms using the animation sequence. We can then generate the skinning matrix as follows,
for(size_t i=0;i < skeleton.size(); i++) {
Bone& b = skeleton[i];
if(b.parent==-1)
b.absoluteXForm = b.relativeXForm;
else
b.absoluteXForm = skeleton[b.parent].absoluteXForm * b.relativeXForm;
skinnedXForm[i] = b.absoluteXForm*invBindPose[i];
}
So as you can see this is simple to do.
Skinning on the GPU
I think there are plenty of places where GPU skinning has been explained elsewhere so I wont touch up on it. I will detail my skinning vertex shader which is as follows,
attribute vec4 blendIndices;
attribute vec4 blendWeights;
varying vec3 out_normal;
varying vec3 es_vpos;
uniform mat4 Bones[62];
void main() {
vec4 newVertex;
vec3 newNormal;
int index = int(blendIndices.x);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.x;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.x;
index= int(blendIndices.y);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.y + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.y + newNormal;
index= int(blendIndices.z);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.z + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.z + newNormal;
index= int(blendIndices.w);
newVertex = (Bones[index] * gl_Vertex) * blendWeights.w + newVertex;
newNormal = (Bones[index] * vec4(gl_Normal, 0.0)) * blendWeights.w + newNormal;
out_normal = normalize(gl_NormalMatrix * newNormal);
es_vpos = (gl_ModelViewMatrix * vec4(newVertex.xyz,1)).xyz;
gl_Position = gl_ProjectionMatrix * vec4(es_vpos.xyz,1);
gl_TexCoord[0] = gl_MultiTexCoord0;
}
As you can see, I use a previous gen shader version since my focus is on detailing the crux. Basically, the skeletal animated mesh will store the skinning information (blend weights and blend indices). You need to pass these to the relevant attributes and finally, you need to pass the skinning matrices. I pass my skinning matrices in one shot as follows,
shader.Use();
glUniformMatrix4fv(shader("Bones"), skinnedXForm.size(), GL_FALSE, glm::value_ptr(skinnedXForm[0]));
shader.UnUse();
There are two things that you have to be careful about.
1) Make sure that the size of the bones array is correct. Often times, you will find that part of the mesh is skinned fine while the other parts are not skinned correctly. If so make sure the size of the bones array is correct.
2) Make sure that the proper vertex attrib array is enabled and the correct parameter types are passed. For my skinning vertex shader, I use the following code to pass this data to the shader.
glEnableVertexAttribArray(shader["blendWeights"]);
glBindAttribLocation(shader.GetProgram(), shader["blendWeights"], "blendWeights");
glVertexAttribPointer(shader["blendWeights"], 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), &vertices[0].blendWeights); glEnableVertexAttribArray(shader["blendIndices"]);
glBindAttribLocation(shader.GetProgram(), shader["blendIndices"], "blendIndices");
glVertexAttribPointer(shader["blendIndices"], 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), &vertices[0].blendIndices);
If you have done all of these steps fine, you should have a nice skinned skeletal mesh on screen. I hope that I have clarified the skeletal animation steps so that it is easier to grasp and understand. I don't know why this information is not detailed in nice simple manner anywhere.
Based on a couple of requests, here is the source code for an EZMesh animation viewer.
https://www.dropbox.com/s/u11f3ldf2nxeayi/EzMeshAnimationViewerSkinned.zip?dl=0
This is what the output looks like
You can get details about EZMesh format from here:
http://codesuppository.blogspot.sg/2009/11/test-application-for-meshimport-library.html
and the EZMesh project is hosted here: http://code.google.com/p/meshimport/
Controls for the program:
Press ',' to go to previous animation frame
Press '.' to go to next animation frame
Press 'l' to enable looping otherwise the animation stops at the last/first frame.
Enjoy !!!
Thanks,
Mobeen
