Importing an MMD model into XNA

This has been a bit of a mystery for me for a while.

Everyone knows MikuMikuDance, right? That program that people make Vocaloid and Idolm@ster dance routines with? You can convert all the PMD models that are made for it to other formats. XNA can import FBX models without needing anything special, so I converted one of those models to FBX and tried to import it into XNA. If you keep up on Miku Miku Dance stuff, it's the 18 year old Haku model by nakao.

Well, there it is, though without any lighting effects. Aside from a texturing quirk (look at her left arm), it's pretty painless from a basic ContentManager.Load call on it. However, the Effect that XNA slaps on all the meshes is a BasicEffect. That's the generic, easy-to-use shader effect that's mostly there to get you started with having something on the screen. Anything more advanced, and you'll have to code your own shaders.

... Except that even if you transfer all the effect parameters right, you might get this.

Ugh. What gives?

I checked all the parameters, and I narrowed down the important parameters down to a few: Texture, DiffuseColor, World, and WorldViewProj. The last two are just transforms, so I wasn't too worried about them. The first two though, had to be copied to my custom shader - PMD models use texture... sometimes, but more often they have the habit of storing color on the vertex instead. However, even getting Texture and DiffuseColor copied to my custom effect, I couldn't get any color on the model. I just kept getting that black silhouette.

Microsoft eventually released the source code of BasicEffect as part of their StockEffects samples, and it includes both the C# code and the associated HLSL code. That was pretty nice of them, and since I got a nice, colored model as long as I stuck with BasicEffect, I had a look to see what it was doing...

... And I found this.

EffectDirtyFlags dirtyFlags = EffectDirtyFlags.All;

Uh, wait. Why do we have dirty flags here? Why are they all set when BasicEffect is initialized? Well, there's this function, OnApply, that looks to be the only function in here that does anything special. And it looks at these dirty flags to...

// Recompute the diffuse/emissive/alpha material color parameters?
if ((dirtyFlags & EffectDirtyFlags.MaterialColor) != 0)
  ...

AAAAAAAAAAAGGGGGGHHHHHH!!!!!!!!!

So, when BasicEffect is first initialized, it sets all of its dirty flags to 1. When it's drawn for the first time, it constructs all its HLSL parameters that I guess it knew from the initial ContentManager.Load call, then blows that dirty flag. Only then can you get the parameters out with Effect.Parameters[].GetValue.

...

So, when importing a PMD model from MikuMikuDance to XNA, you have to tell the model to draw itself once before you start copying its parameters to a custom effect. There's no way to get at these effects directly to invoke OnApply (that I know of), the usual batch of ModelMesh.Draw calls will have to do instead.

m_Model = Content.Load(_modelname);


foreach (ModelMesh Mesh in m_Model.Meshes) {
  Mesh.Draw();
}

If you want to take a look at BasicEffect, there's one more parameter that I haven't mentioned yet: ShaderIndex. See, BasicEffect isn't as basic as one might first suspect, because the designers had to think about all the different situations where it would be used. It's supposed to be a general go-to effect for everything, but some developers might use per-vertex coloring, and others might use only textures. BasicEffect had to handle all of these.

So as it turns out, it comes with 20 different vertex shaders and 10 different pixel shaders for 32 situations, and uses the ShaderIndex parameter (an integer) to figure out which vertex and pixel shader to use. This means cracking into the debugger itself and seeing what this parameter has been set to for each instance of BasicEffect. The model I was using came with 32, and two of them were different from the rest. They made use of texturing, whereas the others didn't.

... And then I finally coded my custom first pass shader, and realized it hardly did anything that BasicEffect didn't do, because per-pixel coloring meant that the color palette was only 32 colors, and all the other special things I was doing including lighting was in postprocessing meaning toon shading was already a bust in preprocessing from the get-go.

The End.