Normals distorted/locked after merging multiple files

[ninjz]

So I have multiple GLB files which all contain the same character model and each contain a single animation. When merging the two files using the following code:

const { Document, WebIO, PropertyType } = require('@gltf-transform/core');
const { prune, dedup } = require('@gltf-transform/functions');
const fs = require('fs');

const io = new WebIO();

const inputFilenames = [
  'model-a.glb',
  'model-b.glb'
];

(async function() {
    const outputDocument = new Document()

    for (let filename of inputFilenames) {
        const asset = fs.readFileSync(`./input/${filename}`);
        const inputDocument = await io.readBinary(new Uint8Array(asset)); // Uint8Array → Document

        outputDocument.merge(inputDocument)
    }

    const root = outputDocument.getRoot();
    const mainScene = root.listScenes()[0];
    const mainBuffer = root.listBuffers()[0];
    const model = mainScene.listChildren()[0];

    // Store map of nodes to handle duplicates
    const targetNodeMap = {};

    // Retrieve all targetNodes from first model
    model.traverse(node => {
        if (!targetNodeMap[node.getName()]) {
            targetNodeMap[node.getName()] = node
        }
    })

    // Retarget all animations to initial model nodes
    root.listAnimations().forEach(anim => {
        anim.listChannels().forEach(channel => {
            channel.setTargetNode(targetNodeMap[channel.getTargetNode().getName()])
        })
    })

    // Remove duplicate meshes; dedup() doesn't seem to clean these up properly
    const meshMap = {}
    for (const mesh of root.listMeshes()) {
        const meshName = mesh.getName()
        if (!meshMap[meshName]) {
            meshMap[meshName] = mesh
            continue;
        }
        mesh.dispose();
    }

    // Remove duplicate nodes; dedup() doesn't seem to clean these up properly
    const nodeMap = {}
    for (const node of root.listNodes()) {
        const nodeName = node.getName()
        if (!nodeMap[nodeName]) {
            nodeMap[nodeName] = node 
            continue;
        }
        node.dispose();
    }

    // Merge into 1 scene
    for (const scene of root.listScenes()) {
        if (scene === mainScene) continue;
        scene.dispose();
    }
    root.setDefaultScene(mainScene)

    // Cleanup
    await outputDocument.transform(
        dedup(),
        prune(),
    );

    // Merge binary resources to a single buffer.
    root.listAccessors().forEach((a) => a.setBuffer(mainBuffer));
    root.listBuffers().forEach((b, index) => index > 0 ? b.dispose() : null);

    const glb = await io.writeBinary(outputDocument); // Document → Uint8Array

    fs.writeFileSync('./output/out.glb', Buffer.from(glb));
})()

I am able to successfully build a single GLB file which contains two animations and everything deduplicated which is great. However, it seems like the normals on some parts of the model for the second animation is distorted or that the normals are locked resulting in the model looking funny in certain parts. For example, here is an image of the animation before applying the script:

And then after applying the script:

You can see the distortion around the armpit area. Now my artist tells me that the mesh looks fine when he analyzed the two files, so my guess is that there is something wrong with the normals.

A theory I have may be that the two files I am merging both have different starting positions (for the character model), which results in the binary data for the mesh and all its primitives to differ for each file; so I wonder if I would need the two files to both start in the exact same initial position for this merging to work properly. The reason I think this, is because if I flip the order of which the merge is applied to the files, the issue is different; and that the issue is always seen in the second animation and not the first one.

[donmccurdy]

Two ideas for what to check here would be:

1. Are the inverse bind matrices of both skins fully identical?
2. Are the meshes that dedup skips fully identical? Particularly the WEIGHTS_0 and JOINTS_0 attributes.

In this case (1) is a different way of thinking about the 'different starting positions' theory that you mention; that does seem like a strong possibility.

[donmccurdy]

(1) How can I check if the inverse bind matrices of both skins are fully identical? ...
(2) I can see that the values of the WEIGHTS_0 and JOINTS_0 attributes ...

Each property has a .equals() method:

skin1.getInverseBindMatrices().equals(skin2.getInverseBindMatrices());

Same thing for the WEIGHTS_0 and JOINTS_0 attributes. The same indices wouldn't mean they're identical, but a deep check in the script would work.

If the issue is really related to 'different starting positions' then this is a huge problem for me ... this seems to be a limitation with the GLB format so far.

Technical questions on skinning and animation retargeting of arbitrary input models tend to be hard questions, but – the bind pose is the position of the character when the skin is attached, before any animation starts playing. The first frame of an animation could start there, but it certainly doesn't have to. So I think it's important that the inverse bind matrices and the skinning joints/weights match across your input models, but the animations themselves can start/end in any position.

[ninjz]

So i did a deep check with those commands and it looks like the inverse bind matrices are equal, as well as the WEIGHTS_0 and JOINTS_0 attributes.

Going through all the attributes on the mesh the inequality is with the NORMAL attribute. Given this, what you recommend as next steps?

[donmccurdy]

Seems like this question is getting pretty complex sorry — I have no idea why Blender would export different normals for the same mesh with the same skin and IBMs... you shouldn't need different normals for two different animations. It might be worth casting a wider net on this question, e.g. by:

1. asking on the glTF repository, glTF-Blender-IO repository, or Blender Stack Exchange
2. sharing the .blend and/or .glb files if possible

[ninjz]

Hey sry for the delay in response. I've ended up trying a bunch of things and ended up figuring it out.

So first of all, the way my artist was using blender was for importing a .fbx file and then using blender to export the .glb file. I got access to the .blend file he was using and I discovered that looking at the blender file, there was a bunch of blocky parts in the mesh:

I found a property in the mesh properties called "Auto Smooth" under "Normals" and upon disabling this option, everything smoothed out. I also saw in a suggestion here to "Clear Custom Split Normals" and did that as well.

Finally I did this to the two different animations and ran it through my script and all the normal properties ended up being equal! Also, previewing the final output file it turned out perfect. Two animations merged into one glb file, with everything deduplicated.

If you know anything about the two options that I used in blender finally to get it working then I'd love to learn more about why this works, but otherwise thank you for your help and making this great library!

[donmccurdy]

Oh that's great! I'm afraid that solution is beyond my understanding of Blender skinning and animation (maybe the Blender Forums would know?) but I'm glad you were able to get things working. 🎉