[Bug Report] Interpenetration of rigid object and robot in Factory Nut and Bot environment

[kbkartik]

newbie to the sim2real world. This bug is similar to the question I raised earlier #1515. I am not sure if this qualifies as a bug or if I am making some mistake.

I import the newly added Factory Nut and Bolt environment and replace the Franka to Shadow Hand. Unlike #1515, I am not experimenting with contact sensor and adding physics material to objects. I simply check if physical interactions between robot and objects are valid.

Describe the bug

There are two observations/bugs I want to highlight.

1. When dropped from a height, nut penetrates through the hand and falls on the floor.
2. When it falls on the bolt, nut and bolt interpenetrate each other.

In both scenarios, collisions should take place between nut and hand, and between nut and bolt.

Steps to reproduce

1. Set the initial state of shadow_hand_config to the following:

init_state=ArticulationCfg.InitialStateCfg(
        pos=(0.0, 0.0, 0.5),
        rot=(0.0, 0.0, 0.0, 0.0),
        joint_pos={".*": 0.0},
    ),

2. Set the initial state of fixed_asset in NutThread class.

init_state=ArticulationCfg.InitialStateCfg(
            pos=(0.0, -0.35, 0.0), rot=(1.0, 0.0, 0.0, 0.0), joint_pos={}, joint_vel={}
        ),

3. Set the initial state of held_asset in NutThread class.

init_state=ArticulationCfg.InitialStateCfg(
            pos=(0.0, -0.35, 3.0), rot=(1.0, 0.0, 0.0, 0.0), joint_pos={}, joint_vel={}
        ),

4. Set disable_gravity to False in held_asset in NutThread class.

5. Add from omni.isaac.lab_assets import SHADOW_HAND_CFG in factory_env_cfg and set robot = SHADOW_HAND_CFG in FactoryEnvCfg class.

System Info

Describe the characteristic of your environment:

• Commit: 019e352
• Isaac Sim Version: 4.2.0-rc.17+release.15988.c99988b1.gl
• OS: Ubuntu 20.04
• GPU: RTX4500
• CUDA: 12.6
• GPU Driver: 560.35.03

Checklist

• I have checked that there is no similar issue in the repo (required)
• I have checked that the issue is not in running Isaac Sim itself and is related to the repo

Acceptance Criteria

Collision should happen when:

• When nut falls on the hand.
• When nut falls on the bolt.