Issues Controlling Articulation Joints via Python — GPU Physics Crash & Joint Velocity Limit

Omniverse Core Platform Kit App Template / SDK
, ,
2

Hi and thanks for posting. What are your exact GPU and drivers? You said its a 30 series. Have you tried changing drivers? I have never seen anything regarding “Illegal CUDU memory access”. CUDA is part of the core driver, so if the driver is not a good match, CUDA could be affected. It may have nothing to do with Kit.

To solve this, you are going to have to break down the workflow and isolate the issue.

  1. Set up the objects and joints in the scene
  2. Without code attached, just hit spacebar to play the scene… does it crash or create messages?
  3. Try a much more simple joint, that is not embedded or restrained as much. Maybe the joint is set up in a way that is not physically allowed.
  4. Try using code to initiate the scene and to set initial velocities, but then let it “run” and do not try to write additional states. Read yes, but not write. You maybe be trying to write velocities against the actual physics engine that are “illegal”.

Here is much detailed breakdown of the situation from our AI. Please read through this and see if any of this helps. It probably comes down to a workflow problem where you are trying to force the GPU PhysX to do something it cannot do. So, as stated, you need to reverse engineer the problem, with a simpler and simpler scene until you find what is breaking it.

The GPU crash is very likely a PhysX GPU‑physics bug or instability triggered by directly mutating articulated joint state every frame, while the velocity “cap” you see on CPU comes from PhysX’s joint‑velocity limiting (maxJointVelocity / maxActuatorVelocity), not from the JointStateAPI itself. docs.omniverse.nvidia

1. About the GPU crash (cudaErrorIllegalAddress)

  • cudaErrorIllegalAddress means some CUDA kernel accessed invalid memory; in Omniverse this usually comes from low‑level PhysX or rendering code, and then Hydra/RTX fail to allocate textures or complete the compute graph. forums.developer.nvidia
  • The OmniPhysics docs say JointStateAPI is intended to set initial position and velocity of articulation joints; it does not explicitly guarantee that you can arbitrarily drive joints by rewriting those attributes every simulation step on GPU. docs.omniverse.nvidia

In practice, there are a few patterns that are known to cause GPU physics instability or crashes:

  • Highly stiff/high‑velocity articulated chains, especially with prismatic joints and complex nested geometry, can produce extreme impulses that destabilize GPU PhysX even when CPU solves them. docs.omniverse.nvidia
  • Mutating articulation joint state in ways that disagree with the solver (for example, large instantaneous jumps in joint velocity or position each frame) can create inconsistent internal state on GPU; CPU is more tolerant but can still clamp values. docs.omniverse.nvidia

Because CPU physics runs without crashing on the same scene, the most likely interpretation is:

  • Your articulation definition is formally valid, but the combination of:
    • multiple prismatic joints,
    • nested slider/rail geometry, and
    • direct per‑frame JointStateAPI writes on GPU
      is hitting an edge‑case / bug in the PhysX GPU backend, causing illegal memory access. forums.developer.nvidia

This is consistent with other reports where custom articulated environments or tensor‑based control cause CUDA illegal‑address errors from omni.physx plugins while CPU runs fine. github

What you can try to narrow it down

  • Run the same scripted control with GPU physics disabled but keep RTX/Hydra on; if the crash disappears, it confirms that the problem is in GPU PhysX, not rendering. forums.developer.nvidia
  • Reduce the structure to a minimal reproduction: one or two prismatic joints, simple boxes instead of nested cylinders, and the same JointStateAPI writes; if that works, re‑introduce complexity stepwise to find the triggering pattern. docs.omniverse.nvidia

Given how specific your setup is, filing a bug with your USD and logs, plus a minimal USD+script that reproduces the GPU crash, is the only way NVIDIA can fix the illegal‑memory‑access issue cleanly. forums.developer.nvidia

2. Why velocities are “capped” on CPU

OmniPhysics/PhysX introduces two separate velocity limits for articulations:

  • PhysxJointAPI.maxJointVelocity: “Maximum joint velocity. Only applies to joints that are part of an articulation. The solver will apply appropriate joint‑space impulses in order to enforce the per‑axis joint‑velocity limit.” docs.omniverse.nvidia
  • PhysxArticulationDriveAPI.maxActuatorVelocity: “maximum achievable velocity for actuated joints,” with speedEffortGradient defining how this limit decreases as joint force/torque increases. docs.omniverse.nvidia

The articulation stability guide explicitly recommends using max joint velocity limits (for example 200–300 deg/s for revolute joints) to avoid instability, and notes that excessive velocities can destabilize simulations. docs.omniverse.nvidia

Putting this together:

  • Even if you write high values into the JointStateAPI velocity attribute, the solver still enforces maxJointVelocity on articulated joints and clamps them to the configured limit. docs.omniverse.nvidia
  • If maxJointVelocity and/or maxActuatorVelocity are at their defaults (which are conservative for stability), you will see an apparent cap (for example around 100 units/s) regardless of what you write directly to the joint state. docs.omniverse.nvidia

That is why:

  • Using a drive with high target velocity and sufficient maxForce can go faster: here you are working within the actuator model, and if maxActuatorVelocity is large enough the limiter is higher. docs.omniverse.nvidia
  • Applying a large force works because velocity then emerges from dynamics, again subject to those limits, but possibly reaching higher sustained speeds than your earlier cap. docs.omniverse.nvidia

To verify and adjust this in your scene:

  1. Inspect each articulation joint’s PhysxJointAPI:maxJointVelocity attribute; if unset, explicitly set it to a larger value in units consistent with your joint (distance/s for prismatic, degrees/s for revolute). docs.omniverse.nvidia
  2. For driven joints, check PhysxArticulationDriveAPI:maxActuatorVelocity, maxForce, speedEffortGradient, and velocityDependentResistance; low maxActuatorVelocity or a high gradient can effectively clamp speed. docs.omniverse.nvidia
  3. Re‑run the simulation and read back the actual JointStateAPI velocity; you should see higher values once limits are increased, though you may also need to reduce step size or adjust solver settings to keep things stable. docs.omniverse.nvidia

3. Recommended control pattern

Based on the OmniPhysics docs and stability guide, a more “supported” approach than rewriting joint state every frame on GPU is:

  • Use JointStateAPI to set initial positions/velocities and to monitor state. docs.omniverse.nvidia
  • Use drives (position or velocity) with carefully chosen maxActuatorVelocity, maxForce, damping, and stiffness to achieve your desired motion profile, and use Python to update targets rather than raw state. docs.omniverse.nvidia

This aligns better with how the articulation solver is designed, respects internal limits, and is less likely to trigger GPU backend bugs.

Would you be able to share (even conceptually) how often you are writing joint positions/velocities (every frame, every few frames, or event‑based)? That cadence is important for judging how risky your current control pattern is and what a safer alternative would look like.