Context
I’m building a synthetic data generation pipeline combining Gaussian Splats with CAD models in Isaac Sim to reduce the sim-to-real gap. Since Gaussian Splats don’t natively receive shadows in the RTX renderer, I developed a shadow compositing approach.
My Pipeline
- RGB render — shadows disabled → base image
- White render — shadows enabled → shadow mask
- Multiply blend in OpenCV → final composite
Problem — Proxy Mesh & Custom Lights
I use a proxy mesh as the shadow caster for the Gaussian Splat.
- Works correctly with the default dome light
- Does not work with custom lights (Distant, Sphere, Rect), the proxy mesh renders black and leaks into the composite
Questions
- Is there a way to make the proxy mesh invisible to the camera but still cast shadows from custom lights?
- Is the black mesh issue a material problem with custom light types?
- Are there better approaches for a clean shadow-only pass?
Any suggestions are appreciated. Thanks!
| Component | Specification |
|---|---|
| CPU | Intel Core i9-11900K |
| RAM | 64 GB |
| GPU | NVIDIA RTX A5000 Mobile (16 GB VRAM) |
| Isaac Sim Version | 5.1.0 |
| OS | Windows |