Operating System:
Linux
Kit Version:
107 (Kit App Template)
Kit Template:
USD Composer
GPU Hardware:
50 series - 5070Ti
GPU Driver:
Latest
Hello,
I am attempting to run the Neural Reconstruction (Stereo Camera Flow) pipeline by strictly following the official documentation: https://docs.nvidia.com/nurec/robotics/neural_reconstruction_stereo.html
I am using the official sample dataset (r2b_galileo) provided by NVIDIA, not custom data. However, I am encountering significant issues with reconstruction quality and coordinate alignment in Isaac Sim.
My Environment:
-
Dataset: Official NVIDIA
r2b_galileo -
ISAAC ROS Version: 3.1.0 (Jazzy)
-
Pipeline: CuSFM → FoundationStereo → Nvblox & Gaussian Splatting
Issue 1: Severe Spikes and Noise (Despite Parameter Tuning) Even though I followed the provided flow exactly using the sample dataset, the resulting Mesh and Gaussian Splatting outputs are filled with severe “spikes” (Gaussian artifacts) and noise.
I have already tried the following to resolve this, but the issue persists:
-
Strict Adherence to Flow: Followed the documentation step-by-step.
-
Distance/Depth Parameter Tuning: I adjusted parameters related to distance filtering (e.g.,
projective_integrator_max_integration_distance_m, and depth estimation ranges) to cut off outliers, but it did not help.
Issue 2: Coordinate System Mismatch in Isaac Sim When I import the resulting Mesh (.ply from nvblox) and the Gaussian Splatting output (.ply) into Isaac Sim, they do not align with each other.
-
The Mesh and the Gaussian model seem to have different origins or orientations.
-
I suspect this is related to the
fit_to_z0option or world frame settings, but I need them to overlap perfectly.
Questions:
-
Why does the official
r2b_galileodataset produce such noisy results with the default configuration? Is there a specific config file for this dataset? -
How can I enforce the exact same coordinate system for both Nvblox and Gaussian Splatting so they align perfectly in Isaac Sim?
Any guidance would be appreciated.