Question about the NCore data format in NuRec – structure, conversion, and relation to USD

Omniverse Core Platform
1

Hi everyone,

I’ve been reading through the NuRec documentation and came across this section:

“NuRec requires a specific format, called the NCore data format, to ingest real-world data. Additionally, a set of auxiliary data generated from the NCore data sets serves as pre-processed input for NuRec. The NCore container includes data quality validation tools to ensure the data is ready for NuRec.”
— NVIDIA NuRec Documentation

I’m trying to understand what the NCore data format actually is in practice. The docs mention it briefly, but I couldn’t find any detailed specification, schema, or example of what an NCore dataset looks like — for instance:

  • its folder and file structure

  • metadata format

  • supported sensor data types (camera, LiDAR, IMU, etc.)

  • and how to convert existing datasets into NCore

Also, I’m curious about how NCore relates to NVIDIA’s USD ecosystem. Is NCore just an internal pre-USD representation, or is it part of the USD framework itself?

Any guidance, documentation, or example datasets would be greatly appreciated.

Thanks a lot!

3

Hi and thanks for the great question.

The NCore data format in NVIDIA Omniverse NuRec is a structured, containerized way to organize multi-modal real-world sensor data for ingestion into NuRec’s neural reconstruction models. However, NVIDIA does not currently provide a fully public, standardized schema or example dataset for NCore; documentation describes its function and purpose, but leaves technical details sparse.nvidia+2​

What is the NCore Data Format?

  • Structured Container: The NCore format organizes input such as camera images, LiDAR, and IMU logs (and corresponding metadata) into a validated directory structure, often packed as an NCore container.developer.nvidia+1​
  • Auxiliary Data: Alongside raw sensor data, NCore includes auxiliary pre-processed files essential for NuRec, including synched timestamps, calibration data, object tracks, and harmony scores for validation.nvidia​
  • Validation Tools: NCore provides scripts or integrated tools to check and validate that sensor data and metadata meet NuRec’s requirements (temporal sync, file formats, metadata completeness).developer.nvidia+1​
  • Conversion Pipeline: From discussion in forums and NVIDIA documentation, the typical approach involves:
    • Organizing raw sensor datasets (camera frames, LiDAR sweeps, GPS/IMU streams) into the expected folder structure.
    • Creating or extracting metadata in JSON for timestamps, calibration, and scenario mapping.
    • Passing these through NCore’s validation step (often a CLI utility or Python API) before ingesting into NuRec.developer.nvidia+1​
  • No Published Schema: As of late 2025, there isn’t a published, example JSON or folder-spec documentation for NCore available outside NVIDIA, but you can infer typical structure from NuRec dataset examples on HuggingFace (sequence_tracks.json, camera, lidar folders, usda scene descriptors).huggingface+1​

Relation to USD and Omniverse

  • Pre-USD Container: NCore is not identical to OpenUSD—it’s an “upstream” format used before neural reconstruction. Raw data in NCore is ingested and processed by NuRec, which outputs reconstructed 3D environments as USDZ scene containers, including USD scene graphs and annotation files for Omniverse-based simulation and editing.omniverse.nvidia+1​
  • Downstream Conversion: Once reconstruction is complete, NuRec writes to USDZ:
    • Output includes the .usd scene, nuRec checkpoint, sequence_tracks.json/scene.usda, and optionally OpenDrive (.xodr) for traffic simulation.huggingface+1​
  • USD Ecosystem: NCore itself is not part of the USD standard, but designed to bridge real-world, unstructured sensor data to the standardized, rich USD format omnipresent in Omniverse workflows.omniverse.nvidia+1​
  • Pipeline: NCore → NuRec → USDZ (for simulation, visualization, Omniverse Kit/Isaac integration).omniverse.nvidia+1​

Converting Existing Datasets

  • Manual Mapping: Organize your data (images, point clouds, GPS/IMU, etc.) into folders matching the NCore expected structure.
  • Metadata Prep: Write or generate JSON files for frame timestamps, calibration, trajectories, and object tracks, matching requirements seen in HuggingFace NuRec examples.huggingface+1​
  • Validation: Use the NCore validation tool or scripts to ensure quality and ingestability.
  • Reference Example: The PhysicalAI-Autonomous-Vehicles-NuRec dataset shows:
    • sequence_tracks.json (object tracks)
    • camera, lidar data folders
    • USD/A scene descriptors for final export.huggingface+1​

Summary Table

Aspect NCore Format (NuRec) OpenUSD/Omniverse USD
Purpose Raw sensor ingest, sync, validation 3D scene description, simulation
Structure Sensor folders, JSON metadata, auxiliary files USD/USDZ file structure, schemas
Usage Input to NuRec recon models Output from NuRec; input for Omniverse tools
Availability Internal pipeline/tools, limited public details Public spec, broad ecosystem support

In summary:
NCore is a pre-USD data staging and validation format, used by NVIDIA NuRec to convert and package physical sensor datasets for neural reconstruction. You must manually organize data and metadata, validate with NuRec’s NCore tools, and then get USD output for Omniverse simulation. NCore is not an OpenUSD variant, but a bridge for real-world ingestion into USD-based workflows. nvidia+3​

  1. NVIDIA Omniverse NuRec — NVIDIA Omniverse NuRec
  2. Question about the NCore data format in NuRec – structure, conversion, and relation to USD
  3. Accelerating AV Simulation with Neural Reconstruction and World Foundation Models | NVIDIA Technical Blog
  4. nvidia/PhysicalAI-Autonomous-Vehicles-NuRec · Datasets at Hugging Face
  5. nvidia/PhysicalAI-Robotics-NuRec · Datasets at Hugging Face
  6. Neural (NuRec) Rendering — Omniverse Materials and Rendering
  7. USD Connections Overview — Omniverse Connect
  8. https://metaverse-standards.org/wp-content/uploads/Gaussian-Splats-Frontier-Workshop-SIGGRAPH-Vancouver-Aug25-FINAL.pdf
  9. USD, Fabric, and USDRT — usdrt 7.5.1 documentation
  10. Neural Volume Rendering — Isaac Sim Documentation
  11. https://www.youtube.com/watch?v=ybtJxQbj2NE
  12. Omniverse - NVIDIA Developer Forums
  13. GitHub - NVIDIA-Omniverse/synthetic-data-examples: Synthetic Data Generation Examples
  14. Omniverse Glossary of Terms — Omniverse USD
  15. NVIDIA Announces NuRec Gaussian Splatting Libraries at SIGGRAPH - Radiance Fields
  16. USD Code API — Omniverse Services
  17. GitHub - NVIDIA-RTX/RTXNTC: NVIDIA Neural Texture Compression SDK
  18. Reddit - The heart of the internet
  19. https://www.bondcap.com/report/pdf/Trends_Artificial_Intelligence.pdf
  20. USD Search API 1.0.0 beta — Omniverse Services
4

Hi Richard3D,

Thank you so much for the detailed follow-up. That clarifies the NCore data format significantly!

Based on your explanation, I was re-reading the documentation, and I have a few more questions about the “nurec container” mentioned in the same section:

  1. The docs mention it includes “data quality validation tools.” Does this container also bundle the core reconstruction and rendering algorithms that NuRec uses? Or is it primarily for data preparation?

  2. Following up on that, are the reconstruction/rendering algorithms within the NuRec pipeline (whether in this container or not) publicly available? For instance, are there published papers, code repositories, or packages related to them?

  3. I’m also trying to understand the relationship between this NuRec pipeline and the open-source 3dgrut (https://github.com/nv-tlabs/3dgrut/). Is 3dgrut the public implementation of the NuRec algorithms, or perhaps a component that works alongside it?

  4. Ultimately, if my goal is to use the NuRec methodology for my own reconstruction projects, which component should I be focusing on? Is 3dgrut the correct tool for that, or should I be looking for access to the nurec container’s tools?

Thanks again for all the help!