How to enable Harmonizer at training time with Nurec 26.04 container

Hi,

I am following the steps in Reconstruct an AV Scene with NuRec — NVIDIA Omniverse NuRec to reconstruct my AD data.

Environment:
Container: nvcr.io/nvidia/nre/nre-ga:26.04

HW: NVIDIA GeForce RTX Blackwell

How could we enable Harmonizer at training time? I checked the available configs, and it seems that only difix.training.enabled=true will enable that at training time. However, enabling it will result in a download model error (reported at How can I enable Difix (Fixer) for training/testing? )

Do we need to use other configs, set model name, URL, etc.?

Thanks & Best Regards,

Tin Dao

You need to have EA access, (Early Access from nvidia contact point)
In next release, it doesn’t require to have EA access.

In the meantime, You might want to download from huggingface directly.

Step would be following

  • Download once (public HF)
    huggingface-cli download nvidia/difix --local-dir ./difix_weights

  • Then mount it into the container’s cache dir and point NuRec at it

    docker run … \
    –volume $(pwd)/difix_weights:/root/.cache/nre/difix \
    nvcr.io/nvidia/nre/nre-ga:26.04 \
    mode=train … \
    difix=cosmos_difix \
    difix.training.enabled=true \
    difix.cache_dir=/root/.cache/nre/difix \
    difix.model_filename=<the_downloaded_filename>.pt

then It might not hit NGC.

Thank you for your response.

I checked nvidia/difix · Hugging Face (models are in safetensors format) and tried to use it with the NuRec container but got the error below.
NuRec run command:

docker run …
–volume /work/nurec/models/difix:/workdir/difix
nvcr.io/nvidia/nre/nre-ga:26.04
mode=train …
difix=cosmos_difix
difix.training.enabled=true
difix.cache_dir=/workdir/difix/unet
difix.model_filename=diffusion_pytorch_model.safetensors

Error message:

NuRec::NRend][INFO] ::: Parsed model . with 0 particles
[NuRec::NRend][INFO] ::: Model nre::sh-gaussians => nre_sh-gaussians@26.4.146 opened
[2026-07-24 09:23:11,723][INFO] Gaussian3DNRenderer: enable_ray_based_culling: True
[2026-07-24 09:23:11,761][INFO] Loading slang module: /tmp/collector_4cd71d17.slang for configurations: [CollectorConfiguration(parameters=(‘PositionsRotationsDensitiesCollector_Deformable’, ‘ScalesCollector_Exp’, ‘SphericalFeaturesCollector_Copy<3,45>’))]
wandb: WARNING The anonymous setting has no effect and will be removed in a future version.
ValidationError
1 validation error for Settings
mode
Input should be ‘online’, ‘offline’, ‘shared’, ‘disabled’, ‘dryrun’ or ‘run’ [type=literal_error, input_value=‘’, input_type=str]
For further information visit Validation errors | Pydantic Docs

Actually, my original question was about enabling Harmonizer (nvidia/Harmonizer · Hugging Face) at training-time. I also tried to download the Harmonizer model and use it with the NuRec container but got the same error.
NuRec run command:

docker run …
–volume /work/nurec/models/harmonizer:/workdir/harmonizer
nvcr.io/nvidia/nre/nre-ga:26.04
mode=train …
difix=cosmos_difix
difix.training.enabled=true
difix.cache_dir=/workdir/harmonizer
difix.model_filename=harmonizer_nontemporal.pt

Error message:

[NuRec::NRend][INFO] ::: Parsed model . with 0 particles
[NuRec::NRend][INFO] ::: Model nre::sh-gaussians => nre_sh-gaussians@26.4.146 opened
[2026-07-24 09:39:50,876][INFO] Gaussian3DNRenderer: enable_ray_based_culling: True
[2026-07-24 09:39:50,928][INFO] Loading slang module: /tmp/collector_4cd71d17.slang for configurations: [CollectorConfiguration(parameters=(‘PositionsRotationsDensitiesCollector_Deformable’, ‘ScalesCollector_Exp’, ‘SphericalFeaturesCollector_Copy<3,45>’))]
wandb: WARNING The anonymous setting has no effect and will be removed in a future version.
ValidationError
1 validation error for Settings
mode
Input should be ‘online’, ‘offline’, ‘shared’, ‘disabled’, ‘dryrun’ or ‘run’ [type=literal_error, input_value=‘’, input_type=str]
For further information visit Validation errors | Pydantic Docs

Do we need to change/add other configs to make it work?

Thanks & Best Regards,
Tin Dao

You might try following, (It happens earlier because an empty WANDB_MODE is passed into the container. Removing WANDB_MODE, setting it to disabled, or adding logger=dummy fixes it.)

-e WANDB_MODE=disabled \
-v “$PWD/harmonizer_weights:/root/.cache/nre/harmonizer:ro” \
nvcr.io/nvidia/nre/nre-ga:26.04 \
mode=train \
… \
logger=dummy \
difix=cosmos_difix \
difix.training.enabled=true \
difix.cache_dir=/root/.cache/nre/harmonizer \
difix.model_url=https://huggingface.co/nvidia/Harmonizer/resolve/main/harmonizer_nontemporal.pt \
difix.model_filename=harmonizer_nontemporal.pt \
difix.model_resolution=‘[576,1024]’

Thank @dkoy8927,
Your command works, and I could train with harmonizer_nontemporal. However, the result was not good with my data. Therefore, I tried to use “temporal harmonization” with the command:

mode=train

logger=dummy
difix=cosmos_difix
difix.training.enabled=true
difix.cache_dir=/root/.cache/nre/harmonizer
difix.model_url=https://huggingface.co/nvidia/Harmonizer/resolve/main/diffusion_harmonizer.pkl
difix.model_filename=diffusion_harmonizer.pkl
difix.model_resolution=[576,1024]

Unfortunately, I got the following errors. Do we need to add other configs to use temporal harmonization?

[2026-07-27 10:50:39,577][INFO] Using cached Difix model (/workdir/models/harmonizer/diffusion_harmonizer.pkl).
[2026-07-27 10:50:39,578][INFO] DifixModelFactory: checkpoint ready at /workdir/models/harmonizer/diffusion_harmonizer.pkl
[2026-07-27 10:50:39,578][INFO] Using cached Difix model (/workdir/models/harmonizer/diffusion_harmonizer.pkl).
[2026-07-27 10:50:40,368][INFO] Loading model with TE saved at /workdir/models/harmonizer/diffusion_harmonizer.pkl, True
[2026-07-27 10:50:40,368][INFO] has rmsnorm_fwd_inf_ts: True
RuntimeError
PytorchStreamReader failed locating file constants.pkl: file not found

Thanks & Best Regards,
Tin Dao

diffusion_harmonizer.pkl cannot be used as a drop-in replacement for harmonizer_nontemporal.pt in NuRec 26.04. The cosmos_difix integration is invoking the TorchScript loader, while diffusion_harmonizer.pkl is the full temporal checkpoint used by the standalone NVIDIA/harmonizer Python pipeline. The missing constants.pkl error confirms the checkpoint-format mismatch.

You need to wait next release