Dear @Morganh
I trained teacher and student model inside 6.26 containers and then I created notebook for training using tao.
I am able to train model both teacher and student and able to generate fp16 engine file.
But when I try to generate calibration cache and int8 file using !tao deploy rtdetr gen_trt_engine -e /workspace/tao-experiments/specs/deploy.yaml
I am getting following error.
2026-06-23 12:30:41,457 [TAO Toolkit] [INFO] root 160: Registry: ['nvcr.io']
2026-06-23 12:30:41,535 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:6.26.3-deploy
2026-06-23 12:30:41,558 [TAO Toolkit] [WARNING] nvidia_tao_cli.components.docker_handler.docker_handler 303:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/smarg/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
2026-06-23 12:30:41,558 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 316: Printing tty value True
2026-06-23 07:00:43,126 - nvidia_tao_core.microservices.utils.job_utils.workflow - INFO - Logging configured at level: INFO
2026-06-23 07:00:45,669 [TAO Toolkit] [INFO] nvidia_tao_core.microservices.utils.job_utils.workflow 61: Logging configured at level: INFOsys:1: UserWarning:
'deploy.yaml' is validated against ConfigStore schema with the same name.
This behavior is deprecated in Hydra 1.1 and will be removed in Hydra 1.2.
See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/automatic_schema_matching for migration instructions.
/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/cv/common/hydra/hydra_runner.py:99: UserWarning:
'deploy.yaml' is validated against ConfigStore schema with the same name.
This behavior is deprecated in Hydra 1.1 and will be removed in Hydra 1.2.
See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/automatic_schema_matching for migration instructions.
_run_hydra(
/usr/local/lib/python3.12/dist-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
ret = run_job(
Gen_trt_engine results will be saved at: /workspace/tao-experiments/results/trt_engine_logs
Log file already exists at /workspace/tao-experiments/results/trt_engine_logs/status.json
Starting rtdetr gen_trt_engine.
No QDQ quantization operators found in ONNX model.
[06/23/2026-07:00:47] [TRT] [I] [MemUsageChange] Init builder kernel library: CPU +989, GPU +4, now: CPU 1278, GPU 894 (MiB)Setting up QAT mode: False
[06/23/2026-07:00:47] [TRT] [I] Successfully created plugin: MultiscaleDeformableAttnPlugin_TRTParsing ONNX modelin_namespace:istil_TeR50_StR18_Res352x576_FP32_V2.1.pth.onnx
List inputs:
Input 0 -> inputs.
(3, 352, 576).
-1.
Network Description
Input 'inputs' with shape (-1, 3, 352, 576) and dtype DataType.FLOAT
Output 'pred_logits' with shape (-1, 300, 4) and dtype DataType.FLOAT
Output 'pred_boxes' with shape (-1, 300, 4) and dtype DataType.FLOAT
Calibrating using ImageBatcher
TensorRT engine build configurations:
OptimizationProfile:
"inputs": (1, 3, 352, 576), (50, 3, 352, 576), (50, 3, 352, 576)
BuilderFlag.FP16
BuilderFlag.INT8
BuilderFlag.TF32
Note: max representabile value is 2,147,483,648 bytes or 2GB.
MemoryPoolType.WORKSPACE = 2147483648 bytes
MemoryPoolType.DLA_MANAGED_SRAM = 0 bytes
MemoryPoolType.DLA_LOCAL_DRAM = 1073741824 bytes
MemoryPoolType.DLA_GLOBAL_DRAM = 536870912 bytes
MemoryPoolType.TACTIC_DRAM = 16606232576 bytes
MemoryPoolType.TACTIC_SHARED_MEMORY = 1073741824 bytes
Tactic Sources = 24
[06/23/2026-07:01:00] [TRT] [I] Starting Calibration.orRT-managed allocation in IExecutionContext creation: CPU +1, GPU +1867, now: CPU 1, GPU 2030 (MiB)Calibrating image 8 / 400
[06/23/2026-07:01:00] [TRT] [E] [calibrator.cpp::calibrateEngine::1219] Error Code 2: Internal Error (Assertion context->executeV2(bindings.data()) failed. In calibrateEngine at optimizer/api/calibrator.cpp:1219)Parameter validation error: 'NoneType' object does not support the context manager protocol
Error executing job with overrides: []Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/cv/common/decorators.py", line 115, in _func
raise e
File "/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/cv/common/decorators.py", line 76, in _func
runner(cfg, **kwargs)
File "/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/cv/rtdetr/scripts/gen_trt_engine.py", line 72, in main
builder.create_engine(**create_engine_kwargs)
File "/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/engine/builder.py", line 507, in create_engine
self._write_engine(engine_path)
File "/usr/local/lib/python3.12/dist-packages/nvidia_tao_deploy/engine/builder.py", line 519, in _write_engine
with self.builder.build_serialized_network(self.network, self.config) as engine_bytes, \
TypeError: 'NoneType' object does not support the context manager protocol
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
PyCUDA WARNING: a clean-up operation failed (dead context maybe?)
cuMemFree failed: an illegal memory access was encountered
2026-06-23 07:01:01,182 - nvidia_tao_deploy.cv.common.entrypoint.entrypoint_hydra - WARNING - Telemetry data couldn't be sent, but the command ran successfully.
2026-06-23 07:01:01,182 - nvidia_tao_deploy.cv.common.entrypoint.entrypoint_hydra - WARNING - 'str' object has no attribute 'decode'
2026-06-23 12:31:01,560 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 381: Stopping container.
Below is my deploy.yaml file
# deploy_spec.yaml
gen_trt_engine:
results_dir: /workspace/tao-experiments/results/trt_engine_logs
onnx_file: /workspace/tao-experiments/results/student/distill/VehicleDetection_RtDetr_Distil_TeR50_StR18_Res352x576_FP32_V2.1.pth.onnx
trt_engine: /workspace/tao-experiments/results/student/distill/VehicleDetection_RtDetr_Distil_TeR50_StR18_Res352x576_int8_V2.1.pth.engine
# input_channel: 3
# input_width: 576
# input_height: 352
tensorrt:
data_type: int8
# data_type: fp16
workspace_size: 2048
min_batch_size: 1
opt_batch_size: 50
max_batch_size: 50
calibration:
cal_image_dir:
- /workspace/tao-experiments/dataset/calibration_data/image_2
cal_cache_file: /workspace/tao-experiments/results/student/distill/VehicleDetection_RtDetr_Distil_TeR50_StR18_Res352x576_FP32_V2.1.pth.cache
cal_batch_size: 8
cal_batches: 50
I had tried directly run inside container but same issue.
Script for calib data generation.
import json
import shutil
import os
from pathlib import Path
# Paths
val_json = '../dataset/val/val_annotations_coco.json'
val_images = '../dataset/val/image_2'
output_dir = '../calibration_data'
output_json = '../calibration_data/calib_annotations.json'
# Create output directory
os.makedirs(output_dir, exist_ok=True)
# Load validation annotations
with open(val_json, 'r') as f:
coco_data = json.load(f)
# Take first 500 images (or use random sampling)
num_images = 2000
selected_images = coco_data['images'][:num_images]
selected_img_ids = {img['id'] for img in selected_images}
# Filter annotations
selected_annotations = [ann for ann in coco_data['annotations'] if ann['image_id'] in selected_img_ids]
# Create new COCO data structure
calib_data = {
'images': selected_images,
'annotations': selected_annotations,
'categories': coco_data['categories']
}
# Save new JSON
with open(output_json, 'w') as f:
json.dump(calib_data, f, indent=2)
# Copy corresponding images
for img in selected_images:
src = os.path.join(val_images, img['file_name'])
dst = os.path.join(output_dir, img['file_name'])
if os.path.exists(src):
shutil.copy(src, dst)
print(f"Created calibration set with {len(selected_images)} images")
print(f"JSON saved to: {output_json}")
print(f"Images saved to: {output_dir}")
Can u please suggest how to resolve this issue so that I can generate calib file and please let me know if further details are needed.
Thanks.