Does TAO 5.0 support exporting a model trained by TAO 3.0?

[Morganh]

@hyperlight

We tried that and got the same error:

[I] [TRT] Calibration table does not match calibrator algorithm type.
[E] Error[3]: IExecutionContext::executeV2: Error Code 3: API Usage Error (Parameter check failed, condition: nullPtrAllowed. Tensor "Input" is bound to nullptr, which is allowed only for an empty input tensor, shape tensor, or an output tensor associated with an IOuputAllocator.)
[E] Error[2]: [calibrator.cpp::calibrateEngine::1236] Error Code 2: Internal Error (Assertion context->executeV2(bindings.data()) failed. )

For running trtexec to generate tensorrt engine, I can generate the int8 engine successfully on one A40 machine. My command and logs are as below. You can refer to my command to use --workspace=10000000 to double check.

$ yolo_v4 gen_trt_engine -m yolo-TAO3-QAT-Export-TAO5/etlt-to-onnx/yolov4_cspdarknet19_epoch_240.onnx \
-e ./spec.txt --data_type int8 -r result \
--batch_size 1 --min_batch_size 1 --opt_batch_size 1 --max_batch_size 1 \
--cal_json_file /localhome/local-morganh/forum_350314_yolov4/customer_data/cal_morgan.json \
--engine_file trt.morgan.int8 \
--cal_data_file /localhome/local-morganh/forum_350314_yolov4/customer_data/cal_morgan.tensorfile \
--cal_image_dir /localhome/local-morganh/forum_350314_yolov4/customer_data/sample_inputs_copy \
--cal_cache_file /localhome/local-morganh/forum_350314_yolov4/customer_data/cal_morgan.cache

$ /usr/src/tensorrt/bin/trtexec \
                --onnx=yolo-TAO3-QAT-Export-TAO5/etlt-to-onnx/yolov4_cspdarknet19_epoch_240.onnx \
                --saveEngine=/localhome/local-morganh/forum_350314_yolov4/customer_data/trt.morgan.int8.trtexec \
                --minShapes=Input:1x3x832x4096 \
                --optShapes=Input:1x3x832x4096 \
                --maxShapes=Input:1x3x832x4096 \
                --int8 \
                --calib=/localhome/local-morganh/forum_350314_yolov4/customer_data/cal_morgan.cache \
                --workspace=10000000

20251201_forum_350314_run_trtexec_generate_engine.txt (42.6 KB)

@hyperlight

We deployed yolov4 model trained by TAO 3.0 as int8 precision egnine using tao-converter up to DS6.4. We are upgrading to DS8.0 and tao-converter is no longer supported.

Then, please use trtexec to generate engine. Or you can directly deploy onnx file and cal.bin file to deepstream apps and let it generate tensorrt engine. The example config file can be found in deepstream_tao_apps/deepstream_app_tao_configs/nvinfer/config_infer_primary_yolov4.txt at release/tao_ds7.1ga · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub.

[yingliu]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.