Error when generating engine file from a TAO trained yolov4_tiny model in Deepstream 6.1.1

Please provide complete information as applicable to your setup.

Running in a container
nvcr.io/nvidia/deepstream-l4t:6.1.1-samples

• Hardware Platform (Jetson / GPU) Jetson Xavier NX
• DeepStream Version 6.1.1-samples container
• JetPack Version (valid for Jetson only) 5.0.2 (part of deepstream container)
• TensorRT Version 8.4.1-1+cuda11.4 (part of the deepstream container
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) Bug
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) Take etlt file generated from tao export and try to run it with the deepstream-test5-app
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

I have trained a custom model using tao and yolov4_tiny. When I export the model using tao and try to use the etlt file in my deepstream application, I’m getting the following output.

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

WARN !! Hardware mode deprecated. Prefer GPU mode instead

(deepstream-test5-app:1): GLib-CRITICAL **: 18:26:58.264: g_strrstr: assertion 'haystack != NULL' failed
nvds_msgapi_connect : connect success
Opening in BLOCKING MODE
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Initialized
0:00:06.074411373     1 0xaaab2463aa00 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/app/resources/custom_models/../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt_b4_gpu0_fp32.engine failed
0:00:06.143553366     1 0xaaab2463aa00 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/app/resources/custom_models/../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt_b4_gpu0_fp32.engine failed, try rebuild
0:00:06.144035480     1 0xaaab2463aa00 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
parseModel: Failed to parse ONNX model
ERROR: Failed to build network, error in model parsing.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:09.108453273     1 0xaaab2463aa00 ERROR                nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1943> [UID = 1]: build engine file failed
ERROR: [TRT]: 2: [logging.cpp::decRefCount::61] Error Code 2: Internal Error (Assertion mRefCount > 0 failed. )
corrupted size vs. prev_size while consolidating

Here is my inference config file:

# Copyright (c) 2018 NVIDIA Corporation.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

# Following properties are mandatory when engine files are not specified:
#   int8-calib-file(Only in INT8)
#   Caffemodel mandatory properties: model-file, proto-file, output-blob-names
#   UFF: uff-file, input-dims, uff-input-blob-name, output-blob-names
#   ONNX: onnx-file
#
# Mandatory properties for detectors:
#   num-detected-classes
#
# Optional properties for detectors:
#   enable-dbscan(Default=false), interval(Primary mode only, Default=0)
#   custom-lib-path,
#   parse-bbox-func-name
#
# Mandatory properties for classifiers:
#   classifier-threshold, is-classifier
#
# Optional properties for classifiers:
#   classifier-async-mode(Secondary mode only, Default=false)
#
# Optional properties in secondary mode:
#   operate-on-gie-id(Default=0), operate-on-class-ids(Defaults to all classes),
#   input-object-min-width, input-object-min-height, input-object-max-width,
#   input-object-max-height
#
# Following properties are always recommended:
#   batch-size(Default=1)
#
# Other optional properties:
#   net-scale-factor(Default=1), network-mode(Default=0 i.e FP32),
#   model-color-format(Default=0 i.e. RGB) model-engine-file, labelfile-path,
#   mean-file, gie-unique-id(Default=0), offsets, gie-mode (Default=1 i.e. primary),
#   custom-lib-path, network-mode(Default=0 i.e FP32)
#
# The values in the config file are overridden by values set through GObject
# properties.


[property]
workspace-size=2500
gie-unique-id=1
gpu-id=0
net-scale-factor=1
offsets=103.939;116.779;123.68
#infer-dims=3;544;960
#infer-dims=3;384;384
infer-dims=3;384;1248

## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0

#0=RGB, 1=BGR, 2=GRAY
model-color-format=1

model-engine-file=../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt_b4_gpu0_fp32.engine
tlt-encoded-model=../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt
tlt-model-key=<model-key>

num-detected-classes=1
labelfile-path=../custom_models/labels.txt

uff-input-order=0
uff-input-blob-name=Input

## 0=Detector
network-type=0

## 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
cluster-mode=2


output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=/opt/nvidia/deepstream/deepstream/lib/libnvds_infercustomparser.so

enable-dla=1
use-dla-core=0

batch-size=4

[class-attrs-all]
pre-cluster-threshold=0.90
nms-iou-threshold=0.20

from the logs, parsing model failed, please check if the model is complete and valid, and tlt-model-key is correct.

The tlt-model-key is the same one used in the tao training command. What’s the best way to determine if the model is complete and valid? I’ve tried copying the model over from the machine running tao multiple times and I’ve tried running the export from the .tlt file multiple times and it yields a .etlt file that’s the same size each time.

Thanks.

I’ve tried running tao inference using the tlt model and that works successfully.

can you convert the model successfully by tao-convert? here is a sample: ./tao-converter -k tlt_encode -t fp32 -p Input,1x3x544x960,4x3x544x960,4x3x544x960
-e yolov4_cspdarknet_tiny_epoch_200.etlt.engine yolov4_cspdarknet_tiny_epoch_200.etlt

I tried that and am getting the following error message:

[INFO] [MemUsageChange] Init CUDA: CPU +187, GPU +0, now: CPU 209, GPU 2114 (MiB)
[INFO] [MemUsageChange] Init builder kernel library: CPU +131, GPU +138, now: CPU 359, GPU 2267 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename:   /tmp/filepNELKp
[INFO] ONNX IR version:  0.0.8
[INFO] Opset version:    15
[INFO] Producer name:
[INFO] Producer version:
[INFO] Domain:
[INFO] Model version:    0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[ERROR] ModelImporter.cpp:773: While parsing node number 54 [Reshape -> "bg_reshape/Reshape:0"]:
[ERROR] ModelImporter.cpp:774: --- Begin node ---
[ERROR] ModelImporter.cpp:775: input: "bg_permute/transpose:0"
input: "shape_tensor2"
output: "bg_reshape/Reshape:0"
name: "bg_reshape"
op_type: "Reshape"

[ERROR] ModelImporter.cpp:776: --- End node ---
[ERROR] ModelImporter.cpp:778: ERROR: ModelImporter.cpp:163 In function parseGraph:
[6] Invalid Node - bg_reshape
Attribute not found: allowzero
Invalid Node - bg_reshape
Attribute not found: allowzero
[ERROR] Failed to parse the model, please check the encoding key to make sure it's correct
[INFO] Detected input dimensions from the model: (-1, 3, 384, 1248)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 544, 960) for input: Input
[INFO] Using optimization profile opt shape: (4, 3, 544, 960) for input: Input
[INFO] Using optimization profile max shape: (4, 3, 544, 960) for input: Input
[ERROR] 4: [network.cpp::validate::2671] Error Code 4: Internal Error (Network must have at least one output)
[ERROR] Unable to create engine
Segmentation fault (core dumped)

please try ./tao-converter -k tlt_encode -t fp32 -p Input,1x3x384x1248,4x3x384x1248,4x3x384x1248 -e yolov4_cspdarknet_tiny_epoch_200.etlt.engine yolov4_cspdarknet_tiny_epoch_200.etlt

Still getting the same error message:

[INFO] [MemUsageChange] Init CUDA: CPU +187, GPU +0, now: CPU 209, GPU 2137 (MiB)
[INFO] [MemUsageChange] Init builder kernel library: CPU +131, GPU +122, now: CPU 359, GPU 2276 (MiB)
[INFO] ----------------------------------------------------------------
[INFO] Input filename:   /tmp/file7yeRjQ
[INFO] ONNX IR version:  0.0.8
[INFO] Opset version:    15
[INFO] Producer name:
[INFO] Producer version:
[INFO] Domain:
[INFO] Model version:    0
[INFO] Doc string:
[INFO] ----------------------------------------------------------------
[WARNING] onnx2trt_utils.cpp:367: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[ERROR] ModelImporter.cpp:773: While parsing node number 54 [Reshape -> "bg_reshape/Reshape:0"]:
[ERROR] ModelImporter.cpp:774: --- Begin node ---
[ERROR] ModelImporter.cpp:775: input: "bg_permute/transpose:0"
input: "shape_tensor2"
output: "bg_reshape/Reshape:0"
name: "bg_reshape"
op_type: "Reshape"

[ERROR] ModelImporter.cpp:776: --- End node ---
[ERROR] ModelImporter.cpp:778: ERROR: ModelImporter.cpp:163 In function parseGraph:
[6] Invalid Node - bg_reshape
Attribute not found: allowzero
Invalid Node - bg_reshape
Attribute not found: allowzero
[ERROR] Failed to parse the model, please check the encoding key to make sure it's correct
[INFO] Detected input dimensions from the model: (-1, 3, 384, 1248)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 384, 1248) for input: Input
[INFO] Using optimization profile opt shape: (4, 3, 384, 1248) for input: Input
[INFO] Using optimization profile max shape: (4, 3, 384, 1248) for input: Input
[ERROR] 4: [network.cpp::validate::2671] Error Code 4: Internal Error (Network must have at least one output)
[ERROR] Unable to create engine
Segmentation fault (core dumped)

@zkaiser
Please refer to Error in integrating Yolov4 in Deepstream 6, 6.1, 6.1.1, and 6.2 - #11 by pallavi.halarnkar

Please export a new etlt model by adding --target_opset .

--target_opset 12

I’ve tried adding the --target_opset 12 flag to the command to no avail.

Here’s the full command:

sudo docker run -it \
				--rm \
				--gpus all \
				--runtime nvidia \
				--shm-size=20g \
				-v /home/administrator/tao:/app \
				nvcr.io/nvidia/tao/tao-toolkit:4.0.0-tf1.15.5 \
					yolo_v4_tiny export \
						--gpus 1 \
						-e /app/yolo_v4_tiny/train_config \
						-m /app/yolo_v4_tiny/training-output/weights/yolov4_cspdarknet_tiny_epoch_200.tlt \
						-o /app/yolo_v4_tiny/training-output/weights/yolov4_cspdarknet_tiny_epoch_200.etlt \
						-k <key> \
						--gen_ds_config \
						--data_type fp32 \
						--target_opset 12

and here is the output message:

(gst-plugin-scanner:21): GStreamer-WARNING **: 12:22:27.232: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstchromaprint.so': libavcodec.so.58: cannot open shared object file: No such file or directory
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 204)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)

(gst-plugin-scanner:21): GStreamer-WARNING **: 12:22:27.837: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory

(gst-plugin-scanner:21): GStreamer-WARNING **: 12:22:28.104: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***

WARN !! Hardware mode deprecated. Prefer GPU mode instead

(deepstream-test5-app:1): GLib-CRITICAL **: 12:22:29.213: g_strrstr: assertion 'haystack != NULL' failed
nvds_msgapi_connect : connect success
Opening in BLOCKING MODE
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is ON
[NvMultiObjectTracker] Initialized
0:00:09.077270112     1 0xaaab0fe3a400 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/app/resources/custom_models/../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt_b4_gpu0_fp32.engine failed
0:00:09.163244458     1 0xaaab0fe3a400 WARN                 nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/app/resources/custom_models/../custom_models/yolov4_cspdarknet_tiny_epoch_200.etlt_b4_gpu0_fp32.engine failed, try rebuild
0:00:09.163640911     1 0xaaab0fe3a400 INFO                 nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
parseModel: Failed to parse ONNX model
ERROR: Failed to build network, error in model parsing.
ERROR: Failed to create network using custom network creation function
ERROR: Failed to get cuda engine from custom library API
0:00:11.909346593     1 0xaaab0fe3a400 ERROR                nvinfer gstnvinfer.cpp:640:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1943> [UID = 1]: build engine file failed
ERROR: [TRT]: 2: [logging.cpp::decRefCount::61] Error Code 2: Internal Error (Assertion mRefCount > 0 failed. )
corrupted size vs. prev_size while consolidating

I tried the export again and now it worked. Copying through my build process must not have worked the first time.

Thanks for your help!!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.