Tao-converter error

NVIDIA-SMI 460.32.03
Driver Version: 460.32.03
CUDA Version: 11.2

i get the above error

command :

./tao-converter -k nvidia_tlt -p Input,1x3x384x1248,8x3x384x1248,16x3x384x1248 -d 3,384,1248 -o BatchedNMS -e trt.engine -m 1 -t fp16 -i nchw /app/tao-convertor/models/model//yolov4_resnet18_epoch_170.etlt

 /app/tao-convertor/models/model//yolov4_resnet18_epoch_170.etlt
[WARNING] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[WARNING] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[INFO] ModelImporter.cpp:135: No importer registered for op: BatchedNMSDynamic_TRT. Attempting to import as plugin.
[INFO] builtin_op_importers.cpp:3771: Searching for plugin: BatchedNMSDynamic_TRT, plugin_version: 1, plugin_namespace: 
[INFO] builtin_op_importers.cpp:3788: Successfully created plugin: BatchedNMSDynamic_TRT
[INFO] Detected input dimensions from the model: (-1, 3, 384, 1248)
[INFO] Model has dynamic shape. Setting up optimization profiles.
[INFO] Using optimization profile min shape: (1, 3, 384, 1248) for input: Input
[INFO] Using optimization profile opt shape: (8, 3, 384, 1248) for input: Input
[INFO] Using optimization profile max shape: (16, 3, 384, 1248) for input: Input
[INFO] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[INTERNAL_ERROR] Assertion failed: status == STATUS_SUCCESS
/home/ubuntu/TensorRT/plugin/batchedNMSPlugin/batchedNMSPlugin.cpp:246
Aborting...




Aborted

Is above correct?
How about /app/tao-convertor/models/model/yolov4_resnet18_epoch_170.etlt

correct

Please refer to Error while converting model using TAO
and Error while converting model using TAO

Make check

  • TRT OSS plugin is built and replaced
  • tao-converter version is compatible
  • Or use official model to test

i was able to generate the engine file , but while running i get the below error message “ABORTED” on V100…

Creating sink bin
sink-bin-00
Creating nvvidconv pre

Creating nvosd

Creating nvvidconv

Creating H264 Encoder
Creating H264 rtppay

 updsink_port_num = %u 6001
linking srcpad to sinkpad # sinbin  0


 uri = %u /ds-test

 updsink_port_num = %u 6001

 *** DeepStream: Launched RTSP Streaming at rtsp:// 0.0.0.0 : 26001 /ds-test ***


Now playing...
1 :  file:///app/uploads/videos/tempUser/337817d77a7d76c251507f0c850be339
Starting pipeline

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:05.253206092   694      0x1af0e90 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 1]: deserialized trt engine from :/app/model/IndianVehicle_model/trt.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 5
0   INPUT  kFLOAT Input           3x384x1248      min: 1x3x384x1248    opt: 8x3x384x1248    Max: 16x3x384x1248
1   OUTPUT kINT32 BatchedNMS      1               min: 0               opt: 0               Max: 0
2   OUTPUT kFLOAT BatchedNMS_1    200x4           min: 0               opt: 0               Max: 0
3   OUTPUT kFLOAT BatchedNMS_2    200             min: 0               opt: 0               Max: 0
4   OUTPUT kFLOAT BatchedNMS_3    200             min: 0               opt: 0               Max: 0

0:00:05.253342517   694      0x1af0e90 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 1]: Use deserialized engine model: /app/model/IndianVehicle_model/trt.engine
0:00:05.271375736   694      0x1af0e90 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:config/Vehicle/PoC_pgie_config.txt sucessfully
Decodebin child added: source

Decodebin child added: decodebin0

Decodebin child added: qtdemux0

Decodebin child added: multiqueue0

Decodebin child added: h264parse0

Decodebin child added: capsfilter0

Decodebin child added: aacparse0

Decodebin child added: avdec_aac0

Decodebin child added: nvv4l2decoder0

In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f643981a3a8 (GstCapsFeatures at 0x7f62a403a600)>
In cb_newpad

gstname= audio/x-raw
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Assertion failed: status == STATUS_SUCCESS
/home/ubuntu/TensorRT/plugin/batchedNMSPlugin/batchedNMSPlugin.cpp:246
Aborting...

Aborted

I am afraid the libnvinfer_plugin.so is not correct. How did you build it?

i changed fp16 to fp32 then i was able to generate engine file for tlt model,

./tao-converter -k nvidia_tlt -p Input,1x3x384x1248,8x3x384x1248,16x3x384x1248 -d 3,384,1248 -o BatchedNMS -e trt.engine -m 1 -t fp32 -i nchw /app/tao-convertor/models/model//yolov4_resnet18_epoch_170.etlt

and when i try to run , i get the above error …

"I’m in a process for moving code from one system to another system and this happened in between "

What do you mean by “moving code” ? What is the code?

from T4 machine to V100 machine

What is the code?

complete deepstream apps

In V100, can you delete engine and try? Just let deepstream generate engine.
Please share the config file too.

im using tao-convertor to generate the engine file , i genereated but am getting the above error while runnning the application

In V100, did you build /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so ?

NO , since we are using tao-convertor , we do not need usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so . Correct ?

Refer to YOLOv4 — TAO Toolkit 3.0 documentation
Strongly recommend you go through
Error while converting model using TAO
and Error in Yolov4 engine conversion, - #22 by Morganh as well.

For dgpu , please check
/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so

It is needed to build a new TRT OSS plugin and replace.

below is ie config file , and i have regerated libnvds_infercustomparser_tlt.so file


[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=../../model/Helmet_model/labels.txt
model-engine-file=/app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine

#tlt-encoded-model=../../model/Helmet_model/yolov4_resnet18_epoch_050.etlt
tlt-model-key=nvidia_tlt
infer-dims=3;384;1248
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
num-detected-classes=2
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
#no cluster
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=/app/deepstream_tlt_apps/post_processor/libnvds_infercustomparser_tlt.so



[class-attrs-0]
post-cluster-threshold=1
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.7
dbscan-min-score=0.95
#minBoxes=3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

below is the error while running :




Now playing...
1 :  file:///app/1080p_length_extension.mp4
Starting pipeline 

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:03.214675083 26633      0x1de76a0 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 1]: deserialized trt engine from :/app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 5
0   INPUT  kFLOAT Input           3x384x1248      min: 1x3x384x1248    opt: 8x3x384x1248    Max: 16x3x384x1248   
1   OUTPUT kINT32 BatchedNMS      1               min: 0               opt: 0               Max: 0               
2   OUTPUT kFLOAT BatchedNMS_1    200x4           min: 0               opt: 0               Max: 0               
3   OUTPUT kFLOAT BatchedNMS_2    200             min: 0               opt: 0               Max: 0               
4   OUTPUT kFLOAT BatchedNMS_3    200             min: 0               opt: 0               Max: 0               

0:00:03.214823575 26633      0x1de76a0 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 1]: Use deserialized engine model: /app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine
0:00:03.232216943 26633      0x1de76a0 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:config/Helmet/PoC_pgie_config.txt sucessfully
Decodebin child added: source 

Decodebin child added: decodebin0 

Starting wsserver
WebSock Server running with port: %d 36201
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1062, in create_server
    sock.bind(sa)
OSError: [Errno 98] Address already in use

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "PoC_Helmet.py", line 1168, in wsserver
    loop.run_until_complete(start_server)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "/usr/lib/python3.6/asyncio/tasks.py", line 537, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/usr/local/lib/python3.6/dist-packages/websockets/legacy/server.py", line 1071, in __await_impl__
    server = await self._create_server()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1066, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 98] error while attempting to bind on address ('0.0.0.0', 36201): address already in use
Decodebin child added: qtdemux0 

Decodebin child added: multiqueue0 

Decodebin child added: h264parse0 

Decodebin child added: capsfilter0 

Decodebin child added: aacparse0 

Decodebin child added: avdec_aac0 

Decodebin child added: nvv4l2decoder0 

In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f00a5fa32e8 (GstCapsFeatures at 0x7eff1002e7a0)>
In cb_newpad

gstname= audio/x-raw
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Assertion failed: status == STATUS_SUCCESS
/home/ubuntu/TensorRT/plugin/batchedNMSPlugin/batchedNMSPlugin.cpp:246
Aborting...

Aborted

Did you build /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so according to deepstream_tao_apps/TRT-OSS/x86 at release/tao3.0 · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub ?