Tao-converter error

complete deepstream apps

In V100, can you delete engine and try? Just let deepstream generate engine.
Please share the config file too.

im using tao-convertor to generate the engine file , i genereated but am getting the above error while runnning the application

In V100, did you build /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so ?

NO , since we are using tao-convertor , we do not need usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so . Correct ?

Refer to YOLOv4 — TAO Toolkit 3.0 documentation
Strongly recommend you go through
Error while converting model using TAO
and Error in Yolov4 engine conversion, - #22 by Morganh as well.

For dgpu , please check
/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so

It is needed to build a new TRT OSS plugin and replace.

below is ie config file , and i have regerated libnvds_infercustomparser_tlt.so file


[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=../../model/Helmet_model/labels.txt
model-engine-file=/app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine

#tlt-encoded-model=../../model/Helmet_model/yolov4_resnet18_epoch_050.etlt
tlt-model-key=nvidia_tlt
infer-dims=3;384;1248
maintain-aspect-ratio=1
uff-input-order=0
uff-input-blob-name=Input
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
num-detected-classes=2
interval=0
gie-unique-id=1
is-classifier=0
#network-type=0
#no cluster
cluster-mode=3
output-blob-names=BatchedNMS
parse-bbox-func-name=NvDsInferParseCustomBatchedNMSTLT
custom-lib-path=/app/deepstream_tlt_apps/post_processor/libnvds_infercustomparser_tlt.so



[class-attrs-0]
post-cluster-threshold=1
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.7
dbscan-min-score=0.95
#minBoxes=3
roi-top-offset=0
roi-bottom-offset=0
detected-min-w=0
detected-min-h=0
detected-max-w=0
detected-max-h=0

below is the error while running :




Now playing...
1 :  file:///app/1080p_length_extension.mp4
Starting pipeline 

gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
0:00:03.214675083 26633      0x1de76a0 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1702> [UID = 1]: deserialized trt engine from :/app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 5
0   INPUT  kFLOAT Input           3x384x1248      min: 1x3x384x1248    opt: 8x3x384x1248    Max: 16x3x384x1248   
1   OUTPUT kINT32 BatchedNMS      1               min: 0               opt: 0               Max: 0               
2   OUTPUT kFLOAT BatchedNMS_1    200x4           min: 0               opt: 0               Max: 0               
3   OUTPUT kFLOAT BatchedNMS_2    200             min: 0               opt: 0               Max: 0               
4   OUTPUT kFLOAT BatchedNMS_3    200             min: 0               opt: 0               Max: 0               

0:00:03.214823575 26633      0x1de76a0 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary-inference> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1806> [UID = 1]: Use deserialized engine model: /app/tao-convertor/cuda11.0-trt7.1-20210820T231035Z-001/cuda11.0-trt7.1/trt.engine
0:00:03.232216943 26633      0x1de76a0 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary-inference> [UID 1]: Load new model:config/Helmet/PoC_pgie_config.txt sucessfully
Decodebin child added: source 

Decodebin child added: decodebin0 

Starting wsserver
WebSock Server running with port: %d 36201
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1062, in create_server
    sock.bind(sa)
OSError: [Errno 98] Address already in use

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "PoC_Helmet.py", line 1168, in wsserver
    loop.run_until_complete(start_server)
  File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
    return future.result()
  File "/usr/lib/python3.6/asyncio/tasks.py", line 537, in _wrap_awaitable
    return (yield from awaitable.__await__())
  File "/usr/local/lib/python3.6/dist-packages/websockets/legacy/server.py", line 1071, in __await_impl__
    server = await self._create_server()
  File "/usr/lib/python3.6/asyncio/base_events.py", line 1066, in create_server
    % (sa, err.strerror.lower()))
OSError: [Errno 98] error while attempting to bind on address ('0.0.0.0', 36201): address already in use
Decodebin child added: qtdemux0 

Decodebin child added: multiqueue0 

Decodebin child added: h264parse0 

Decodebin child added: capsfilter0 

Decodebin child added: aacparse0 

Decodebin child added: avdec_aac0 

Decodebin child added: nvv4l2decoder0 

In cb_newpad

gstname= video/x-raw
features= <Gst.CapsFeatures object at 0x7f00a5fa32e8 (GstCapsFeatures at 0x7eff1002e7a0)>
In cb_newpad

gstname= audio/x-raw
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Assertion failed: status == STATUS_SUCCESS
/home/ubuntu/TensorRT/plugin/batchedNMSPlugin/batchedNMSPlugin.cpp:246
Aborting...

Aborted

Did you build /usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so according to deepstream_tao_apps/TRT-OSS/x86 at release/tao3.0 · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub ?

im trying to build from TensorRT ,

export PATH=/usr/local/cuda-11.1/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-11.1/lib64:$LD_LIBRARY_PATH
export CPATH=/usr/local/cuda-11.1/include:$CPATH
export LIBRARY_PATH=/usr/local/cuda-11.1/lib64:$LD_LIBRARY_PATH
export TRT_LIBPATH=/app/TensorRT
export TENSORRT_LIBRARY_INFER=/usr/lib/x86_64-linux-gnu/libnvinfer.so.7
export TENSORRT_LIBRARY_INFER_PLUGIN=/usr/lib/x86_64-linux-gnu/libnvinfer_plugin.so.7
export TRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/






  cmake .. -DDCUDA_VERSION=11.1 -DGPU_ARCHS=70 -DCUDNN_LIB=/usr/local/cuda-11.1/lib64  -DTRT_INC_DIR=$TRT_LIBPATH/include/ -DTRT_LIB_DIR=$TRT_LIBPATH/lib/  -DTRT_OUT_DIR=`pwd`/out   -DTENSORRT_LIBRARY_INFER=/usr/lib/x86_64-linux-gnu/ -DTENSORRT_LIBRARY_INFER_PLUGIN=/usr/lib/x86_64-linux-gnu/ -DCMAKE_CUDA_ARCHITECTURES=70   -DCMAKE_CUDA_COMPILER=/usr/local/cuda/cuda-11.1/bin/nvcc -DTENSORRT_LIBRARY_INFER_PLUGIN=/usr/lib/x86_64-linux-gnu/ -DTENSORRT_LIBRARY_MYELIN=/usr/lib/x86_64-linux-gnu/

but i get below error


Building for TensorRT version: 7.2.3, library version: 7
CMake Error at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.21/Modules/CMakeDetermineCUDACompiler.cmake:212 (message):
  Couldn't find CUDA library root.
Call Stack (most recent call first):
  CMakeLists.txt:46 (project)


-- Configuring incomplete, errors occurred!
See also "/app/TensorRT/build/CMakeFiles/CMakeOutput.log".
See also "/app/TensorRT/build/CMakeFiles/CMakeError.log".

below is the log



Checking whether the CUDA compiler is NVIDIA using "" did not match "nvcc: NVIDIA \(R\) Cuda compiler driver":

Checking whether the CUDA compiler is Clang using "" did not match "(clang version)":

Checking whether the CUDA compiler is NVIDIA using "" did not match "nvcc: NVIDIA \(R\) Cuda compiler driver":

Checking whether the CUDA compiler is Clang using "" did not match "(clang version)":

You can refer to the steps in YOLOv4 — TAO Toolkit 3.0 documentation

But please note that there is one small issue in one step.

/usr/local/bin/cmake .. -DGPU_ARCHS=xy  -DTRT_LIB_DIR=/usr/lib/aarch64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=`pwd`/out

should be

/usr/local/bin/cmake .. -DGPU_ARCHS=xy  -DTRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=`pwd`/out

“error staing :Couldn’t find CUDA library root” on build . Could you help on this

root@64877619f02d:/app/new/TensorRT/build# /usr/local/bin/cmake .. -DGPU_ARCHS=70  -DTRT_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCMAKE_C_COMPILER=/usr/bin/gcc -DTRT_BIN_DIR=`pwd`/out
Building for TensorRT version: 7.2.2, library version: 7
-- The CXX compiler identification is GNU 7.5.0
CMake Error at /usr/local/lib/python3.6/dist-packages/cmake/data/share/cmake-3.21/Modules/CMakeDetermineCUDACompiler.cmake:212 (message):
  Couldn't find CUDA library root.
Call Stack (most recent call first):
  CMakeLists.txt:46 (project)


-- Configuring incomplete, errors occurred!
See also "/app/new/TensorRT/build/CMakeFiles/CMakeOutput.log".

Did you install CUDA in your dgpu device?

is there a command to check ?

$ dpkg -l |grep cuda

root@64877619f02d:/app# dpkg -l |grep cuda
ii  cuda-compat-11-1                     455.45.01-1                         amd64        CUDA Compatibility Platform
ii  cuda-cudart-11-1                     11.1.74-1                           amd64        CUDA Runtime native Libraries
ii  cuda-libraries-11-1                  11.1.1-1                            amd64        CUDA Libraries 11.1 meta-package
ii  cuda-nvrtc-11-1                      11.1.105-1                          amd64        NVRTC native runtime libraries
ii  cuda-nvtx-11-1                       11.1.74-1                           amd64        NVIDIA Tools Extension
ii  graphsurgeon-tf                      7.2.2-1+cuda11.1                    amd64        GraphSurgeon for TensorRT package
ii  libcudnn8                            8.0.5.39-1+cuda11.1                 amd64        cuDNN runtime libraries
hi  libnccl2                             2.8.3-1+cuda11.1                    amd64        NVIDIA Collective Communication Library (NCCL) Runtime
ii  libnvinfer-plugin7                   7.2.2-1+cuda11.1                    amd64        TensorRT plugin libraries
ii  libnvinfer7                          7.2.2-1+cuda11.1                    amd64        TensorRT runtime libraries
ii  libnvonnxparsers7                    7.2.2-1+cuda11.1                    amd64        TensorRT ONNX libraries
ii  libnvparsers7                        7.2.2-1+cuda11.1                    amd64        TensorRT parsers libraries
ii  python-libnvinfer                    7.2.2-1+cuda11.1                    amd64        Python bindings for TensorRT
ii  python3-libnvinfer                   7.2.2-1+cuda11.1                    amd64        Python 3 bindings for TensorRT
ii  uff-converter-tf                     7.2.2-1+cuda11.1                    amd64        UFF converter for TensorRT package

You are running inside a docker, right? Which docker? Can you try outside the docker?

Another machine is T4 ( i was able to build also there successfully)
Current is V100 (only in docker deepsteam sdk is there , )

Which dgpu is in another machine?
And which dgpu is in your current machine?