Tao-converter fails on Nvidia Orin

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware Orin
• Network Type Yolo_v4

viktor@nvidiaorin:~/nvidia$ dpkg -l | grep cuda
ii cuda-cccl-11-4 11.4.167-1 arm64 CUDA CCCL
ii cuda-command-line-tools-11-4 11.4.14-1 arm64 CUDA command-line tools
ii cuda-compiler-11-4 11.4.14-1 arm64 CUDA compiler
ii cuda-cudart-11-4 11.4.167-1 arm64 CUDA Runtime native Libraries
ii cuda-cudart-dev-11-4 11.4.167-1 arm64 CUDA Runtime native dev links, headers
ii cuda-cuobjdump-11-4 11.4.167-1 arm64 CUDA cuobjdump
ii cuda-cupti-11-4 11.4.167-1 arm64 CUDA profiling tools runtime libs.
ii cuda-cupti-dev-11-4 11.4.167-1 arm64 CUDA profiling tools interface.
ii cuda-cuxxfilt-11-4 11.4.167-1 arm64 CUDA cuxxfilt
ii cuda-documentation-11-4 11.4.167-1 arm64 CUDA documentation
ii cuda-driver-dev-11-4 11.4.167-1 arm64 CUDA Driver native dev stub library
ii cuda-gdb-11-4 11.4.167-1 arm64 CUDA-GDB
ii cuda-libraries-11-4 11.4.14-1 arm64 CUDA Libraries 11.4 meta-package
ii cuda-libraries-dev-11-4 11.4.14-1 arm64 CUDA Libraries 11.4 development meta-package
ii cuda-nvcc-11-4 11.4.166-1 arm64 CUDA nvcc
ii cuda-nvdisasm-11-4 11.4.167-1 arm64 CUDA disassembler
ii cuda-nvml-dev-11-4 11.4.167-1 arm64 NVML native dev links, headers
ii cuda-nvprof-11-4 11.4.166-1 arm64 CUDA Profiler tools
ii cuda-nvprune-11-4 11.4.167-1 arm64 CUDA nvprune
ii cuda-nvrtc-11-4 11.4.166-1 arm64 NVRTC native runtime libraries
ii cuda-nvrtc-dev-11-4 11.4.166-1 arm64 NVRTC native dev links, headers
ii cuda-nvtx-11-4 11.4.166-1 arm64 NVIDIA Tools Extension
ii cuda-samples-11-4 11.4.166-1 arm64 CUDA example applications
ii cuda-sanitizer-11-4 11.4.166-1 arm64 CUDA Sanitizer
ii cuda-toolkit-11-4 11.4.14-1 arm64 CUDA Toolkit 11.4 meta-package
ii cuda-toolkit-11-4-config-common 11.4.167-1 all Common config package for CUDA Toolkit 11.4.
ii cuda-toolkit-11-config-common 11.4.167-1 all Common config package for CUDA Toolkit 11.
ii cuda-toolkit-config-common 11.4.167-1 all Common config package for CUDA Toolkit.
ii cuda-tools-11-4 11.4.14-1 arm64 CUDA Tools meta-package
ii cuda-visual-tools-11-4 11.4.14-1 arm64 CUDA visual tools
ii graphsurgeon-tf 8.4.0-1+cuda11.4 arm64 GraphSurgeon for TensorRT package
ii libcudnn8 8.3.2.49-1+cuda11.4 arm64 cuDNN runtime libraries
ii libcudnn8-dev 8.3.2.49-1+cuda11.4 arm64 cuDNN development libraries and headers
ii libcudnn8-samples 8.3.2.49-1+cuda11.4 arm64 cuDNN samples
ii libnvinfer-bin 8.4.0-1+cuda11.4 arm64 TensorRT binaries
ii libnvinfer-dev 8.4.0-1+cuda11.4 arm64 TensorRT development libraries and headers
ii libnvinfer-doc 8.4.0-1+cuda11.4 all TensorRT documentation
ii libnvinfer-plugin-dev 8.4.0-1+cuda11.4 arm64 TensorRT plugin libraries
ii libnvinfer-plugin8 8.4.0-1+cuda11.4 arm64 TensorRT plugin libraries
ii libnvinfer-samples 8.4.0-1+cuda11.4 all TensorRT samples
ii libnvinfer8 8.4.0-1+cuda11.4 arm64 TensorRT runtime libraries
ii libnvonnxparsers-dev 8.4.0-1+cuda11.4 arm64 TensorRT ONNX libraries
ii libnvonnxparsers8 8.4.0-1+cuda11.4 arm64 TensorRT ONNX libraries
ii libnvparsers-dev 8.4.0-1+cuda11.4 arm64 TensorRT parsers libraries
ii libnvparsers8 8.4.0-1+cuda11.4 arm64 TensorRT parsers libraries
ii nvidia-cuda 5.0.1-b118 arm64 NVIDIA CUDA Meta Package
ii nvidia-l4t-cuda 34.1.1-20220516211757 arm64 NVIDIA CUDA Package
ii python3-libnvinfer 8.4.0-1+cuda11.4 arm64 Python 3 bindings for TensorRT
ii python3-libnvinfer-dev 8.4.0-1+cuda11.4 arm64 Python 3 development package for TensorRT
ii tensorrt 8.4.0.11-1+cuda11.4 arm64 Meta package of TensorRT
ii uff-converter-tf 8.4.0-1+cuda11.4 arm64 UFF converter for TensorRT package

I have successfully transfer-learned a yolov_4 model using tao-toolkit on a x86 desktop Ubuntu 22.04 machine, following this guide: YOLOv4 — TAO Toolkit 3.22.05 documentation

I have after that exported the model, using the same desktop x86 machine and ended up with a .etlt model file.

In order to use the model in a deepstream application on my Nvidia Orin target device, I have copied the .etlt to the Nvidia Orin, dowloaded the
tao-converter_vv3.22.05_trt8.4_aarch64 to the same Nvidia Orin device.
When running the converter I end up with a Segmentation fault.

/home/viktor/dev/tools/tao_converter/tao-converter_vv3.22.05_trt8.4_aarch64/tao-converter -k sib_yolo_box_label_220822 -p Input,1x3x800x1376,8x3x800x1376,16x3x800x1376 -d 3,384,384 -o BatchedNMS -e trt.engine -m 1 -t fp16 -i nchw $(pwd)/exported/yolov4_resnet18_epoch_090.etlt

I tried omitting all optional parameters to the tao-converter but still the same segmentation fault without any error message.

How should I proceed to generate a model engine that can be consumed by deepstream on Nvidia Orin?

2

Hi @viktor.nilsson Have already installed DeepStream on your Orin? Please be noted that current JP5.0.2GA doesn’t support DeepStream6.1 officially, please wait for the next DeepStream release in the coming weeks.

3

Hi Yingliu,
Yes, I have successfully been running a python deepstream app in a docker container which is based on nvcr.io/nvidia/deepstream-l4t:6.1-samples on the Orin.
Everything works fine and object detection works well using the sample model from nvidia located in:
/opt/nvidia/deepstream/deepstream-6.1/samples/models/Primary_Detector/

I would just like to switch the object detector model to my own. Shouldn’t that be possible without conflicting with deepstream since the tao-converter is only depending on the tensorRT version and device architecture?

Thanks!

/ Viktor

4

Could you share the full log?

How about
$ chmod +x /home/viktor/dev/tools/tao_converter/tao-converter_vv3.22.05_trt8.4_aarch64/tao-converter
and then
$ ./tao-converter -h

BTW, please wget below again. We confirm that there is no issue to use tao-converter.
$ wget --content-disposition ‘https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-converter/versions/v3.22.05_trt8.4_aarch64/files/tao-converter’

5

Hi Morganh,

After re-installing the tao-converter, it now works!
Thank you!

/ Viktor