I have trained a model(Resnet) using my own dataset using the Transfer Learning toolkit. I want to use the model for inference from a video and camera using the Driveworks SDK(sample_object_detector). Running the sample_object_detector with the trt file generated from the tlt-converter i am getting the following error:- The engine plan file is not compatible with this version of TensorRT, expecting library version 5.1.4 got 5.1.5, please rebuild." this error which i will fix.
What would be a possible fix to this problem? Or my approach of deploying wrong here?
I will run the inference on a DRIVE AGX Xavier which i believe is jetson. The version of CUDA is as follows:-
ii cuda-command-line-tools-10-2 10.2.19-1 arm64 CUDA command-line tools
ii cuda-compiler-10-2 10.2.19-1 arm64 CUDA compiler
ii cuda-cudart-10-2 10.2.19-1 arm64 CUDA Runtime native Libraries
ii cuda-cudart-dev-10-2 10.2.19-1 arm64 CUDA Runtime native dev links, headers
ii cuda-cufft-10-2 10.2.19-1 arm64 CUFFT native runtime libraries
ii cuda-cuobjdump-10-2 10.2.19-1 arm64 CUDA cuobjdump
ii cuda-cupti-10-2 10.2.19-1 arm64 CUDA profiling tools runtime libs.
ii cuda-curand-10-2 10.2.19-1 arm64 CURAND native runtime libraries
ii cuda-cusolver-10-2 10.2.19-1 arm64 CUDA solver native runtime libraries
ii cuda-cusparse-10-2 10.2.19-1 arm64 CUSPARSE native runtime libraries
ii cuda-driver-dev-10-2 10.2.19-1 arm64 CUDA Driver native dev stub library
ii cuda-gdb-10-2 10.2.19-1 arm64 CUDA-GDB
ii cuda-gpu-library-advisor-10-2 10.2.19-1 arm64 CUDA GPU Library Advisor.
ii cuda-license-10-2 10.2.19-1 arm64 CUDA licenses
ii cuda-memcheck-10-2 10.2.19-1 arm64 CUDA-MEMCHECK
ii cuda-misc-headers-10-2 10.2.19-1 arm64 CUDA miscellaneous headers
ii cuda-npp-10-2 10.2.19-1 arm64 NPP native runtime libraries
ii cuda-nvcc-10-2 10.2.19-1 arm64 CUDA nvcc
ii cuda-nvdisasm-10-2 10.2.19-1 arm64 CUDA disassembler
ii cuda-nvgraph-10-2 10.2.19-1 arm64 NVGRAPH native runtime libraries
ii cuda-nvprof-10-2 10.2.19-1 arm64 CUDA Profiler tools
ii cuda-nvprune-10-2 10.2.19-1 arm64 CUDA nvprune
ii cuda-nvrtc-10-2 10.2.19-1 arm64 NVRTC native runtime libraries
ii cuda-nvrtc-dev-10-2 10.2.19-1 arm64 NVRTC native dev links, headers
ii cuda-nvtx-10-2 10.2.19-1 arm64 NVIDIA Tools Extension
ii cuda-repo-ubuntu1804-10-2-local-10.2.19 1.0-1 arm64 cuda repository configuration files
I used the tlt-converter provided with the tlt docker on the same machine i trained the model in.
These came preinstalled with the AGX and I found the version on the documentation.
I tried running tlt-converter on the AGX but I am getting the following error:-
./tlt-converter: error while loading shared libraries: libnvinfer.so.7: cannot open shared object file: No such file or directory
Moreover I have looked for the library libnvinfer.so.7, which the AGX does not have, however it has libnvinfer.so.5. Currently the environment variable TRT_LIB_PATH is set to the destination where the libnvinfer.so.5 library is.
./tlt-converter ./resnet18_detector.etlt -k #mykey -c ./calibration.bin -o output_cov/Sigmoid,output_bbox/BiasAdd -d 3,384,1248 -i nchw -m 64 -t int8 -e ./resnet.trt -b 4
[ERROR] UffParser: Unsupported number of graph 0
[ERROR] Failed to parse uff model
[ERROR] Network must have at least one output
[ERROR] Unable to create engine
Segmentation fault (core dumped)
Hi SKoyal,
If you were using Jetpack 4.4 to install cuda/trt/cudnn into xavier, the cuda version will be 10.2.89, cudnn version will be 8.0, trt version will be 7.1.
Due to the old cudnn/trt version, I suggest you run with 1.0.1 tlt docker instead of 2.0_dp tlt docker.
tlt-converter worked. It was the key issue. Thank you.
However, I still would want to use the model in Driveworks SDK using ./sample_object_detector_tracker. When I am using the ./sample_object_detector_tracker on the generated TRT it is giving me the following error:-
./sample_object_detector_tracker --input-type=video --video=/home/nvidia/sk/test1.h264 --tensorRT_model=/home/nvidia/sk/tlt-converter-trt5.1/resnet.trt
[20-05-2020 10:55:59] Platform: Detected DDPX - Tegra A
[20-05-2020 10:55:59] TimeSource: monotonic epoch time offset is 1589956455391598
[20-05-2020 10:55:59] PTP Time is available from NVPPS Driver
[20-05-2020 10:55:59] Platform: number of GPU devices detected 1
[20-05-2020 10:55:59] Platform: currently selected GPU device integrated ID 0
[20-05-2020 10:55:59] SDK: Resources mounted from .././data/
[20-05-2020 10:55:59] SDK: Create NvMediaDevice
[20-05-2020 10:55:59] SDK: use EGL display as provided
[20-05-2020 10:56:00] TimeSource: monotonic epoch time offset is 1589956455391598
[20-05-2020 10:56:00] PTP Time is available from NVPPS Driver
[20-05-2020 10:56:00] Initialize DriveWorks SDK v2.2.3136
[20-05-2020 10:56:00] Release build with GNU 7.3.1 from heads/buildbrain-branch-0-gca7b4b26e65 against Drive PDK v5.1.6.1
[20-05-2020 10:56:00] Initialize DriveWorks VisualizationSDK v2.2.3136
[20-05-2020 10:56:00] Initialize DriveWorksGL SDK v2.2.3136
[20-05-2020 10:56:00] SensorFactory::createSensor() -> camera.virtual, camera-group=a,camera-index=0,camera-type=ar0231-rccb-bae-sf3324,input-type=video,offscreen=0,profiling=1,slave=0,tensorRT_model=/home/nvidia/sk/tlt-converter-trt5.1/resnet.trt,video=/home/nvidia/sk/test1.h264
[20-05-2020 10:56:00] CameraVirtualNvMedia: no seek table found at /home/nvidia/sk/test1.h264.seek, seeking is not available.
SimpleCamera: Camera image: 1920x1080
Camera image with 1920x1080 at 60 FPS
[20-05-2020 10:56:00] StreamConsumerGL: successfully initialized
[20-05-2020 10:56:00] StreamProducerCUDA: successfully initialized
[20-05-2020 10:56:00] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[20-05-2020 10:56:00] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[20-05-2020 10:56:00] Glob Size is 2176512 bytes.
[20-05-2020 10:56:00] Added linear block of size 122683392
[20-05-2020 10:56:00] Added linear block of size 122683392
[20-05-2020 10:56:00] Added linear block of size 30670848
[20-05-2020 10:56:05] Deserialize required 5020426 microseconds.
[20-05-2020 10:56:05] DNN: Metadata json file could not be found. Metadata has been filled with default values. Please place <network_filename>.json in the same directory as the network file if custom metadata is needed.
[20-05-2020 10:56:05] Driveworks exception thrown: DW_INVALID_ARGUMENT: blobName not found
terminate called after throwing an instance of 'std::runtime_error'
what(): [2020-05-20 10:56:05] DW Error DW_INVALID_ARGUMENT executing DW function:
dwDNN_getOutputIndex(&m_cvgIdx, coverageBlobName, m_dnn)
at /dvs/git/dirty/gitlab-master_av/dw/sdk/samples/dnn/sample_object_detector_tracker/main.cpp:316
Aborted (core dumped)
Checking the documentation for TensorRT Optimizer Tool it is mentioned that “The NVIDIA® DriveWorks TensorRT Optimizer Tool enables optimization for a given Caffe, UFF or ONNX model using TensorRT.”. The tlt does not produce either of the three model types. Is there a converter that can convert my trt to either of these model types?
I am trying to avoid using deepstream because I do not want to upgrade the dependencies or flash my AGX during the installation of the deepstream as I other teams with existing dependencies are working on it and their work may get affected.