How to export model using tlt-converter for Jetson Nano

Accelerated Computing Intelligent Video Analytics TAO Toolkit
41

Hi sdeep038,
The lib locates at out folder when you finish https://devtalk.nvidia.com/default/topic/1064407/transfer-learning-toolkit/how-to-export-model-using-tlt-converter-for-jetson-nano/post/5393964/#5393964

$ ls pwd/out

42

Hi Morgan,

by mistake i have removed libnvinfer.so.5 please help me.

i am facing following error.

import tensorrt
Traceback (most recent call last):
File “”, line 1, in
File “/usr/lib/python2.7/dist-packages/tensorrt/init.py”, line 1, in
from .tensorrt import *
ImportError: libnvinfer.so.5: cannot open shared object file: No such file or directory

whai i did wrong??

where its(libnvinfer.so.5) located?

43

Please check if libnvinfer.so.5.1.6 is still available.

nvidia@nvidia:~$ ll /usr/lib/aarch64-linux-gnu/libnvinfer.so*
lrwxrwxrwx 1 root root        19 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so -> libnvinfer.so.5.1.6
lrwxrwxrwx 1 root root        19 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so.5 -> libnvinfer.so.5.1.6
-rw-r--r-- 1 root root 150375912 6月   5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer.so.5.1.6
44

No Moragan,

All libnvinfer.so* removed.

this is output:

ll /usr/lib/aarch64-linux-gnu/libnvinfer_*
lrwxrwxrwx 1 root root 26 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so → libnvinfer_plugin.so.5.1.5
lrwxrwxrwx 1 root root 28 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5 → libnvinfer_plugin.so.5.1.5.0
-rw-r–r-- 1 root root 2619200 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5.0

what should i do?

45

Suggest that you install TRT with Jetpack again.

46

is there any hack like copy .so from other machine ?

47

Yes, you can copy .so from the same machine as long as it is built by the same version of Jetpack.

Notice: We did not want to replace libnvinfer.so. For this topic, we are talking about replacing libnvinfer_plugin.so.

48

I have confusion.
then which location libnvinfer_plugin.so* lib should be available?

either /usr/local/cuda-10.0/lib64 or /usr/lib/aarch64-linux-gnu/ ???

49

Hi morgan,

I have question.

when libnvinfer_plugin.so* and libnvinfer.so* created?

when we install trt with jetpack at that time libinfer.so* is created at /usr/lib/aarch64-linux-gnu
is it my correct understanding??

waiting for your reply…

Thanks

50

Yes, you’re right.
The original libnvinfer_plugin.so* locates at /usr/lib/aarch64-linux-gnu

And we’re talking about replacing libnvinfer_plugin.so* all the time. Please do not consider libnvinfer.so*.
See How to export model using tlt-converter for Jetson Nano - TAO Toolkit - NVIDIA Developer Forums.
The new libnvinfer_plugin.so* lib will be avaiable at pwd/out folder.
Replace the original libnvinfer_plugin.so* with the new compiled libnvinfer_plugin.so*

Please backup the original libnvinfer_plugin.so* in case you remove it by mistake.

51

Hi Morgan,
Thanks for your quick reply .

so what about /usr/local/cuda-10.0/lib64/ location,In this location libnvinfer.so.* and libnvinfer_plugin.so.* located or not?
In my case

ll /usr/local/cuda-10.0/lib64/libnvinfer*

lrwxrwxrwx 1 root root 26 Nov 7 20:45 /usr/local/cuda-10.0/lib64/libnvinfer_plugin.so → libnvinfer_plugin.so.5.1.5
lrwxrwxrwx 1 root root 28 Nov 7 20:45 /usr/local/cuda-10.0/lib64/libnvinfer_plugin.so.5.1.5 → libnvinfer_plugin.so.5.1.5.0
-rw-r–r-- 1 root root 2619200 Nov 7 20:45 /usr/local/cuda-10.0/lib64/libnvinfer_plugin.so.5.1.5.0
lrwxrwxrwx 1 root root 19 Nov 7 21:58 /usr/local/cuda-10.0/lib64/libnvinfer.so.5 → libnvinfer.so.5.1.5*
-rwxr-xr-x 1 root root 141788184 Nov 7 21:58 /usr/local/cuda-10.0/lib64/libnvinfer.so.5.1.5*

so i don’t know how libnvinder.so* and libnvinfer_plugin.so* its availble at location…

waiting for your reply…

Thanks,
Deep

52

By default, after installed via Jetpack, libnvinfer_plugin.so* are only available at /usr/lib/aarch64-linux-gnu/

53

Can i download only TRT with jetpack bcz through jetpack i need to downlaod other stuff like ooencv compulsory…

Thanks.

54

Sorry, ignore my comment.

55

Hi,

somehow i copied libnvinfer.so.5.1.6 to /usr/lib/aarch64-linux-gnu/ but when i run sample app like,
./deepstream-test1-app …/…/…/…/…/samples/streams/sample_720p.h264
i got following error.

One element could not be created. Exiting.

56

Hi Sdeep,
Please file a new topic to describe your issue and paste the full log along with command and config file.
Thanks.

57

Hi Morgan,

how should be /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so* look like?

I have

ls -l /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so*
lrwxrwxrwx 1 root root 26 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so → libnvinfer_plugin.so.5.1.5
lrwxrwxrwx 1 root root 26 Jun 5 01:22 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5 → libnvinfer_plugin.so.5.1.6
lrwxrwxrwx 1 root root 28 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5 → libnvinfer_plugin.so.5.1.5.0
-rw-r–r-- 1 root root 2619200 Nov 7 19:45 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5.0
-rw-r–r-- 1 root root 2619200 Nov 8 19:43 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6

is it correct?

58

I have copied sudo cp out/libnvinfer_plugin.so.5.1.5 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6 .

59

Below is an example how to replace libnvinfer_plugin.so.

$ md5sum /home/nvidia/trt-oss/TensorRT/build/out/libnvinfer_plugin.so.5.1.5
237f90d6d666fa8c6e7bc25f47236ad0 /home/nvidia/trt-oss/TensorRT/build/out/libnvinfer_plugin.so.5.1.5

$  sudo cp /home/nvidia/trt-oss/TensorRT/build/out/libnvinfer_plugin.so.5.1.5    /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6


$ md5sum /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6
237f90d6d666fa8c6e7bc25f47236ad0 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6

$ ll /usr/lib/aarch64-linux-gnu/libnvinfer_plugin*

lrwxrwxrwx 1 root root 26 6月 5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so -> libnvinfer_plugin.so.5.1.6
lrwxrwxrwx 1 root root 26 6月 5 03:52 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5 -> libnvinfer_plugin.so.5.1.6
lrwxrwxrwx 1 root root 26 9月 25 18:15 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.5 -> libnvinfer_plugin.so.5.1.6
-rw-r--r-- 1 root root 2619200 9月 19 17:58 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.5.1.6
60

Thanks morgan for quick reply,
but still i m facing isse…

./deepstream-custom pgie_frcnn_uff_config.txt sample_720p.h264
Now playing: pgie_frcnn_uff_config.txt

Using winsys: x11
Opening in BLOCKING MODE
Creating LL OSD context new
0:00:00.621955738 26108 0x55c9056640 INFO nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger: NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
0:00:00.940023898 26108 0x55c9056640 ERROR nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger: NvDsInferContext[UID 1]:log(): UffParser: Could not read buffer.
NvDsInferCudaEngineGetFromTltModel: Failed to parse UFF model
0:00:00.940728252 26108 0x55c9056640 ERROR nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger: NvDsInferContext[UID 1]:generateTRTModel(): Failed to create network using custom network creation function
0:00:00.940823072 26108 0x55c9056640 ERROR nvinfer gstnvinfer.cpp:511:gst_nvinfer_logger: NvDsInferContext[UID 1]:initialize(): Failed to create engine from model files
0:00:00.941257173 26108 0x55c9056640 WARN nvinfer gstnvinfer.cpp:692:gst_nvinfer_start: error: Failed to create NvDsInferContext instance
0:00:00.941284726 26108 0x55c9056640 WARN nvinfer gstnvinfer.cpp:692:gst_nvinfer_start: error: Config file path: pgie_frcnn_uff_config.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
Running…
ERROR from element primary-nvinference-engine: Failed to create NvDsInferContext instance
Error details: gstnvinfer.cpp(692): gst_nvinfer_start (): /GstPipeline:ds-custom-pipeline/GstNvInfer:primary-nvinference-engine:
Config file path: pgie_frcnn_uff_config.txt, NvDsInfer Error: NVDSINFER_CUSTOM_LIB_FAILED
Returned, stopping playback
Deleting pipeline

Thanks,
Deep