Issue while Feeding Pytorch TRT 2D-U-net model into Deepstream 6

Accelerated Computing Intelligent Video Analytics TAO Toolkit
, ,
1

Hello,

We have trained our Pytorch Unet model and converted that into TRT using the torch2trt but while feeding that into the Deepstream- 6 we are facing the errors logged below

vm:~$ deepstream-segmentation-app -t infer /home/elio_admin/Downloads/deepstream_tao_apps/configs/unet_tao/pgie_unet_tao_config.txt *************.h264
Now playing: /***********3969791.h264,
ERROR: [TRT]: 1: [stdArchiveReader.cpp::StdArchiveReader::29] Error Code 1: Serialization (Serialization assertion magicTagRead == magicTag failed.Magic tag does not match)
ERROR: [TRT]: 4: [runtime.cpp::deserializeCudaEngine::76] Error Code 4: Internal Error (Engine deserialization failed.)
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:1528 Deserialize engine failed from file: /home/elio_admin/Downloads/deepstream_tao_apps/models/unet/CD_CD_ICG_100_swiched.trt
0:00:00.501323373  5912 0x5598ad991520 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/home/elio_admin/Downloads/deepstream_tao_apps/models/unet/CD_CD_ICG_100_swiched.trt failed
0:00:00.501387106  5912 0x5598ad991520 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/home/elio_admin/Downloads/deepstream_tao_apps/models/unet/CD_CD_ICG_100_swiched.trt failed, try rebuild
0:00:00.501400602  5912 0x5598ad991520 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:861 failed to build network since there is no model file matched.
ERROR: ../nvdsinfer/nvdsinfer_model_builder.cpp:799 failed to build network.
0:00:00.501915044  5912 0x5598ad991520 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1934> [UID = 1]: build engine file failed
0:00:00.501935112  5912 0x5598ad991520 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2020> [UID = 1]: build backend context failed
0:00:00.501945933  5912 0x5598ad991520 ERROR                nvinfer gstnvinfer.cpp:632:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1257> [UID = 1]: generate backend failed, check config file settings
0:00:00.501976872  5912 0x5598ad991520 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary-nvinference-engine> error: Failed to create NvDsInferContext instance
0:00:00.501984748  5912 0x5598ad991520 WARN                 nvinfer gstnvinfer.cpp:841:gst_nvinfer_start:<primary-nvinference-engine> error: Config file path: /home/elio_admin/Downloads/deepstream_tao_apps/configs/unet_tao/pgie_unet_tao_config.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
Running...
ERROR from element primary-nvinference-engine: Failed to create NvDsInferContext instance
Error details: gstnvinfer.cpp(841): gst_nvinfer_start (): /GstPipeline:dstest-image-decode-pipeline/GstNvInfer:primary-nvinference-engine:
Config file path: /home/elio_admin/Downloads/deepstream_tao_apps/configs/unet_tao/pgie_unet_tao_config.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
Returned, stopping playback
Deleting pipeline

We have converted our model into trt on same machine as deepstream execution server, having same TensorRT version using the snippet below

    import torch2trt
    x = torch.ones((1, 3, 320, 480)).cuda().half()
    model = SegModel(n_classes=3)
    checkpoint = args.model_file
    a = torch.load(checkpoint)
    model.load_state_dict(a['state_dict'], strict=False)
    model1 = model.cuda().eval().half()
    model_w = model1.half()
    #args.tensorrt_file: 'alexnet_trt.pth'
    model_trt = torch2trt.torch2trt(model_w, [torch.tensor(x, dtype=torch.half, device='cuda:0')], fp16_mode=True)
    torch.save(model_trt.state_dict(), args.tensorrt_file)

Any help and suggestion would be appreciated
Thanks in Advance !!!

** Hardware Platform : NVIDIA RTX A6000
** DeepStream Version:
deepstream-app version 6.0.1, DeepStreamSDK 6.0.1, CUDA Driver Version: 11.6, CUDA Runtime Version: 11.4, TensorRT Version: 8.4, cuDNN Version: 8.4, libNVWarp360 Version: 2.0.1d3
** NVIDIA GPU Driver Version : Driver Version: 510.60.02

4

Make sure the trt version is the same between generating trt engine and running deepstream-segmentation-app.

5

Moved to TAO forum.

6

@Morganh I indeed came across a similar kind of suggestion on one of posts therefore made sure where we are converting the model and using deepstream have the same trt version.

7

Need to make sure even minor version.
For example, in a machine where runs deepstream-segmentation-app.
$ dpkg -l |grep cuda
…
ii tensorrt 8.0.1.6-1+cuda11.3 amd64 Meta package of TensorRT

Need to use the same 8.0.1.6 to generate trt engine.

8

I tried getting same version for both now it seems like i am getting new error

(pytorch) nkumar@u117282:~/deepstream_trt$ deepstream-segmentation-app -t infer deepstream_tao_apps/configs/unet_tao/pgie_unet_tao_config.txt deepstream_tao_apps/***********67.11704647123835-81.03094573969791.h264 

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.186: External plugin loader failed. This most likely means that the plugin loader helper binary was not found or could not be run. You might need to set the GST_PLUGIN_SCANNER environment variable if your setup is unusual. This should normally not be required though.

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.186: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.187: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_nvblender.so': libgstbadvideo-1.0.so.0: cannot open shared object file: No such file or directory

(deepstream-segmentation-app:4023651): GStreamer-WARNING **: 10:20:01.187: Failed to load plugin '/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
One element could not be created. Exiting.
Failed to create source bin. Exiting.

My configuration file looks like :

[property]

labelfile-path=unet_labels.txt
model-engine-file=../../models/unet/**_**_ICG_100_2switched.trt_ew
#CD_CD_ICG_100_swiched.trt
#CA_CD_ICGGrayscale.trt
#engine-create-func-name=NvDsInferYoloCudaEngineGet


#tlt-encoded-model=../../models/unet/CA_CD.etlt

net-scale-factor=0.00784313725490196
offsets=127.5;127.5;127.5
infer-dims=3;512;512
#tlt-model-key=CA_CD_ICG
network-type=2
num-detected-classes=3
model-color-format=1
segmentation-threshold=0.0
output-blob-names=softmax_1
segmentation-output-order=1
gie-unique-id=1
9

Please create a new topic in Deepstream forum for this error. This is not related to TAO. So, please create a deepstream forum topic for better help. Thanks.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.