Error in nvOCDR DeepStream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Hardware Platform (Jetson / GPU) : Jetson Orin Nano Dev kit
• DeepStream Version : 7.1
• JetPack Version (valid for Jetson only) : 6.1(rev1)
• TensorRT Version :10.3

I am working on nvOCD solution on Jetson Orin Nano Dev kit but I am getting below error.

gst-launch-1.0 filesrc location=/home/rptech/projects/ocdr/onnx_models/NVIDIA-Optical-Character-Detection-and-Recognition-Solution/c++_samples/test_img/scene_text.jpg ! jpegparse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! nvdsvideotemplate customlib-name=/home/rptech/projects/ocdr/onnx_models/NVIDIA-Optical-Character-Detection-and-Recognition-Solution/libnvocdr.so customlib-props=“ocdnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine” customlib-props=“ocdnet-input-shape:3,736,1280” customlib-props=“ocdnet-binarize-threshold:0.1” customlib-props=“ocdnet-polygon-threshold:0.3” customlib-props=“ocdnet-max-candidate:200” customlib-props=“ocrnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine” customlib-props=“ocrnet-dict-path:/home/rptech/projects/ocdr/onnx_models/character_list.txt” customlib-props=“ocrnet-input-shape:1,32,100” ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! ‘video/x-raw,format=I420’ ! jpegenc ! jpegparse ! filesink location=/home/rptech/projects/ocdr/onnx_models/NVIDIA-Optical-Character-Detection-and-Recognition-Solution/c++_samples/test_img/output.jpg
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
ERROR: from element /GstPipeline:pipeline0/GstNvDsVideoTemplate:nvdsvideotemplate0: Runtime error: createCustomAlgoCtx function not found in library
Additional debug info:
Exception occurred
ERROR: pipeline doesn’t want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL …
Freeing pipeline …

Please suggest on this.

3

For DeepStream sample, please refer to deepstream_tao_apps/apps/tao_others/deepstream-nvocdr-app at master · NVIDIA-AI-IOT/deepstream_tao_apps

4

Hi,

I am referring to same link for this project. But the thing is i am doing it on-prem not in container.

5

You don’t need to do it in container. What you have to do is to install the DeepStream SDK according to Installation — DeepStream documentation

6

already installed via sdk manager on the jetson.

7

The “customlib-name=/home/rptech/projects/ocdr/onnx_models/NVIDIA-Optical-Character-Detection-and-Recognition-Solution/libnvocdr.so” is wrong, it should be the “deepstream/libnvocdr_impl.so”.

8

Hi,

It is mentioned to compile in our directory in the same repo itself. Please find the attachment FYR.

libocdr
libocdr882×314 18.4 KB

And even I searched in /opt/nvidia/deepstream/deepstream-7.1/sources/gst-plugins for libnvocdr_impl.so but I could not find it there.

Looking forward for your suggestions.

9

You are running with DeepStream but not the sample in NVIDIA-AI-IOT/NVIDIA-Optical-Character-Detection-and-Recognition-Solution: This repository provides optical character detection and recognition solution optimized on Nvidia devices.. Why don’t you follow the DeepStream instruction? deepstream_tao_apps/apps/tao_others/deepstream-nvocdr-app at master · NVIDIA-AI-IOT/deepstream_tao_apps

10

Hi,

Thanks for your suggestions.I am following the same. now there is a new challenge. No errors are generating but it is stuck in between. attaching the log fyr.

gst-launch-1.0 filesrc location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/scene_text.jpg ! jpegparse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! nvdsvideotemplate customlib-name=/home/rptech/projects/ocdr/onnx_models/repo/deepstream/libnvocdr_impl.so customlib-props=“ocdnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine” customlib-props=“ocdnet-input-shape:3,736,1280” customlib-props=“ocdnet-binarize-threshold:0.1” customlib-props=“ocdnet-polygon-threshold:0.3” customlib-props=“ocdnet-max-candidate:200” customlib-props=“ocrnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine” customlib-props=“ocrnet-dict-path:home/rptech/projects/ocdr/onnx_models/character_list.txt” customlib-props=“ocrnet-input-shape:1,32,100” ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! ‘video/x-raw,format=I420’ ! jpegenc ! jpegparse ! filesink location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/output.jpg
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=3,736,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=home/rptech/projects/ocdr/onnx_models/character_list.txt
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,32,100
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.705: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.706: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.707: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.707: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.734: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.734: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.757: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.757: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.758: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.758: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink
nvstreammux: Successfully handled EOS for source_id=0

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.758: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.758: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.759: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.759: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.759: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.759: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.759: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.759: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:6384): GStreamer-CRITICAL **: 10:39:17.760: gst_caps_set_simple: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:6384): GStreamer-Base-CRITICAL **: 10:39:17.760: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

11

Please refer to the following pipeline, I’m using the latest models with DeepStream 7.1, you can just refer to the pipeline itself.

gst-launch-1.0 filesrc location=/home/xxxx/in.jpg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! 'video/x-raw(memory:NVMM),format=NV12' ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! nvdsvideotemplate customlib-name=/home/xxxx/libnvocdr_impl.so customlib-props="ocdnet-engine-path:/home/xxxx/ocdnet.fp16.engine" customlib-props="ocdnet-input-shape:3,736,1280" customlib-props="ocdnet-binarize-threshold:0.1" customlib-props="ocdnet-polygon-threshold:0.3" customlib-props="ocdnet-max-candidate:200" customlib-props="ocrnet-engine-path:/home/xxxx/ocrnet.fp16.engine" customlib-props="ocrnet-dict-path:/home/xxxx/character_list" customlib-props="ocrnet-input-shape:1,64,200" customlib-props="ocrnet-decode:Attention" ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvdsosd ! nvvideoconvert ! 'video/x-raw,format=I420' ! jpegenc ! jpegparse ! filesink location=/home/xxxx/output.jpg
12

Hi,

followed your suggestions. but no luck. getting error in reading the OCRnet dict file. command and log is attached below.

gst-launch-1.0 filesrc location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/scene_text.jpg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! ‘video/x-raw(memory:NVMM),format=NV12’ ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! nvdsvideotemplate customlib-name=/home/rptech/projects/ocdr/onnx_models/repo/deepstream/libnvocdr_impl.so customlib-props=“ocdnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine” customlib-props=“ocdnet-input-shape:3,736,1280” customlib-props=“ocdnet-binarize-threshold:0.1” customlib-props=“ocdnet-polygon-threshold:0.3” customlib-props=“ocdnet-max-candidate:200” customlib-props=“ocrnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine” customlib-props=“ocrnet-dict-path:home/rptech/projects/ocdr/onnx_models/character_list.txt” customlib-props=“ocrnet-input-shape:1,64,200” customlib-props=“ocrnet-decode:Attention” ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvdsosd ! nvvideoconvert ! ‘video/x-raw,format=I420’ ! jpegenc ! jpegparse ! filesink location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/output.jpg
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=3,736,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=home/rptech/projects/ocdr/onnx_models/character_list.txt
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,64,200
Inside Custom Lib : Setting Prop Key=ocrnet-decode Value=Attention
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[ERROR] Error reading OCRNet dict file home/rptech/projects/ocdr/onnx_models/character_list.txt

13

I’m using different models. Please replace this part with your original configurations.

14

Hi,

what does original configurations mean? I am downloading models from NGC only which is mentioned in TAO documentation(Deploying nvOCDR to DeepStream - NVIDIA Docs).

15

Use your configuration.

16

Sorry!

I am not understanding what you actually mean by “your configuration” because i am using my files/directories in the above command provided by you.

17

My nvdsvideotemplate configuration is different to your nvdsvideotemplate configuration

This is yours:

This is mine:

This is because I used different OCD/OCR models. Please use your configuration with your models.

18

Hi,

I dont have any custom model right now. I am using the same pre-trained model model(downloaded from the link mentioned in the repo) and compiling(for trt model) them with the same name that are mentioned in the repo.

Intially, i want to benchmark on my jetson and then i will customise as per my requirements.

Please suggest!

19

What is your issue? Your pipeline can work after adding nvvideoconvert and capsfilter after the nvv4l2decoder.

20

Hi,

I tried but this time there is some different error. Logs are as follows

gst-launch-1.0 filesrc location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/scene_text.jpg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! capsfilter ! ‘video/x-raw(memory:NVMM),format=NV12’ ! m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! nvdsvideotemplate customlib-name=/home/rptech/projects/ocdr/onnx_models/repo/deepstream/libnvocdr_impl.so customlib-props=“ocdnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine” customlib-props=“ocdnet-input-shape:3,736,1280” customlib-props=“ocdnet-binarize-threshold:0.1” customlib-props=“ocdnet-polygon-threshold:0.3” customlib-props=“ocdnet-max-candidate:200” customlib-props=“ocrnet-engine-path:/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine” customlib-props=“ocrnet-dict-path:/home/rptech/projects/ocdr/onnx_models/character_list” customlib-props=“ocrnet-input-shape:1,64,200” customlib-props=“ocrnet-decode:Attention” ! nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvdsosd ! nvvideoconvert ! ‘video/x-raw,format=I420’ ! jpegenc ! jpegparse ! filesink location=/home/rptech/projects/ocdr/onnx_models/repo/c++_samples/test_img/output.jpg
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=3,736,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=/home/rptech/projects/ocdr/onnx_models/ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=/home/rptech/projects/ocdr/onnx_models/character_list
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,64,200
Inside Custom Lib : Setting Prop Key=ocrnet-decode Value=Attention
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 277
NvMMLiteBlockCreate : Block : BlockType = 277
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
nvstreammux: Successfully handled EOS for source_id=0
Line 93: ‘’ failed: an illegal memory access was encountered
Aborted (core dumped)

Please suggest!

21

I think your model shape is customlib-props=“ocrnet-input-shape:1,32,100” as what you post in the Error in nvOCDR DeepStream