Setting Up nvOCDR example on Orin Nano

I am running Jetpack 6.0, Deepstream 6.4 on a Orin Dev Kit and am trying to get the nvOCDR example running using these instructions

I’m running in a nvcr.io/nvidia/deepstream:6.4-triton-multiarch container, used this start command

docker run --runtime nvidia --name testOCR -it --privileged --net=host nvcr.io/nvidia/deepstream:6.4-triton-multiarch bash

and followed all the instructions for a Jetson device, as far as I can tell with no errors

When I run this basic pipeline I get a good udp stream at my receiving PC

GST_DEBUG=3 gst-launch-1.0 \
v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, framerate=30/1 ! \
nvvideoconvert ! \
x264enc tune=zerolatency speed-preset=veryfast ! \
h264parse ! rtph264pay config-interval=1 pt=126 ! \
udpsink host=192.168.8.179 port=10101

When I add the nvstreammux and nvmultistreamtiler elements, I get a UDP stream, but its a blank green screen

GST_DEBUG=3 gst-launch-1.0 \
v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, framerate=30/1 ! \
nvvideoconvert ! \
m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! \
nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! \
x264enc tune=zerolatency speed-preset=veryfast ! \
h264parse ! rtph264pay config-interval=1 pt=126 ! \
udpsink host=192.168.8.179 port=10101

And when I try to do the full example pipeline, the pipeline freezes with one CPU maxed out but otherwise nothing seeming to happen on the device

Here is the pipeline

GST_DEBUG=3 gst-launch-1.0 \
v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, framerate=30/1 ! \
nvvideoconvert ! \
m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! \
nvdsvideotemplate customlib-name=./libnvocdr_impl.so \
customlib-props="ocdnet-engine-path:./ocdnet.fp16.engine" \
customlib-props="ocdnet-input-shape:1,720,1280" \
customlib-props="ocdnet-binarize-threshold:0.1" \
customlib-props="ocdnet-polygon-threshold:0.3" \
customlib-props="ocdnet-max-candidate:200" \
customlib-props="ocrnet-engine-path:./ocrnet.fp16.engine" \
customlib-props="ocrnet-dict-path:./character_list.txt" \
customlib-props="ocrnet-input-shape:1,32,100" ! \
nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! \
x264enc tune=zerolatency speed-preset=veryfast ! \
h264parse ! rtph264pay config-interval=1 pt=126 ! \
udpsink host=192.168.8.179 port=10101

And here are the logs

root@ubuntu:~/testFiles# GST_DEBUG=3 gst-launch-1.0 \
v4l2src device=/dev/video0 ! video/x-raw, width=1280, height=720, framerate=30/1 ! \
nvvideoconvert ! \
m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=720 ! \
nvdsvideotemplate customlib-name=./libnvocdr_impl.so \
customlib-props="ocdnet-engine-path:./ocdnet.fp16.engine" \
customlib-props="ocdnet-input-shape:1,720,1280" \
customlib-props="ocdnet-binarize-threshold:0.1" \
customlib-props="ocdnet-polygon-threshold:0.3" \
customlib-props="ocdnet-max-candidate:200" \
customlib-props="ocrnet-engine-path:./ocrnet.fp16.engine" \
customlib-props="ocrnet-dict-path:./character_list.txt" \
customlib-props="ocrnet-input-shape:1,32,100" ! \
nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! \
x264enc tune=zerolatency speed-preset=veryfast ! \
h264parse ! rtph264pay config-interval=1 pt=126 ! \
udpsink host=192.168.8.179 port=10101
Setting pipeline to PAUSED ...
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=./ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=1,720,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=./ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=./character_list.txt
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,32,100
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
Redistribute latency...
0:00:00.980723116   367 0xaaab092c3360 WARN          v4l2bufferpool gstv4l2bufferpool.c:842:gst_v4l2_buffer_pool_start:<v4l2src0:pool0:src> Uncertain or not enough buffers, enabling copy threshold

After a bit of waiting I then get this log, and the pipeline dies

CUDA runtime error an illegal memory access was encountered at line 763 in file nvocdrlib_impl.cpp

I am running this in a directory which contains these files

root@ubuntu:~/test# ls
character_list.txt  libnvocdr_impl.so  ocdnet.fp16.engine  ocrnet.fp16.engine  output.mp4  vid.mp4

Why is it showing a green screen with the deep stream elements and then freezing/dying with the ocr element?

Note: I tried this with Jetpack 5.1.2 and a Deepstream 6.2 container with completely different results (though still not functional). What container / environment was this example supposed to run on?

You can try this sample. It is based on DS-6.4.

Is it supposed to run after following the setup instructions? When I try to run after going through all the steps i get these logs, and the pipeline fails

root@ubuntu:~/deepstream_tao_apps-master/apps/tao_others/deepstream-nvocdr-app# ./deepstream-nvocdr-app nvocdr_app_config.yml

(deepstream-nvocdr-app:538): GStreamer-WARNING **: 19:56:05.814: Trying to link elements nvvid-converter1 and nvvideo-caps that don't share a common ancestor: nvvideo-caps hasn't been added to a bin or pipeline, and nvvid-converter1 is in nvocdr-pipeline

(deepstream-nvocdr-app:538): GStreamer-WARNING **: 19:56:05.814: Trying to link elements nvvid-converter1 and nvvideo-caps that don't share a common ancestor: nvvideo-caps hasn't been added to a bin or pipeline, and nvvid-converter1 is in nvocdr-pipeline
OSD and sink elements link failure.
root@ubuntu:~/deepstream_tao_apps-master/apps/tao_others/deepstream-nvocdr-app#

Im running in the same DS 6.4 container

The error seems to indicate that the bug is in the .cpp demo app. Can someone confirm the app works as is on their system?

If I try to use the gst-launch script from the previous example (but with the libraries from the new example) I get the same logs as I got in my first post. Can someone confirm if they can get the new example working with gst-launch?

Or can anyone confirm if they are able to run either this example

Or this example

Without modifying source code? I cant get them working but I am hesitant to start messing with the code if the examples are working for others.

Have you run it successfully so far?

Have you tried modifying the code or configuration files? You may need to modify the source-list in nvocdr_app_config.yml to your file

source-list:
   #list: file:///workspace/nvocdr/img_0.jpg;file:///workspace/nvocdr/img_1.jpg
   list: file:///workspace/nvocdr/pexels-george-morina-5266796.mp4

I have not been able to get it running. I have modified the configuration file, but have not yet started modifying source code. Are you able to get it running by only modifying configuration files?

Yes, please follow the README. We tested on DS-6.4 and it works fine.

Ok, making some progress

First, the code uses nvv4l2h264enc, but the Orin Nano does not have the hardware decoder, so I switched the output to fakesink (option 2 in the config file)

Now the code seems to be running, but I am getting this error

 Now playing!
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=../../../models/nvocdr/ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=3,736,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=../../../models/nvocdr/ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=character_list.txt
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,32,100
Decodebin child added: source
Decodebin child added: decodebin0
Running...
In cb_newpad
###Decodebin did not pick nvidia decoder plugin.
Error reading serialized TensorRT engine: ../../../models/nvocdr/ocdnet.fp16.engine
terminate called after throwing an instance of 'std::length_error'
  what():  cannot create std::vector larger than max_size()
Aborted (core dumped)

With this config file

source-list:
   list:  file:///home/beauceron/Workspace/deepstream_tao_apps/apps/tao_others/deepstream-nvocdr-app/pexels-george-morina-5266796.mp4

output:
  ## 1:file ouput  2:fake output 3:eglsink output
  type: 2
  ## 0: H264 encoder  1:H265 encoder
  enc: 0
  bitrate: 2000000
  ##The file name without suffix
  filename: test

streammux:
  width: 1280
  height: 720
  batched-push-timeout: 40000

video-template:
  #x86
  #customlib-name: nvocdr_libs/x86/libnvocdr_impl.so
  #aarch64
  customlib-name: nvocdr_libs/aarch64/libnvocdr_impl.so
  customlib-props:
    - ocdnet-engine-path:../../../models/nvocdr/ocdnet.fp16.engine
    - ocdnet-input-shape:3,736,1280
    - ocdnet-binarize-threshold:0.1
    - ocdnet-polygon-threshold:0.3
    - ocdnet-max-candidate:200
    - ocrnet-engine-path:../../../models/nvocdr/ocrnet.fp16.engine
    - ocrnet-dict-path:character_list.txt
    - ocrnet-input-shape:1,32,100

Going back to running in a container, just to make sure the environment is correct.

I ran this command to start the container

docker run -it --net=host --runtime nvidia  -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.4 -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream:6.4-triton-multiarch

I then ran through all the instructions here

And I changed the config file to this (the test.mp4 file exists)

source-list:
   #list: file:///workspace/nvocdr/img_0.jpg;file:///workspace/nvocdr/img_1.jpg
   list: file:///root/test.mp4

output:
  ## 1:file ouput  2:fake output 3:eglsink output
  type: 2
  ## 0: H264 encoder  1:H265 encoder
  enc: 0
  bitrate: 2000000
  ##The file name without suffix
  filename: test

streammux:
  width: 1280
  height: 720
  batched-push-timeout: 40000

video-template:
  #x86
  #customlib-name: nvocdr_libs/x86/libnvocdr_impl.so
  #aarch64
  customlib-name: nvocdr_libs/aarch64/libnvocdr_impl.so
  customlib-props:
    - ocdnet-engine-path:../../../models/nvocdr/ocdnet.fp16.engine
    - ocdnet-input-shape:3,736,1280
    - ocdnet-binarize-threshold:0.1
    - ocdnet-polygon-threshold:0.3
    - ocdnet-max-candidate:200
    - ocrnet-engine-path:../../../models/nvocdr/ocrnet.fp16.engine
    - ocrnet-dict-path:character_list.txt
    - ocrnet-input-shape:1,32,100

And when I run I get this error

root@ubuntu:~/deepstream_tao_apps/apps/tao_others/deepstream-nvocdr-app# ./deepstream-nvocdr-app nvocdr_app_config.yml
nvbufsurftransform: Could not get EGL display connection
 Now playing!
Inside Custom Lib : Setting Prop Key=ocdnet-engine-path Value=../../../models/nvocdr/ocdnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocdnet-input-shape Value=3,736,1280
Inside Custom Lib : Setting Prop Key=ocdnet-binarize-threshold Value=0.1
Inside Custom Lib : Setting Prop Key=ocdnet-polygon-threshold Value=0.3
Inside Custom Lib : Setting Prop Key=ocdnet-max-candidate Value=200
Inside Custom Lib : Setting Prop Key=ocrnet-engine-path Value=../../../models/nvocdr/ocrnet.fp16.engine
Inside Custom Lib : Setting Prop Key=ocrnet-dict-path Value=character_list.txt
Inside Custom Lib : Setting Prop Key=ocrnet-input-shape Value=1,32,100
Decodebin child added: source
Decodebin child added: decodebin0
Running...
Decodebin child added: qtdemux0
Decodebin child added: multiqueue0
Decodebin child added: h264parse0
Decodebin child added: capsfilter0
/bin/bash: line 1: lsmod: command not found
/bin/bash: line 1: modprobe: command not found
Decodebin child added: nvv4l2decoder0
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
In cb_newpad
###Decodebin pick nvidia decoder plugin.
nvbufsurface: Could not get EGL display connection
nvbufsurface: Can't get EGL display
ERROR from element nvocdr_template: SetInitParams Error
Error details: SetInitParams Error
Returned, stopping playback
nvbufsurface: Could not get EGL display connection
nvbufsurface: Can't get EGL display
free(): invalid pointer
Aborted (core dumped)

Why am I getting EGL Display connection errors? I am not using a display, and its set to fakesink, not eglsink output

Small update,

I did get the test.cpp program from the NVIDIA-Optical-Character-Detection-and-Recognition-Solution repo working

But the deepstream example is still not working.

With this pipeline

gst-launch-1.0 filesrc location=test.jpg ! jpegparse ! nvv4l2decoder  ! \
m.sink_0 nvstreammux name=m batch-size=1 width=1280 height=1080 ! \
nvdsvideotemplate customlib-name=./deepstream/libnvocdr_impl.so \
customlib-props="ocdnet-engine-path:./ocdnet.fp16.engine" \
customlib-props="ocdnet-input-shape:3,736,1280" \
customlib-props="ocdnet-binarize-threshold:0.1" \
customlib-props="ocdnet-polygon-threshold:0.3" \
customlib-props="ocdnet-max-candidate:200" \
customlib-props="ocrnet-engine-path:./ocrnet.fp16.engine" \
customlib-props="ocrnet-dict-path:./character_list.txt" \
customlib-props="ocrnet-input-shape:1,32,100" ! \
nvmultistreamtiler rows=1 columns=1 width=1280 height=720 ! nvvideoconvert ! nvdsosd ! \
nvvideoconvert ! 'video/x-raw,format=I420' ! jpegenc ! jpegparse ! filesink location=output.jpg

I get these errors

Pipeline is PREROLLING ...
NvMMLiteBlockCreate : Block : BlockType = 256
[JPEG Decode] BeginSequence Display WidthxHeight 1118x1063

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.099: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.099: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.100: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.100: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.127: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.127: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.141: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.141: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.141: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.142: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink
nvstreammux: Successfully handled EOS for source_id=0

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.142: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.142: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.143: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.143: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.143: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.143: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.143: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.143: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

(gst-launch-1.0:1316): GStreamer-CRITICAL **: 04:14:22.144: gst_caps_set_simple: assertion 'GST_IS_CAPS (caps)' failed

(gst-launch-1.0:1316): GStreamer-Base-CRITICAL **: 04:14:22.144: basetransform: second attempt to fixate caps returned invalid (NULL) caps on pad nvdsvideotemplate0:sink

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Because some plugins use CUDA and GLES (not eglsink), the device will be initialized.

This error is usually not related to the monitor.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.