DeepStream-Dewarp app usage with live cam

Hi, I have DeepStream 6.0 with 4.6.0 JetPack with Jetson Nano 4GB.
I have uncalibrated fisheye camera and I want to calibrate it through the dewarp then process for some detection from deepstream but when I seacrh in issues in forum I found this issue that actually fits my problem but when I try to run same command in terminal I always get URI error like:

$sudo ./deepstream-dewarper-app 3 1 v4l2:///dev/video0 0 config_dewarper_perspective.txt 
Now playing: 3, 1,

Using winsys: x11 
Running...
ERROR from element uri-decode-bin: Invalid URI "3".
Error details: gsturidecodebin.c(1384): gen_source_element (): /GstPipeline:dewarper-app-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin
Returned, stopping playback
Deleting pipeline

In this command I tried every possibilites I can : changing tracker types, sink types, removing all numbers giving only v4l2 source , with and w/o config file but every time it gave me same o/p. I don’ know what I can do.
here is another trial:

$ sudo ./deepstream-dewarper-app v4l2:///dev/video0 0 config_dewarper_perspective.txt 
Now playing: v4l2:///dev/video0,

Using winsys: x11 
Decodebin child added: source
Decodebin child added: decodebin0
Running...
In cb_newpad
Error: Decodebin did not pick nvidia decoder plugin.
ERROR from element source: Internal data stream error.
Error details: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:dewarper-app-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstV4l2Src:source:
streaming stopped, reason not-linked (-1)
Returned, stopping playback
Deleting pipeline

and the third:

$ sudo ./deepstream-dewarper-app 2 1 v4l2:///dev/video0 0 config_dewarper_perspective.txt 
Now playing: 2, 1,

Using winsys: x11 
Running...
ERROR from element uri-decode-bin: Invalid URI "2".
Error details: gsturidecodebin.c(1384): gen_source_element (): /GstPipeline:dewarper-app-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin
Returned, stopping playback
Deleting pipeline

Note that: I used default config just to see it’s working or not.

About how to use v4l2 as source, you can refer to the link below: https://forums.developer.nvidia.com/t/deepstream-sdk-faq/80236/36.
Like: uri=v4l2:///dev/video2.

Same error persists:

 sudo ./deepstream-dewarper-app uri=v4l2:///dev/video0 0 config_dewarper_perspective.txt 
Now playing: uri=v4l2:///dev/video0,

Using winsys: x11 
Running...
ERROR from element uri-decode-bin: Invalid URI "uri=v4l2:///dev/video0".
Error details: gsturidecodebin.c(1384): gen_source_element (): /GstPipeline:dewarper-app-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin
Returned, stopping playback
Deleting pipeline

,

/opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-dewarper-test$ sudo ./deepstream-dewarper-app  3 1 v4l2:///dev/video0 0
Now playing: 3, 1,

Using winsys: x11 
Running...
ERROR from element uri-decode-bin: Invalid URI "3".
Error details: gsturidecodebin.c(1384): gen_source_element (): /GstPipeline:dewarper-app-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin
Returned, stopping playback
Deleting pipeline

EXTRA:
tried gst pipeline too:

/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins$ gst-launch-1.0 v4l2src device="/dev/video0" ! 'image/jpeg, width=1280, height=720, framerate=100/1' ! nvv4l2decoder mjpeg=1 ! nvvideoconvert ! nvdewarper config-file=/home/ubuntu/calib.txt  ! m.sink_0 nvstreammux name=m width=1280 height=720 batch-size=2 num-surfaces-per-frame=2 ! nvmultistreamtiler width=1280 height=720 ! nvegltransform ! nveglglessink sync=0
Setting pipeline to PAUSED ...

Using winsys: x11 
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.003601767
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

and camera infos:

$ v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : uvcvideo
	Card type     : HD USB CAMERA
	Bus info      : usb-70090000.xusb-2.2
	Driver version: 4.9.255
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 1: ok)
Format Video Capture:
	Width/Height      : 3840/2160
	Pixel Format      : 'MJPG'
	Field             : None
	Bytes per Line    : 0
	Size Image        : 16588800
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Full Range)
	Flags             : 
Crop Capability Video Capture:
	Bounds      : Left 0, Top 0, Width 3840, Height 2160
	Default     : Left 0, Top 0, Width 3840, Height 2160
	Pixel Aspect: 1/1
Selection: crop_default, Left 0, Top 0, Width 3840, Height 2160
Selection: crop_bounds, Left 0, Top 0, Width 3840, Height 2160
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 30.000 (30/1)
	Read buffers     : 0
                     brightness 0x00980900 (int)    : min=-64 max=64 step=1 default=0 value=-32
                       contrast 0x00980901 (int)    : min=0 max=95 step=1 default=1 value=1
                     saturation 0x00980902 (int)    : min=0 max=100 step=1 default=60 value=60
                            hue 0x00980903 (int)    : min=-2000 max=2000 step=1 default=0 value=0
 white_balance_temperature_auto 0x0098090c (bool)   : default=1 value=1
                          gamma 0x00980910 (int)    : min=64 max=300 step=1 default=100 value=100
                           gain 0x00980913 (int)    : min=0 max=255 step=1 default=100 value=100
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=1 value=1
      white_balance_temperature 0x0098091a (int)    : min=2800 max=6500 step=1 default=4600 value=4600 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=7 step=1 default=0 value=2
         backlight_compensation 0x0098091c (int)    : min=0 max=100 step=1 default=64 value=64
                  exposure_auto 0x009a0901 (menu)   : min=0 max=3 default=3 value=3
              exposure_absolute 0x009a0902 (int)    : min=1 max=10000 step=1 default=156 value=166 flags=inactive
                   pan_absolute 0x009a0908 (int)    : min=-648000 max=648000 step=3600 default=0 value=0
                  tilt_absolute 0x009a0909 (int)    : min=-648000 max=648000 step=3600 default=0 value=0
                 focus_absolute 0x009a090a (int)    : min=0 max=1023 step=1 default=0 value=0 flags=inactive
                     focus_auto 0x009a090c (bool)   : default=1 value=1
                  zoom_absolute 0x009a090d (int)    : min=0 max=60 step=1 default=0 value=0

Your command line is all wrong,please enter the parameters in the correct order. You can refer to our README to learn how to use the command.

Example

// Single Stream
$ ./deepstream-dewarper-app file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_cam6.mp4 6

// Single Stream for Perspective Projection type
$ ./deepstream-dewarper-app --config config_dewarper_perspective.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/yoga.mp4 0

// Multi Stream
$ ./deepstream-dewarper-app file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_cam6.mp4 6 file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_cam6.mp4 6

// Multi Stream for Perspective Projection type
$ ./deepstream-dewarper-app --config config_dewarper_perspective.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/yoga.mp4 0 file:///opt/nvidia/deepstream/deepstream/samples/streams/yoga.mp4 0

// Fish eye distortion correction and perspective projetion
$ ./deepstream-dewarper-app --config config_dewarper_fish_dist_correction.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/fisheye_dist.mp4 0

// Example usage of rot-axes config to get a dewarped surface useful for inferencing applications
$ ./deepstream-dewarper-app --config config_dewarper_rot_axes.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_office.mp4 6

// Example usage of rot-matrix config to use the conventional rotation matrix
$ ./deepstream-dewarper-app --config config_dewarper_rot_matrix.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_office.mp4 6

About the URI, please use the “v4l2:///dev/video0” in the command. But the format of your camera may not be compatible with the source code, you need to modify the source code according to your Camera.

I don’t want to calibrate saved video, I just want to open fish eye cam with dewarper then feed into engine

I already tried this solution above.

It’s just a example to show how to use the command. Please do not randomly change the order of parameters. When you use the camera, you can change the uri to v4l2:///dev/video0 in the command. You should modify some source code to adapt your own camera, you can refer to the source code: create_camera_source_bin in sources\apps\apps-common\src\deepstream_source_bin.c.

Actually w/o dewarp plugin deepstream can run with my fish eye cam. even with this do I need to make changes ?

I added dewarper to my config file like:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=0
rows=1
columns=1
width=720
height=480
gpu-id=0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1280
height=720
enable-padding=0
nvbuf-memory-type=0
max-latency=0
attach-sys-ts-as-ntp=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=3
#1=mp4 2=mkv
container=2
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=1
#iframeinterval=10
bitrate=6000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=/home/ubuntu/Documents/gap/out_koctas.mkv
source-id=0

[source0]
enable=1
type=1
camera-width=3840
camera-height=2160
camera-fps-n=1
camera-fps-d=1
#drop-frame-interval=18

[dewarper]
enable=1
source-id=0
config-file=/home/ubuntu/calib.txt

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=utils/yolotiny_infer_primary.txt

[tracker]
enable=1
# For NvDCF and DeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
# ll-config-file required to set different tracker types
# ll-config-file=config_tracker_IOU.yml
ll-config-file=config_tracker_NvDCF_perf.yml
# ll-config-file=config_tracker_NvDCF_accuracy.yml
# ll-config-file=config_tracker_DeepSORT.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[tests]
file-loop=0

but it doesnot effect to o/p file. How can I connect dewarper into source?

What do you mean w/o dewarp plugin deepstream and o/p file?

I am able to run this command with fish eye camera:
deepstream-app -c /home/ubuntu/Documents/gap/deepstream_app_config.txt /dev/video0
and here is the config file:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=1

[tiled-display]
enable=0
rows=1
columns=1
width=720
height=480
gpu-id=0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1280
height=720
enable-padding=0
nvbuf-memory-type=0
max-latency=0
attach-sys-ts-as-ntp=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=3
#1=mp4 2=mkv
container=2
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=1
#iframeinterval=10
bitrate=6000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=/home/ubuntu/Documents/gap/out_koctas.mkv
source-id=0

[source0]
enable=1
type=1
camera-width=3840
camera-height=2160
camera-fps-n=1
camera-fps-d=1
#drop-frame-interval=18

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=utils/yolotiny_infer_primary.txt

[tracker]
enable=1
# For NvDCF and DeepSORT tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
# ll-config-file required to set different tracker types
# ll-config-file=config_tracker_IOU.yml
ll-config-file=config_tracker_NvDCF_perf.yml
# ll-config-file=config_tracker_NvDCF_accuracy.yml
# ll-config-file=config_tracker_DeepSORT.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[tests]
file-loop=0

1.Deepstream-app demo cannot support dewarper with v4l2 source now. If you want to support it, you need to make some modifications to the source code on your end. Add the create_dewarper_bin logic in the create_camera_source_bin function in sources\apps\apps-common\src\deepstream_source_bin.c file.
2.Could you try to add the diff below to the deepstream_dewarper_test.c to test deepstream-dewarper-app again?

diff
diff --git a/deepstream_dewarper_test.c.bk b/deepstream_dewarper_test.c
index a7c39fa..179c630 100644
--- a/deepstream_dewarper_test.c.bk
+++ b/deepstream_dewarper_test.c
@@ -119,6 +119,7 @@ cb_newpad (GstElement * decodebin, GstPad * decoder_src_pad, gpointer data)
   /* Need to check if the pad created by the decodebin is for video and not
    * audio. */
   if (!strncmp (name, "video", 5)) {
     /* Link the decodebin pad only if decodebin has picked nvidia
      * decoder plugin nvdec_*. We do this by checking if the pad caps contain
      * NVMM memory features. */
@@ -132,6 +133,12 @@ cb_newpad (GstElement * decodebin, GstPad * decoder_src_pad, gpointer data)
       gst_object_unref (bin_ghost_pad);
     } else {
       g_printerr ("Error: Decodebin did not pick nvidia decoder plugin.\n");
+      GstPad *bin_ghost_pad = gst_element_get_static_pad (source_bin, "src");
+      if (!gst_ghost_pad_set_target (GST_GHOST_PAD (bin_ghost_pad),
+              decoder_src_pad)) {
+        g_printerr ("Failed to link decoder src pad to source bin ghost pad\n");
+      }
+      gst_object_unref (bin_ghost_pad);
     }

Could you make it more clear the code , please?

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Yes. There may be some problems with the display format. I have made some changes. You can take a look again and it’s provided in the form of git diff.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.