How to use nvjpegdec?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

I have a usb uvc mjpg device, but failed to use nvjpegdec to playback the video stream.

I tried software jpegdec, it works,
gst-launch-1.0 v4l2src device=/dev/video0 ! jpegdec ! nvvidconv ! nv3dsink

even this pipeline works,

gst-launch-1.0 v4l2src device=/dev/video0 ! jpegdec ! jpegenc ! nvjpegdec ! nv3dsink

But directly using nvjpegdec failed,

$ gst-launch-1.0 v4l2src device=/dev/video0 ! nvjpegdec ! nv3dsink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.111874888
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

2

For nvjpegdec, the video output should be in I420 format in NVMM memory. You may try:

$ gst-launch-1.0 v4l2src device=/dev/video0 ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! nv3dsink
3

It does not work.

$ gst-launch-1.0 v4l2src device=/dev/video0 ! jpegparse ! nvjpegdec ! ‘video/x-raw(memory:NVMM),format=I420’ ! nvvidconv ! nv3dsink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason error (-5)
Execution ended after 0:00:00.111333314
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

4

Be sure the resolution and framerate you want to get from your camera:

v4l2-ctl -d0 --list-formats-ext

Note that v4l2-ctl command is provided by apt package v4l-utils.

Then specify the width, height and framerate such as:

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! nv3dsink
5

$ v4l2-ctl -d0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
		Size: Discrete 1600x1200
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
			Interval: Discrete 0.040s (25.000 fps)
			Interval: Discrete 0.050s (20.000 fps)
			Interval: Discrete 0.100s (10.000 fps)
...
...

To my surprise is such two pipelines works

gst-launch-1.0 v4l2src device=/dev/video0 ! jpegdec ! nvvidconv ! nv3dsink
gst-launch-1.0 v4l2src device=/dev/video0 ! jpegdec ! jpegenc ! nvjpegdec ! nv3dsink

But “v4l2src ! nvjpegdec …” does not works even I specify caps filter

6

Try:

gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=30/1,format=MJPG ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=RGBA' ! nv3dsink

Not sure I can help further. It may also be a USB issue. What gives:

lsusb -t
7

$ lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 10000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 480M
|__ Port 1: Dev 4, If 0, Class=Video, Driver=uvcvideo, 480M
|__ Port 1: Dev 4, If 1, Class=Video, Driver=uvcvideo, 480M
|__ Port 1: Dev 4, If 2, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 1: Dev 4, If 3, Class=Audio, Driver=snd-usb-audio, 480M
|__ Port 1: Dev 4, If 4, Class=Human Interface Device, Driver=usbhid, 480M
|__ Port 2: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 4: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M

I did more tests,

“gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=60/1 ! jpegdec ! xvimagesink” works

“gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=60/1 ! nvjpegdec ! xvimagesink” does not work

“gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=60/1 ! jpegdec ! jpegenc ! nvjpegdec ! xvimagesink” works, but latency is considerably higher that software decoder case and there are frame losts.

“gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=30/1 ! nvjpegdec ! xvimagesink” works, but latency is considerably higher that software decoder case and there are frame losts.

“gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1280,height=720,framerate=60/1 ! nvjpegdec ! xvimagesink” works, but latency is considerably higher that software decoder case and there are frame losts.

“… jpegdec ! nvvidconv ! nv3dsink” works, but “… nvjpegdec ! nv3dsink” or " … nvjpegdec ! nvvidconv !nv3dsink" cases do not work with error messages below

New clock: GstSystemClock
Caught SIGSEGV
exec gdb failed: No such file or directory
Spinning. Please run ‘gdb gst-launch-1.0 9154’ to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

8

In summary, software decoder jpegdec works for all W*H@fps, xvimagesink and “nvvidconv ! nv3dsink” cases. And video stream playback is smooth.

But hardware decoder nvjpegdec failed in pipeline negotiating for some cases such as 1080p60. Even pipeline is created, latency is very high, and there are much frame losts. With nv3dsink, there are sigsegv or nvbuffer error messages, video window is blank.