Gst-launch-1.0 nvjpegdec cannot apply video resize in combination with videoscale

I have noticed that nvjpegdec will work fine to show images using the following command:

gst-launch-1.0 filesrc location=<image.jpg> ! nvjpegdec ! imagefreeze ! xvimagesink

The problem is that I cannot combine nvjpegdec with videoscale doing the following:

$ startx /usr/bin/gst-launch-1.0 -vvv filesrc location=<image.jpg> ! nvjpegdec ! imagefreeze ! videoscale ! video/x-raw,width=1000,height=333 ! xvimagesink


X.Org X Server 1.19.6
Release Date: 2017-12-20
X Protocol Version 11, Revision 0
Build Operating System: Linux 4.4.0-168-generic aarch64 Ubuntu
Current Operating System: Linux jetson-nano 4.9.140-tegra #1 SMP PREEMPT Wed Apr 8 18:10:49 PDT 2020 aarch64
Kernel command line: tegraid=21.1.2.0.0 ddr_die=4096M@2048M section=512M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 console=ttyS0,115200n8 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=0x1000@0xff780000 core_edp_mv=1075 core_edp_ma=4000 tegra_fbmem=0x800000@0x92cb4000 is_hdmi_initialised=1  root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0    root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyS0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 quiet
Build Date: 14 November 2019  06:20:13PM
xorg-server 2:1.19.6-1ubuntu4.4 (For technical support please see http://www.ubuntu.com/support) 
Current version of pixman: 0.34.0
	Before reporting problems, check http://wiki.x.org
	to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
	(++) from command line, (!!) notice, (II) informational,
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(==) Log file: "/var/log/Xorg.0.log", Time: Fri May 29 22:21:21 2020
(==) Using config file: "/etc/X11/xorg.conf"
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1000, height=(int)333, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)74/125, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1000, height=(int)333, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)74/125, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1000, height=(int)333, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)74/125, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1000, height=(int)333, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)74/125, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
WARNING: from element /GstPipeline:pipeline0/GstVideoScale:videoscale0: Internal GStreamer error: code not implemented.  Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
Additional debug info:
gstvideofilter.c(293): gst_video_filter_transform (): /GstPipeline:pipeline0/GstVideoScale:videoscale0:
invalid video buffer received
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstVideoScale:videoscale0: Internal GStreamer error: code not implemented.  Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
Additional debug info:
gstvideofilter.c(293): gst_video_filter_transform (): /GstPipeline:pipeline0/GstVideoScale:videoscale0:
invalid video buffer received
WARNING: from element /GstPipeline:pipeline0/GstVideoScale:videoscale0: Internal GStreamer error: code not implemented.  Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
Additional debug info:
gstvideofilter.c(293): gst_video_filter_transform (): /GstPipeline:pipeline0/GstVideoScale:videoscale0:
invalid video buffer received
...

Hi,
Looks like it is not implemented in videoscale plugin. You can use hardware converter nvvidconv.

$ gst-launch-1.0 filesrc location= IMAG0561.jpg ! nvjpegdec ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1000,height=332' ! nvvidconv ! imagefreeze ! xvimagesink

Hi Dane (and thank you for replying on both of my messages).

As soon as I introduce the nvvidconv element I start getting an error:

ubuntu@jetsonnano:~$ /usr/bin/gst-launch-1.0 -v filesrc location=/home/ubuntu/Downloads/background.jpg ! nvjpegdec ! nvvidconv ! 'video/x-raw(memory:NVMM),width=2560,height=1440' ! nvvidconv ! imagefreeze ! xvimagesink -e
0:00:00.052625490  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "filesrc"
0:00:00.128783247  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvjpegdec"
0:00:00.129858051  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvvidconv"
0:00:00.130248264  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvvidconv"
0:00:00.130731343  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "imagefreeze"
0:00:00.131729220  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "xvimagesink"
0:00:00.132092714  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "pipeline"
0:00:00.132674961  8051   0x55b34d1d00 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "capsfilter"
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)I420
/GstPipeline:pipeline0/Gstnvvconv:nvvconv1.GstPad:src: caps = video/x-raw, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, format=(string)YUY2, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:src: caps = video/x-raw, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, format=(string)YUY2, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = video/x-raw, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, format=(string)YUY2, framerate=(fraction)25/1
/GstPipeline:pipeline0/GstImageFreeze:imagefreeze0.GstPad:sink: caps = video/x-raw, width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, format=(string)YUY2, framerate=(fraction)0/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)I420
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, format=(string)I420
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
ERROR: from element /GstPipeline:pipeline0/GstFileSrc:filesrc0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstFileSrc:filesrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

Funny thing is that I can have this working by replacing nvjpegdec by jpegdec (and still using nvvidconv). I am using NVIDIA L4T 32.4.2 which is the latest available release. My other issue also points to nvjpegdec. Could it be a problem in nvjpegdec and its capabilities?

You would try to specify NVMM memory for nvjpegdec output :

gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! 'video/x-raw(memory:NVMM),width=2560,height=1440' ! nvvidconv ! imagefreeze ! xvimagesink -e

Hi,

Thank you so much for your suggestion. Yeah, I tried that to ensure nvjpegdec and nvvidconv uses the same medium for the stream, but I receive the same error anyway (that filesrc0: Internal data stream error).

Is that command working for you? I am about to reflash de sd card and start from the scratch, as maybe there could be something wrong in my current installation.

Yes, it works for me, but my setup is not clean…

Be sure you have the file, I used another one from visionworks in my command.

You would check where it fails with fakesink. I’d suggest to specify I420 as nvjpegdec output format. Does these work ?

gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM), format=I420' ! fakesink

gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM), format=I420' ! nvvidconv ! fakesink

gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM), format=I420' ! nvvidconv ! imagefreeze ! fakesink

#if ok so far, try xvimasink instead
gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM), format=I420' ! nvvidconv ! imagefreeze ! xvimagesink

#if ok so far, try videscale with nvvidconv:
gst-launch-1.0 -v filesrc location=/usr/share/visionworks/sources/data/lena.jpg ! nvjpegdec ! 'video/x-raw(memory:NVMM), format=I420' ! nvvidconv ! 'video/x-raw(memory:NVMM), width=2560, height=1440' ! nvvidconv ! imagefreeze ! xvimagesink

Where does it fail ? What errors ?

1 Like

This may be the key point: I have patched libnvjpegdec. The patched version was taken from this forum from a post from @DaneLLL. I have no time for searching now, but shouldn’t be hard to find.

1 Like

Thank you again!

Your commands work fine for me. I have found the issue. It seems nvjpegdec fails with some jpg images. I have run the gst-discoverer-1.0 command over some images: if the discoverer does not show the “Internal data stream error.”, then the image will be shown an everything will work perfectly. Also, running gst-typefind-1.0 will work anycase.

Before running those commands I used the following to have the created elements:

$ export GST_DEBUG=*FACTORY*:4

Here are the gst-discoverer-1.0 calls on two images that I am including as attachments:

ubuntu@jetsonnano:~$ gst-discoverer-1.0 /home/ubuntu/Downloads/background.jpg 
0:00:00.051293790 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "pipeline" named "Discoverer"
0:00:00.052692152 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "uridecodebin" named "discoverer-uri"
0:00:00.053113931 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "decodebin"
0:00:00.054421353 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
Analyzing file:///home/ubuntu/Downloads/background.jpg
0:00:00.055940186 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "filesrc" named "source"
0:00:00.056325922 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "decodebin"
0:00:00.056381392 14869   0x55912d6610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
0:00:00.127144271 14869   0x55912e6e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvjpegdec"
0:00:00.242082307 14869   0x55912e6e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "queue" named "discoverer-queue-src_0"
0:00:00.242406949 14869   0x55912e6e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "fakesink" named "discoverer-sink-src_0"
Done discovering file:///home/ubuntu/Downloads/background.jpg
An error was encountered while discovering the file
 Internal data stream error.

Topology:
  video(image): JPEG

Properties:
  Duration: 0:00:00.000000000
  Seekable: no
  Live: no
ubuntu@jetsonnano:~$ gst-discoverer-1.0 /home/ubuntu/Downloads/mouse.jpg 
0:00:00.050858990 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "pipeline" named "Discoverer"
0:00:00.052290945 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "uridecodebin" named "discoverer-uri"
0:00:00.052684495 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "decodebin"
0:00:00.054057439 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
Analyzing file:///home/ubuntu/Downloads/mouse.jpg
0:00:00.055543407 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "filesrc" named "source"
0:00:00.055971332 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "decodebin"
0:00:00.056029614 14855   0x555b8d8610 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
0:00:00.135922676 14855   0x555b8e8e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element "nvjpegdec"
0:00:00.141259398 14855   0x555b8e8e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "queue" named "discoverer-queue-src_0"
0:00:00.141591905 14855   0x555b8e8e80 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "fakesink" named "discoverer-sink-src_0"
Done discovering file:///home/ubuntu/Downloads/mouse.jpg

Topology:
  video(image): JPEG

Properties:
  Duration: 0:00:00.000000000
  Seekable: no
  Live: no

And here the calls to gst-typefind-1.0:

ubuntu@jetsonnano:~$ gst-typefind-1.0 /home/ubuntu/Downloads/background.jpg 
0:00:00.051972137 14955   0x55a46cce10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "pipeline" named "pipeline"
0:00:00.053697640 14955   0x55a46cce10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "filesrc" named "source"
0:00:00.053957489 14955   0x55a46cce10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
0:00:00.054181452 14955   0x55a46cce10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "fakesink" named "fakesink"
/home/ubuntu/Downloads/background.jpg - image/jpeg, width=(int)1920, height=(int)1080, sof-marker=(int)2
ubuntu@jetsonnano:~$ gst-typefind-1.0 /home/ubuntu/Downloads/mouse.jpg 
0:00:00.052823716 14950   0x5594a10e10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "pipeline" named "pipeline"
0:00:00.054532240 14950   0x5594a10e10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "filesrc" named "source"
0:00:00.054786360 14950   0x5594a10e10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "typefind" named "typefind"
0:00:00.055014437 14950   0x5594a10e10 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element "fakesink" named "fakesink"
/home/ubuntu/Downloads/mouse.jpg - image/jpeg, width=(int)640, height=(int)471, sof-marker=(int)0

Next step is to find the patch to apply to libnvjpegdec as maybe it will fix the issue.

These are the images that I was using for the test:

mouse

With patched lib, I am able to process the first image (background.jpeg). I see some weird artifacts on right and bottom borders, though.

For Mouse.jpeg, the issue may be that nvvidconv may not support some odd sizes, but this image has height=471. You may crop to 470 and retry.

Seems nvv4l2decoder can decode odd size but it seems to have something wrong with colors, you would have to investigate if this can be fixed:

gst-launch-1.0 -v filesrc location=/home/nvidia/Pictures/Mouse.jpeg ! jpegparse ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! imagefreeze ! xvimagesink

[EDIT: If you just have to decode one image, the easiest way would just be using CPU jpegdec:

gst-launch-1.0 -v filesrc location=/home/nvidia/Pictures/Mouse.jpeg ! jpegparse ! jpegdec ! imagefreeze ! xvimagesink

]

Hi,
background.jpg looks to be 1920x1080. You may try if downscale works:

gst-launch-1.0 -v filesrc location=/home/ubuntu/Downloads/background.jpg ! nvjpegdec ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1280,height=720' ! nvvidconv ! imagefreeze ! xvimagesink

Os proabbly you don’t set export DISPLAY=:0(or 1).

1 Like