Hardware-decoding of MJPEG

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

I am trying to process MJPEG streams with gstreamer. Doing it in software it works:

gst-launch-1.0 videotestsrc ! jpegenc ! jpegdec ! autovideosink

Unfortunately, it leaks memory. When we saw the same under h264, we fixed it with hardware decoding. The following attempts just display the first frame and freeze:

gst-launch-1.0 videotestsrc ! jpegenc ! nvjpegdec ! autovideosink 
gst-launch-1.0 videotestsrc ! nvjpegenc ! nvjpegdec ! videoconvert ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! jpegparse ! nvjpegdec ! videoconvert ! autovideosink

The same happens with nvv4l2decoder

gst-launch-1.0 videotestsrc ! jpegenc ! nvv4l2decoder  ! nvvidconv ! videoconvert ! autovideosink 
gst-launch-1.0 videotestsrc ! jpegenc ! jpegparse ! nvv4l2decoder ! nvvidconv ! videoconvert ! autovideosink

There are no error messages, and the hardware decoders turn on, but the pipeline is frozen. I have checked the capabilities with gst-inspect, and they should match.

Any ideas?

Jetson Orin Nano dev kit:
GStreamer 1.16.3
L4J: 35.4.1
Jetpack: 5.1.2

Also tried, following the advice from Honey_Patouceul](Gstreamer pipeline for MJPEG RTSP stream):

Stuck on first frame:

gst-launch-1.0 videotestsrc ! jpegenc ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! videoconvert !  autovideosink

Data stream error:

gst-launch-1.0 videotestsrc ! nvjpegenc ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! jpegparse ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink

Erroneous pipeline:

gst-launch-1.0 videotestsrc ! jpegenc ! jpegparse ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
gst-launch-1.0 videotestsrc ! jpegenc ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! jpegparse ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
gst-launch-1.0 videotestsrc ! nvjpegenc ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! autovideosink
3

Hi,
Orin Nano does not have hardware encoder, so please not to use nvjpegenc.

And there is a known issue about MJPEG decoding on Jetpack 5.1.2. Please use Jetpack 5.1.3.

4

Same behaviour after upgrading.

I have gotten some help off the forums:

gst-launch-1.0 videotestsrc ! nvjpegenc ! nvv4l2decoder mjpeg=1 ! nvvidconv ! autovideosink

This works under both 5.1.2 and .3, despite using nvjpegenc, and with better perfromance than raw jpegenc, although they are both interchangeable.

5

Hi,
Please try nvjpegdec plugin in Jetpack 5.1.3. We have some fix in the plugin.

6

Can you share an example pipeline using nvpegdec? The ones I posted earlier are stuck.

7

Hi,
Please try the command:

$ DISPLAY=:0 gst-launch-1.0 videotestsrc is-live=1 ! jpegenc ! queue ! jpegparse ! nvjpegdec mjpegdecode=1 ! video/x-raw ! xvimagesink sync=0
8

Same problem, the first frame is displayed, but nothing else, although both HW decoders turn on.

Does it work on your end?

9

Hi,
We noticed MJPEG decoding through nvjpegdec in gstreamer, or NvJpegDecoder in jetson_multiemdia_api, still does not work well on Jetpack 5.1.3. Will check further and update.

10

Hi,
Please apply this:
NvJPEGDecoder generates the same output if called twice with different input buffer - #7 by DaneLLL

And try the command:

$ DISPLAY=:0 gst-launch-1.0 videotestsrc is-live=1 ! jpegenc ! queue ! jpegparse ! nvjpegdec mjpegdecode=1 ! video/x-raw ! xvimagesink sync=0
11

This one works, but it is glitchy (it flickers on an off). Also, when stress-testing at FHD, I get 11 fps, but the nvv4l2decoder-based pipeline gets 18, without flicker.

12

Hi,
If the source JPEG files are not compressed from YUV420, we would suggest use nvv4l2decoder mjpeg=1

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.