nvjpegdec slower then jpegdec in gstreamer

makes sense, but i’m getting error

NvxBaseWorkerFunction[2575] comp OMX.Nvidia.std.iv_renderer.overlay.yuv420 Error -2147479552

pipeline

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! 'image/jpeg,width=1920,height=1080, framerate=30/1' ! nvjpegdec ! 'video/x-raw(memory:NVMM)' ! fpsdisplaysink video-sink=nvoverlaysink text-overlay=false -v

Here you are using nvoverlaysink. Do you have a local display (such as a local GUI session with monitor attached) ?. If you are remotely logged into xavier, you would use videosink xvimagesink (if you logged in with ssh -Y for example) if you need to see remote display.

Even with local display, i’m getting the same error :/

Hi,
Please note [nvjpegdec ! ‘video/x-raw(memory:NVMM)’] does not work in continuous motion jpeg decoding. It is verified in single jpeg decoding.

For continuous motion jpeg decoding, please run [nvjpegdec ! ‘video/x-raw’].
Or 12_camera_v4l2_cuda in tegra_multimedia_api

Hi,
Attach a patch for supporting [nvjpegdec ! ‘video/x-raw(memory:NVMM)’] in nvjpegdec.
1 Download gst-jpeg and nvbuf_utils.h for your release version(ex: r32.1)

2 Apply the patch and follow gstjpeg_src\README.txt
// Step 7 is with minor difference

7) sudo cp -i ext/jpeg/.libs/libgstjpeg.so /usr/lib/aarch64-linux-gnu/gstreamer-1.0/<b>libgstnvjpeg.so</b>

3 Run your pipeline. Below is an example of running e-con See3CAM_CU135

$ gst-launch-1.0 v4l2src device=/dev/video1 io-mode=2 ! 'image/jpeg,width=1920,height=1080, framerate=60/1,format=UYVY' ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! nvoverlaysink

0001-gst-jpeg-support-MJPEG-decoding.zip (1.53 KB)

Hi DaneLLL

  1. Can you please explain how to download gst-jpeg and nvbuf_utils.h for my release.
    My system is:
nvidia@jetson:~$ head -n 1 /etc/nv_tegra_release
# R31 (release), REVISION: 1.0, GCID: 13194883, BOARD: t186ref, EABI: aarch64, DATE: Wed Oct 31 22:26:16 UTC 2018
  1. I tried to apply the patch without above downloading, it shows below error
nvidia@jetson:~/Downloads$ ./0001-gst-jpeg-support-MJPEG-decoding.patch 
diff: unrecognized option '--git'
diff: Try 'diff --help' for more information.
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 2: index: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 3: ---: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 4: +++: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 5: @@: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 9: +#ifdef: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 10: +#include: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 11: +#endif: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 12: +: command not found
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 16: syntax error near unexpected token `('
./0001-gst-jpeg-support-MJPEG-decoding.patch: line 16: `@@ -1065,6 +1069,16 @@ gst_jpeg_dec_negotiate (GstJpegDec * dec, gint width, gint height, gint clrspc)'

Hi jaiforfriend,
Please upgrade to r32 releases. r31.1 is Developer Preview release and not suggested for production.

Can you share any link or something to know how to upgrade.
Sorry for bothering much.

Thank you.

Hi,
Please install through sdkmanager.

I dont want to flash jetson.
Please correct me if I am wrong. For this particular problem only gstjpeg_src needs to be upgraded.

Actually, I am wishing to modify as less as possible.
What would be the possible way to have least modification?

Let me try with your post
https://devtalk.nvidia.com/default/topic/1049311/jetson-agx-xavier/nvjpegdec-slower-then-jpegdec-in-gstreamer/post/5348034/#5348034

I downloaded
https://developer.nvidia.com/embedded/dlc/l4t-sources-31-1-0

If I get success, will update this reply.

Thanks

Hi,
Also replied to
https://devtalk.nvidia.com/default/topic/1066469/jetson-agx-xavier/how-to-re-install-nvjpeg-plugins-/post/5401177/#5401177

For better stability, we would recommend upgrade your system.

On r31.1, suggest you run #17

Actually I am restricting myself to upgrade, because my whole project is configured on jetson.

I carefully followed steps shared by you in one of your previos posts.
✔ I downloaded L4T R31.1 Sources:
https://developer.nvidia.com/embedded/dlc/l4t-sources-31-1-0
✔ Untar the gstjpeg_src.tbz2 on target at $HOME.
✔ I Applied patch through patch -p1 < 0001-gst-jpeg-support-MJPEG-decoding.patch
Output: patching file gst-jpeg-1.0/ext/jpeg/gstjpegdec.c

Then I followed gstjpeg_src\README.txt.
✔ export NOCONFIGURE=true
✔ export CFLAGS=“-I$HOME/gstjpeg_src/nv_headers -DUSE_TARGET_TEGRA”
✔ ./autogen.sh
✔ ./configure
✔ make
✔ sudo cp -i ext/jpeg/.libs/libgstjpeg.so /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so
Output: Asked for overwrite. Then I confirmed, yes.
✔ sudo ln -sf tegra/libnvjpeg.so libjpeg.so
✔ sudo ln -sf tegra/libnvjpeg.so libjpeg.so.8

Now if I perform ‘gst-inspect-1.0 nvjpegdec’ (Good news)

Factory Details:
  Rank                     primary - 10 (246)
  Long-name                JPEG image decoder
  Klass                    Codec/Decoder/Image
  Description              Decode images from JPEG format
  Author                   Wim Taymans <wim@fluendo.com>

Plugin Details:
  Name                     nvjpeg
  Description              Nvidia accelerated JPeg plugin library
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so
  Version                  1.2.3
  License                  LGPL
  Source module            gst-jpeg-1.0
  Source release date      2014-02-08
  Binary package           GStreamer Good Jpeg Plug-ins source release
  Origin URL               Unknown package origin
.
.

Good for me.
nvjpegdec pipeline working good.

gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! 'image/jpeg,width=1920,height=1080, framerate=60/1,format=I420' ! nvjpegdec ! 'video/x-raw(memory:NVMM),format=I420' ! fpsdisplaysink video-sink=fakesink text-overlay=false -v

Problem 1: But if I perform ‘gst-inspect-1.0 jpegdec’ (Strange news)

Segmentation fault (core dumped)        // Right now I can ignore this

Problem 2: And also if perform ‘v4l2-ctl --list-devices’ I get below warning

v4l2-ctl: /usr/lib/aarch64-linux-gnu/libjpeg.so.8: no version information available (required by /usr/lib/aarch64-linux-gnu/libv4lconvert.so.0)
3.0 USB Camera (usb-3610000.xhci-3):
	/dev/video0

Problem 3(main): My ultimate aim is h265 encoding

nvidia@jetson:~/jai$ gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 num-buffers=200 ! 'image/jpeg,width=1920,height=1080, framerate=60/1,format=I420' ! videorate drop-only = true max-rate=30 ! nvjpegdec ! nvvidconv ! omxh265enc ! qtmux ! filesink location=test2.mp4 -v
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)I420, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)I420, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:src: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, format=(string)I420, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, format=(string)I420, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstVideoRate:videorate0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)I420, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = image/jpeg, width=(int)1920, height=(int)1080, framerate=(fraction)60/1, format=(string)I420, pixel-aspect-ratio=(fraction)1/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstNvJpegDec:nvjpegdec0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8 
NVMEDIA: H265 : Profile : 1 
/GstPipeline:pipeline0/GstOMXH265Enc-omxh265enc:omxh265enc-omxh265enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)I420, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)I420, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)1:4:0:0, framerate=(fraction)30/1
NvMMLiteVideoEncDoWork: Surface resolution (0 x 0) smaller than encode resolution (1920 x 1080)
VENC: NvMMLiteVideoEncDoWork: 4105: BlockSide error 0x4
Event_BlockError from 0BlockHevcEnc : Error code - 4
Sending error event from 0BlockHevcEncERROR: from element /GstPipeline:pipeline0/GstOMXH265Enc-omxh265enc:omxh265enc-omxh265enc0: GStreamer encountered a general supporting library error.
Additional debug info:
/dvs/git/dirty/git-master_linux/external/gstreamer/gst-omx/omx/gstomxvideoenc.c(1331): gst_omx_video_enc_loop (): /GstPipeline:pipeline0/GstOMXH265Enc-omxh265enc:omxh265enc-omxh265enc0:
OpenMAX component in error state Bad parameter (0x80001005)
Execution ended after 0:00:00.380105692
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Caught SIGSEGV
#0  0x0000007f7aa25ce4 in __waitpid (pid=<optimized out>, stat_loc=0x7fd2711bf4, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x0000007f7aa612a0 in g_on_error_stack_trace ()
#2  0x00000055928cec3c in  ()
#3  0x000000559fe1e0f0 in  ()
Spinning.  Please run 'gdb gst-launch-1.0 10492' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Can you help me to infer this problem?

Thank you

Hi,
Please realize the patch is for r32.1. And r31.1 is developer preview release. If you would like to go to production, you have to upgrade to r32 releases. If you have to use r31.1, suggest you revert the patch.

Ah ok, I see.

Hopefully I will buy a nano, for this experiment. :D

Thank you. :)

Hi,
Xavier is a much more powerful platform than Nano. However, r31.1 is not stable. It would be better to upgrade SW to Xavier .

r32 releases are well tested and verified, and shall bring better user experience.

Hello,
How do I apply this nvjpegdec patch if I am running on Yocto?

When I compile the latest version of gstjpeg_src (L4T sources, version 32.3.1) with the patch applied I get the following error during the make step:

gstjpegenc.c: In function ‘gst_jpegenc_term_destination’:
gstjpegenc.c:300:21: error: ‘struct jpeg_compress_struct’ has no member named ‘IsVendorbuf’
   if(!jpegenc->cinfo.IsVendorbuf)
                     ^
gstjpegenc.c: In function ‘gst_video_enc_check_nvfeatures’:
gstjpegenc.c:356:19: error: ‘struct jpeg_compress_struct’ has no member named ‘IsVendorbuf’
         enc->cinfo.IsVendorbuf = TRUE;
                   ^
gstjpegenc.c: In function ‘gst_jpegenc_handle_frame’:
gstjpegenc.c:498:21: error: ‘struct jpeg_compress_struct’ has no member named ‘IsVendorbuf’
   if(!jpegenc->cinfo.IsVendorbuf)
                     ^
gstjpegenc.c:509:20: error: ‘struct jpeg_compress_struct’ has no member named ‘IsVendorbuf’
   if(jpegenc->cinfo.IsVendorbuf)
                    ^
gstjpegenc.c:514:21: error: ‘struct jpeg_compress_struct’ has no member named ‘pVendor_buf’
       jpegenc->cinfo.pVendor_buf = map.data; /* TODO:Try to copy to base[i] and line instead of pNv_buf*/
                     ^
gstjpegenc.c:544:17: error: ‘struct jpeg_compress_struct’ has no member named ‘outputBuffSize’
   jpegenc->cinfo.outputBuffSize = jpegenc->bufsize + (512 << 10);
                 ^
gstjpegenc.c:546:66: error: ‘struct jpeg_compress_struct’ has no member named ‘outputBuffSize’
   jpegenc->output_mem = gst_allocator_alloc (NULL, jpegenc->cinfo.outputBuffSize, &params);
                                                                  ^
gstjpegenc.c:559:17: error: ‘struct jpeg_compress_struct’ has no member named ‘bMeasure_ImageProcessTime’
   jpegenc->cinfo.bMeasure_ImageProcessTime = jpegenc->bMeasure_ImageProcessTime;
                 ^
gstjpegenc.c:587:20: error: ‘struct jpeg_compress_struct’ has no member named ‘IsVendorbuf’
   if(jpegenc->cinfo.IsVendorbuf)

Has anyone encountered this issue before?

I applied patch to my system on tx2. And i still get so much cpu usage.

cat /etc/nv_tegra_release

R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: t186ref, EABI: aarch64, DATE: Fri Oct 16 19:37:08 UTC 2020

uname -a
Linux TX2-441 4.9.140-tegra #2 SMP PREEMPT Wed Dec 23 13:26:40 +03 2020 aarch64 aarch64 aarch64 GNU/Linux

When I used this gst commandline
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2
! ‘image/jpeg,width=3840,height=2160, framerate=30/1’
! nvjpegdec ! video/x-raw ! nvvidconv ! ‘video/x-raw(memory:NVMM)’
! fpsdisplaysink video-sink=fakesink text-overlay=false -v

I still get too much use of cpu , is it normal ?
this is tegrastats output.
RAM 1148/7860MB (lfb 1165x4MB) SWAP 0/3930MB (cached 0MB) CPU [8%@2024,0%@2034,0%@2035,59%@2035,0%@2035,0%@2034] EMC_FREQ 3%@1866 GR3D_FREQ 0%@1300 NVJPG 857 APE 150 MTS fg 0% bg 0% PLL@69.5C MCPU@69.5C PMIC@100C Tboard@63C GPU@68C BCPU@69.5C thermal@69.1C Tdiode@68C VDD_SYS_GPU 324/324 VDD_SYS_SOC 1567/1567 VDD_4V0_WIFI 0/0 VDD_IN 5597/5586 VDD_SYS_CPU 918/907 VDD_SYS_DDR 1487/1483

this is fpsdisplaysink output
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1376, dropped: 0, current: 18,13, average: 18,06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1386, dropped: 0, current: 18,11, average: 18,06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1396, dropped: 0, current: 18,13, average: 18,06
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 1406, dropped: 0, current: 18,11, average: 18,06