NvJpegDecoder JP5.1.2 Issue

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
,
1

Hi,

I am porting some code from 4.6 to 5.1.2,
The code does MJpeg to H265 Transcoding, Mjpeg frames are captured from a Black & White USB camera.
In 5.1.2 NvJpegDecoder only output a single/identical yuv frame (of the first jpeg that was fed to it), subseqent calls to the decoder with new jpegs will return ok, but the resulting YUV buffer is identical to the very first yuv output that the decoder generated.

In order to eliminate our hardware/software involvment in the issue I attempted and was succesful in recreating the issue
on a pure jetson xavier nx EVM.
Steps taken in order to recreate issue:

  • using sdkmanager installed JP 5.1.2 on xavier NX EVM
  • used sdkmanager to install all nvidia packages including jetson-multimedia-api
  • Compile the samples
  • use 06_jpeg_decode to recreate the issue:
    i have 3 jpeg files that were grabbed from the usb camera,
    I have removed any zero padding that may cause premature jepg end error and called the jpeg decode app like so:
    ./jpeg_decode num_files 3 0.jpeg 0.yuv 1.jpeg 1.yuv 2.jpeg 2.yuv
    Ouput will allway contain 3 idential YUV images that are the decoding result of 0.jpeg.
    I also tried both decode modes with the sample, decode to buffer or decode to fd, both yields duplicate frames

A workaround i have attmpetd was to delete the NvJepgDecoder and create it again before each decoding, this solution works but each time an NvJpegDecoder is created a few lines are printed to output log, I guess these prints are from NV jpeg libraries.
Since i have to process 4 cameras running at 60fps each it is impossible to do since the log gets flooded with NvJpegDecoder generation messages.

These are jpegs 0,1,2 that i have used in the test:
Please note the camera is Black and White.

0
01280×720 25.6 KB

1
11280×720 28.9 KB

2
21280×720 26 KB

Can out suggest a workaround for this issue ?

Thanks
Amir

3

Hi,
The issue is known. Please upgrade to 5.1.3 and apply this prebuilt lib:
NvJPEGDecoder generates the same output if called twice with different input buffer - #7 by DaneLLL

4

Hello Dane,

I have followed your instructions and updated to 5.1.3,
And also replaced libnvjpeg to the attached one ( r35_5_TEST_libnvjpeg.zip)

After the change i attempted to run own my application which works fine with 4.6 sdk, but has problems with 5.1.3.

The jpeg files that I try to convert are grabbed from typical usb cameras such as this usb camera:

The Jpegs that I am attempting to convert are attached to this and to the previous message,
I noticed that every few frames the output buffer from jpeg decode is zeroed, the decodeToFd returns ok but output buffer contains all zeros.

I went back to the Xavier NX EVM running a clean 5.1.3 with the updated libnvjpeg.
Please Note that the problem only happens with the jpegs that are grabbed from the USB camera (I also tried one more USB cam that also has this issue), When I do the test with nvidia logo jpeg i dont see the issue, so it must be some internal jpeg problem.
When I test these jpegs with jpeg_decode sample i have notcied the following:
every forfth decoded frame that jpeg_decode sample generates is zeroed, Recraete by calling jpeg_decode with more than 3 files,
in my testing i have fed it 10 frames (any more than that and the app seg faults) like so:
./jpeg_decode num_files 10 0.jpeg 0.yuv 1.jpeg 1.yuv 2.jpeg 2.yuv 3.jpeg 3.yuv 4.jpeg 4.yuv 5.jpeg 5.yuv 6.jpeg 6.yuv 7.jpeg 7.yuv 8.jpeg 8.yuv 9.jpeg 9.yuv

I also attempted to run the same input jpeg, got same results
./jpeg_decode num_files 10 0.jpeg 0.yuv 0.jpeg 1.yuv 0.jpeg 2.yuv 0.jpeg 3.yuv 0.jpeg 4.yuv 0.jpeg 5.yuv 0.jpeg 6.yuv 0.jpeg 7.yuv 0.jpeg 8.yuv 0.jpeg 9.yuv
Output 4.yuv & 8.yuv are blank all other yuv files are ok

Also if I use decode-buffer I get an output raw buffer, but the colors are not correct, it seems like they are miss placed.
I used jpegs from 2 diffrent camera models for this test, they both show same issues:

  • Decode to Fd generates blank frames every 4 frames
  • Decode to Buffer generates incorrect color

Jpeg from camera (B&W):

1
11280×720 28.9 KB

After Conversion using --decode-buffer:
image

Jpeg from 2nd camera:

0
01280×720 116 KB

After Conversion using --decode-buffer:
image

With JP 4.6 using the exact same hardware everything work perfect with both USB cameras that i have tested here and with several other more,
The 4.6 application runs 4 usb cameras transcoding all of them to H265 at about 30 frames per second each, no issues at all.

Please help me fix this problem
Thanks,
Amir

5

Hi,
Do you run 06 sample of Jetpack 5.1.3? There is one change added to NvJpegDecoder.cpp:

    // Enable MJPEG decode
    cinfo.mjpeg_decode = TRUE;
6

Hi,

Yes as described above,
I am using a pure 5.1.3 jetpack on EVM and see the issue with the attached jpegs as described in prevoius message.
So yes, since i am using 5.1.3 the cinfo.mjpeg_decode = TRUE; line of code is present in 5.1.3 06_jepg_decode that demostraes the issue.
when the cinfo.mjpeg_decode is set to false the demo only decodes the first frame.

Please try to check 06_jpeg_decode sample of 5.1.3 with attached jpegs, also replaced the nvidia jpeg library for the test with this one:
NvJPEGDecoder generates the same output if called twice with different input buffer - #7 by DaneLLL

Please help me fix the issue or if possible check if you also see the issue so i will know that this is not some strange issue that my EVM has

Thanks
Amir

7

Hi,
We don’t observe the issue when cinfo.mjpeg_decode = TRUE is set on Jetpack 5.1.3 + the prebuilt lib. Please share us the JPEG files and 06 command so that we can check.

8

Hi,

Like i explained in the above messages,
The issue only happens when using Jpegs that were originated from several modules of USB cameras,
The nvidia logo jpeg that is provided with the samples doesnt show the issue when it is decoded.
Only the jpegs that i grabbed from the USB camera show the issue.

From my review of 06 jpeg decode the default value of after full/pure 5.1.3 install on EVM of cinfo.mjpeg_decode is allready true, i didnot change it,
I made sure that it was set to True, and also replaced the libnvjpeg.so with the new one.

Take this Jpeg for example, try to decode it like in messages above:

1
11280×720 28.9 KB

Thanks

9

Hi,
We run the command on Jetpack 5.1.3 + the prebuilt lib with USB camera AVerMedia CAM513:

12_v4l2_camera_cuda$ ./v4l2_camera_cuda -d /dev/video6 -s 1280x720 -f MJPEG

The camera preview is good. Please try your camera with the sample.

10

Hi,

I have tried two diffrent camera models and get the same results,
I did exactly what you asked, I ran: ( my node is /dev/video0 not 6)
./v4l2_camera_cuda -d /dev/video0 -s 1280x720 -f MJPEG
after replacing the prebuilt lib

I made a short video of issue with my phone,
I get the same issues i have described in the above messages
I run the same command as you did:
./v4l2_camera_cuda -d /dev/video0 -s 1280x720 -f MJPEG
This is what i get for the B&W Camera
B&W Camera.zip (15.5 MB)

This is what I get for the color camera
Color Camera.zip (10.8 MB)

Please fix this issue, it doesnt exist in SDK 4.6
Thanks

11

Hi,
Please check md5sum to ensure you replace the lib correctly:

$ md5sum /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so
a1ea8675a478906db00f454f7264e6a3  /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so

And please try other USB cameras to see if the issue is specific to certain camera.

12

Hi,

I have tested the md5sum and get same as you:
root@ubuntu:/lib/aarch64-linux-gnu/tegra# md5sum libnvjpeg.so
a1ea8675a478906db00f454f7264e6a3 libnvjpeg.so

We use basic USB cameras, I have conducted the following test,
Please acknowledge the findings, this take me time and effort to do,
This time i also used guvcview software in order to open the cameras without Nvidia hardware decoding, guvcview uses software decoding.

Camera #1:

With v4l2_camera_cuda every forfth frame is blank (all zeros), same as seen on the videos that i have attaced to the previous message.

With guvcview there are no issues, video plays fine:

20240414_110300
20240414_1103001920×1081 124 KB

Camera #2:

With v4l2_camera_cuda every forfth frame is blank (all zeros), same as seen on the videos that i have attaced to the previous message.

With guvcview there are no issues, video plays fine:

20240414_112344
20240414_1123441920×1081 127 KB

Camera #3:

With v4l2_camera_cuda every forfth frame is blank (all zeros), same as seen on the videos that i have attaced to the previous message.

With guvcview there are no issues, video plays fine:

20240414_113613
20240414_1136131920×1081 154 KB

For all the cameras i have ran the following command:
./v4l2_camera_cuda -d /dev/video0 -s 1280x720 -f MJPEG
and/or
./guvcview --resolution=1280x720 -d /dev/video0

The results are the same for all the cameras i have tested,
With v4l2_camera_cuda every forfth frame is blank,
With guvcview no issue.

These are all the cameras that I have here and can test,
Please also keep in mind that in 4.6 everything runs fine.

Thanks
Amir

13

Hi,

This is the behavior if you use default libnvjpeg.so in Jetpack 5.1.3. So probably the path of the lib is not correct in your rootfs. In default rootfs, it is in

/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so

But looks like you put it in

/lib/aarch64-linux-gnu/tegra/

14

Hi,

I am using the correct library,
In the default ubuntu rootfs of 5.1.3 /lib is a symbolic link to /usr/lib
so this /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so and this /lib/aarch64-linux-gnu/tegra/libnvjpeg.so point to the same file.

I can run the md5sum on both links and also diff:
nv@ubuntu:/$ md5sum /lib/aarch64-linux-gnu/tegra/libnvjpeg.so
a1ea8675a478906db00f454f7264e6a3 /lib/aarch64-linux-gnu/tegra/libnvjpeg.so
nv@ubuntu:/$ md5sum /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so
a1ea8675a478906db00f454f7264e6a3 usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so

also if I run diff:
nv@ubuntu:/$ diff /lib/aarch64-linux-gnu/tegra/libnvjpeg.so /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so
nv@ubuntu:/$

command returns nothing as expected when files are the same.

other then that If I run a find command to locate all libnvjpeg.so in the entire filesystem it only comes up with one location:
nv@ubuntu:/$ sudo find . -name “libnvjpeg.so”
./usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so

15

Hi,
Please try the attached lib and see if it works with the cameras.

$ md5sum libnvjpeg.so
fff1701242bf2bfd66613f3f17f43d2d *libnvjpeg.so

r35_5_TEST_libnvjpeg_1.zip (151.2 KB)

17

Hi Dane,

Thank you so much for your help,
This library works perfectly.

Thanks again
Amir