Xavier NX hardware JPEG encoder extremely slow

Hi everybody

I am using the hardware JPEG encoder on a Xavier NX. Erraticaly the encoder will take insane amounts of time to finish. The behavior can sometimes be recovered by reboot, but will show up and persist untill I reboot again. The issue hits the example code from the multimedia API as well as my custom code:

root@ws-jetsonNX:/usr/src/jetson_multimedia_api/samples/05_jpeg_encode# time ./jpeg_encode ../../data/Picture/nvidia-lo
go.yuv 1920 1080 test.jpg --encode-buffer
App run was successful

real    0m10,213s
user    0m0,044s
sys     0m0,016s

The encode-fd variant will yield seg fault, but I suppose that’s due to the thread timing out (only 2s)

root@ws-jetsonNX:/usr/src/jetson_multimedia_api/samples/05_jpeg_encode# time ./jpeg_encode ../../data/Picture/nvidia-lo
go.yuv 1920 1080 test.jpg
libv4l2_nvvidconv (0):(802) (INFO) : Allocating (1) OUTPUT PLANE BUFFERS Layout=0
libv4l2_nvvidconv (0):(818) (INFO) : Allocating (1) CAPTURE PLANE BUFFERS Layout=1
[ERROR] (NvV4l2ElementPlane.cpp:920) <conv> Capture Plane:Timed out waiting for dqthread
Segmentation fault (core dumped)

real    0m10,738s
user    0m0,036s
sys     0m0,024s

The time is always a bit above 10s, independent of image size, quality, etc.
Checking jtop shows, that the JPEG hw encoder is active during the whole execution time. The CPU load is low at around 25% on all cores.
I have the same issue on two devices. Although on one, it was recoverable with a reboot once.

I found a post in this forum, that decribes similiar behavior. There is also no solution, as far as I understand. After a jetpack upgrade the program seemed to work. However my jetpack version is higher than the faulty one:

System info:

NVIDIA NVIDIA Jetson Xavier NX Developer Kit
 L4T 32.4.4 [ JetPack 4.4.1 ]
   Ubuntu 18.04.6 LTS
   Kernel Version: 4.9.140-tegra
 CUDA 10.2.89
   CUDA Architecture: 7.2
 OpenCV version: 4.1.1
   OpenCV Cuda: NO
 CUDNN: 8.0.0.180
 TensorRT: 7.1.3.0
 Vision Works: 1.6.0.501
 VPI: ii vpi 0.4.4 arm64 Vision Programming Interface
 Vulcan: 1.2.70

Maybe somebody can help me…
Best
Alex

Hi,
Could you upgrade to Jetpack 4.6.2 or 4.6.3 and try again? The later release has certain fixes and will be more stable. Suggest upgrade the release version and give it a try.

Hello DaneLLL

Thank you for the suggestion.
I tried updating the live system with this guide:
https://docs.nvidia.com/jetson/archives/r34.1/DeveloperGuide/text/SD/SoftwarePackagesAndTheUpdateMechanism.html#to-update-to-a-new-minor-release

however after the update the encoder example produces a segmentation fault (also after recompiling, no warnings during compilation)

root@ws-jetsonNX:/usr/src/jetson_multimedia_api/samples/05_jpeg_encode# time ./jpeg_encode ../../data/Picture/nvidia-lo
go.yuv 1920 1080 test.jpg --encode-buffer
Opening channel /dev/nvhost-nvjpg failed
Inside the function InitNVJPG NvRmChannelOpen failed
Segmentation fault (core dumped)

real    0m0,649s
user    0m0,048s
sys     0m0,012s

We will flash a fresh system (jetpack 4.6.3) on the board and try again with that.

Best
Alex

The new Image with jetpack 4.6.1 does not show the 10s per FHD image problem any more. It actually worked a few times. After working a bit on the system the following would happen:

root@ws-jetson:/usr/src/jetson_multimedia_api/samples/05_jpeg_encode# ./jpeg_encode ../../data/Picture/nvidia-logo.yuv
1920 1080 test.jpg
libv4l2_nvvidconv (0):(802) (INFO) : Allocating (1) OUTPUT PLANE BUFFERS Layout=0
libv4l2_nvvidconv (0):(818) (INFO) : Allocating (1) CAPTURE PLANE BUFFERS Layout=1
Opening channel /dev/nvhost-nvjpg failed
Inside the function InitNVJPG NvRmChannelOpen failed
Empty JPEG image (DNL not supported)

I don’t know what I changed (mounted drives, and changed network settings) that could have triggered this new error. After another reboot the encoder is working fine again. But I don’t trust it to stay like this.

Hi,
Do you re-flash system image and then install SDK components through SDKManager? This is recommended and shall be stable. If not, please try to install the system through SDKManager and try again.

Currently I can not reproduce the error on the board with jetpack 4.6.3

I kept the 2nd board on jetpack 4.4.1 where I still get the 10s encoding time. Find attached, the dmesg log from running the 05_encodeJpeg example code.
dmesg.txt (5.0 KB)

No. We flashed an image to an SD card and bootet from there.

So can we close this issue as improved at JP4.6.3?

We are still struggeling. JP>4.6 solves the issue, when installed via the SDK manager.
Only “soft” updating via apt or booting an updated SD card image from the nvidia website did not work.

Confusingly, after the SDK manager installation, when running an older SD card image, the error did not appear again.
Does the SDK manager do any firmware updates or similiar, that stays on the system, when we roll back to an old JP?
We are currently trying to get all the necessary installation software for an SDK manager install to our remote (“production”) system. However our bandwidth is low and the tools are pretty heavy, so this might take a while.

We flashed our remote Xavier NX with JetPack 4.6.3 (via SDK Manager) and the issue reappeared.
Same error message as before. Same erratic behavior. Sometimes it works after boot, sometimes not.

Opening channel /dev/nvhost-nvjpg failed
Inside the function InitNVJPG NvRmChannelOpen failed

This is the software stack:

NVIDIA NVIDIA Jetson Xavier NX Developer Kit
 L4T 32.7.3 [ JetPack 4.6.3 ]
   Ubuntu 18.04.6 LTS
   Kernel Version: 4.9.299-tegra
 CUDA 10.2.300
   CUDA Architecture: 7.2
 OpenCV version: 4.1.1
   OpenCV Cuda: NO
 CUDNN: 8.2.1.32
 TensorRT: 8.2.1.9
 Vision Works: 1.6.0.501
 VPI: 1.2.3
 Vulcan: 1.2.70

I will implement a test for the encoder functionality, run that right after boot, and reboot if necessary. Just to get the device working somehow.
However, I would still be very happy about any suggestions.