Encoding speed

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hi,

I’m trying to to encode 3 separate 4K video streams on a Orin AGX 32GB with a C++ app using the LibArgus Camera API. According to these technical specifications https://developer.nvidia.com/embedded/jetson-modules, the Orin AGX 32GB should be able to handle 3x4K with 30 fps.

In my use case I only need 25 fps, but I struggle to achieve even that. I can get it working with 2 cameras, but when adding a third the frame rate drops.

It does not seems to be a RAM/CPU issue:

$ sudo tegrastats
02-19-2024 07:16:05 RAM 6043/30597MB (lfb 5683x4MB) SWAP 0/15299MB (cached 0MB) CPU [9%@2201,18%@2201,17%@2201,20%@2201,10%@2201,6%@2201,off,off] EMC_FREQ 5%@3199 GR3D_FREQ 0%@[0,0] NVENC 998 VIC_FREQ 31%@729 APE 174 CV0@-256C CPU@49.906C Tboard@38C SOC2@46.468C Tdiode@39C SOC0@45.968C CV1@-256C GPU@-256C tj@49.906C SOC1@46.843C CV2@-256C VDD_GPU_SOC 6137mW/6800mW VDD_CPU_CV 1535mW/1801mW VIN_SYS_5V0 3875mW/4303mW NC 0mW/0mW VDDQ_VDD2_1V8AO 1388mW/1637mW NC 0mW/0mW

Does anybody have any ideas as to what could be the issue, or which parameters I can tune to get this working ?

Thanks!

3

Hi,
Please check Table 4-6 of
Log in | NVIDIA Developer

For encoding 3x 4Kp30, please try H265 encoding.

4

Hi @DaneLLL ,

Sorry, I forgot to mention that I’m already trying H.265 encoding with the main profile (V4L2_MPEG_VIDEO_H265_PROFILE_MAIN).

And I’ve used V4L2_ENC_HW_PRESET_ULTRAFAST and enabling the maxPerfMode.

The application outputs:

Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 8
===== NvVideo: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
892744264
842091865
892744264
842091865
NvVideo: H265 : Profile : 1
5

Hi,
Do you run the 3 encoding tasks in 3 individual processes? Or it is single process with 3 individual encoding threads?

6

The latter: A single process with 3 individual encoding threads.

7

Hi,
Please confirm if you send NV12 block linear NvBufSurface to encoder. If the buffer is I420 pitch linear or NV12 pitch linear, it will be converted to NV12 block linear internally. Would like to eliminate this conversion and see if there is improvement. And please run the script to enable hardware converter at maximum clock:
VPI - Vision Programming Interface: Performance Benchmark

8

Hi,

We were using NVBUF_LAYOUT_PITCH for the NvBuf NvBufSurfaceLayout.
Changing to NVBUF_LAYOUT_BLOCK_LINEAR and running the provided script did not help.

However, we use NVBUF_COLOR_FORMAT_YUV420 for the NvBuf NvBufSurfaceColorFormat. Should I try NVBUF_COLOR_FORMAT_NV12 ?

9

I tried this, it did not change anything.

10

Hi,
Please try the command and see if the 3 videotestsrc sources can achieve 30fps:

$ gst-launch-1.0 -v videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 videotestsrc is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0

And do you use Jetpack 5.1.2?

11

Yes, I use Jetpack 5.1.2.

I get this error when trying to run that gstreamer command:

WARNING: erroneous pipeline: no element “nvvidconv”

EDIT:
I had to install nvidia-l4t-gstreamer

12

The Gstreamer pipeline seems to average 30Fps just fine:

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4663, dropped: 0, current: 27.87, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4671, dropped: 0, current: 30.62, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4678, dropped: 0, current: 31.40, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4679, dropped: 0, current: 30.83, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4687, dropped: 0, current: 29.63, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4693, dropped: 0, current: 28.54, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4694, dropped: 0, current: 29.66, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4702, dropped: 0, current: 28.60, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4711, dropped: 0, current: 33.98, average: 30.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4711, dropped: 0, current: 32.40, average: 30.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4718, dropped: 0, current: 29.87, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4725, dropped: 0, current: 26.12, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4725, dropped: 0, current: 26.38, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4735, dropped: 0, current: 33.87, average: 30.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4742, dropped: 0, current: 32.78, average: 30.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4742, dropped: 0, current: 32.60, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 4748, dropped: 0, current: 25.21, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4758, dropped: 0, current: 30.33, average: 30.01
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 4758, dropped: 0, current: 30.51, average: 30.00
13

Hi,
Please replace videotestsrc with nvarguscamerasrc sensor-id=0/1/2 and check if it can achieve 25fps. It can achieve 30fps with videotestsrc. Please try the camera sources and see if it achieves target frame rate.

14

I did as you suggested and got:

$ gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=1 is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=2 is-live=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
WARNING: erroneous pipeline: no property "is-live" in element "nvarguscamerasrc0"

Then I tried to remove “is-live”:

$ gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=1 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=2 ! video/x-raw,width=1280,height=720 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
WARNING: erroneous pipeline: could not link nvarguscamerasrc0 to nvvconv0, nvarguscamerasrc0 can't handle caps video/x-raw, width=(int)1280, height=(int)720

Then I tried to set the correct resolution:

$ gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 ! video/x-raw,width=3856,height=2180 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=1 ! video/x-raw,width=3856,height=2180 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=2 ! video/x-raw,width=3856,height=2180 ! nvvidconv ! "video/x-raw(memory:NVMM),width=3840,height=2160" ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
WARNING: erroneous pipeline: could not link nvarguscamerasrc0 to nvvconv0, nvarguscamerasrc0 can't handle caps video/x-raw, width=(int)3856, height=(int)2180

I’m not really familiar with GStreamer, so any ideas as how to fix this ? :)

15

Hi,
Please try

$ gst-launch-1.0 -v nvarguscamerasrc sensod-id=0 ! nvvidconv ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensod-id=1 ! nvvidconv ! ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensod-id=2 ! nvvidconv ! ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
16

Hi,

It seems that the video encoder is able to do 30Hz:

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7008, dropped: 0, current: 29.50, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7015, dropped: 0, current: 29.24, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7015, dropped: 0, current: 29.79, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7025, dropped: 0, current: 31.77, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7030, dropped: 0, current: 29.24, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7031, dropped: 0, current: 29.33, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7040, dropped: 0, current: 29.83, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7045, dropped: 0, current: 28.80, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7045, dropped: 0, current: 27.85, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7056, dropped: 0, current: 30.82, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7062, dropped: 0, current: 33.93, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7062, dropped: 0, current: 31.80, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7070, dropped: 0, current: 27.61, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7076, dropped: 0, current: 27.97, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7079, dropped: 0, current: 31.29, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7087, dropped: 0, current: 33.32, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7090, dropped: 0, current: 27.36, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7093, dropped: 0, current: 26.64, average: 29.99
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink2: last-message = rendered: 7103, dropped: 0, current: 30.42, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink1: last-message = rendered: 7108, dropped: 0, current: 34.53, average: 30.00
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 7110, dropped: 0, current: 32.38, average: 30.00

So the issue is likely in my application. Could be the video encoder setup, or the image consumer taking to much time. For the Image consumer I’m using the FrameConsumer object.

NOTE:
There were a few typos / syntax errors in that last pipeline you sent, here is an updated working one:

gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 ! nvvidconv ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=1 ! nvvidconv ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 nvarguscamerasrc sensor-id=2 ! nvvidconv ! nvv4l2h265enc ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0
17

Hi,
The gstreamer plugins are implemented based on Argus and jetson_multimedia_api interfaces, so we would expect both to have identical perfomance. It might be something wrong in the application. Please refer to the sample and investigate further:

/usr/src/jetson_multimedia_api/samples/10_argus_camera_recording/

1 Like
18

Thanks, @DaneLLL .

I’ll post the solution here when/if I find it .

19

Comparing our encoding setup against the /usr/src/jetson_multimedia_api/samples/10_argus_camera_recording/ sample I can say we at least did one unnecessary copy of the image.

Removing one of these copy steps seems to do the trick, at least for achieving 25Hz.

1 Like
21

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.