GPU Usage of Argus ISP Pipeline

[MyMylou]

Hello,

When running camera stream using v4l, I don’t see any GPU usage
v4l2-ctl -d 1 --set-fmt-video=width=1936,height=1552,pixelformat=RG12 --set-ctrl=bypass_mode=0,preferred_stride=3904 --stream-mmap

When running camera stream using nvgstcapture or gst-launch-1.0, I can see a lot of GPU usage

I modified the argus sample to

• remove the display
• request the image as YUV to avoid color conversion
  • m_iImageNativeBuffer->createNvBuffer(m_iEGLOutputStream->getResolution(), NVBUF_COLOR_FORMAT_YUV420, NVBUF_LAYOUT_PITCH);
  • m_iEglOutputStreamSettings->setPixelFormat(Argus::PIXEL_FMT_YCbCr_420_888);

Why and for which operations, is ISP pipeline using so many GPU ressources ?

I was thinking that Jetson contains a dedicated ISP component and do not use GPU for ISP pipeline.
Thanks

[carolyuu]

Hi,
We support jetson_multimedia_api and gstreamer. Please make sure you have checked the document:
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Multimedia.html
https://docs.nvidia.com/jetson/archives/r36.3/ApiReference/index.html
There are some examples in

Jetson AGX Orin FAQ

Q: How to apply PCN210100 overlay patch? PCN210100 is supported from Jetpack 5.1.3. If you are using 5.1.1 or 5.1.2, there are overlay packages in: Jetson Linux 35.3.1 | NVIDIA Developer Jetson Linux 35.4.1 | NVIDIA Developer If your device is AGX Orin 64GB, please also add the patch mentioned in below link: AGX orin 64GB flash fail - #3 by DaveYYY Without this patch, Orin AGX 64GB with PCN update may not get flashed Q: I cannot control GPIO in JP6.0 and JP6.1, but it works with JP5.x It…

Q: Is there any example of running RTSP streaming?
Q: Is there an example for running UDP streaming?
Q: I have a USB camera. How can I launch it on AGX Orin?

For further issues, please share a method to replicate the issue through gstreamer command, or either sample. We will set up developer kit and check.

Thanks!

[MyMylou]

Hello,

Here is the setup to replicate:

• V4l2 command
  v4l2-ctl --set-fmt-video=width=1936,height=1552,pixelformat=RG12 --set-ctrl=bypass_mode=0,preferred_stride=3904 --stream-mmap
  Result: no GPU usage

• nvgstcapture-1.0
  nvgstcapture-1.0
  Result: High GPU usage

• gstreamer
  gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1936, height=(int)1552, format=(string)NV12, framerate=(fraction)30/1' ! nvvidconv ! queue ! xvimagesink
  result: High GPU usage

My question is:
What part of the ISP pipeline is using the GPU ?

thanks

[JerryChang]

hello MyMylou,

@MyMylou

result: High GPU usage

may I also know the actual GPU usage? do you have numbers for reference?

besides,
you may give another try by preview disabled and shows frame-rate only.
for instance,
$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v

[MyMylou]

@JerryChang

gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! ‘video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12’ ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v

Hello, I shared the IGPU load for each command in my first post (capture with jetson power GUI)

[MyMylou]

@JerryChang

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 sensor-mode=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_0 video-sink=fakesink sync=0 -v

indeed, previous command does not show any GPU load. I will have a look to the implementation to understand what’s the difference
Thanks!

[JerryChang]

FYI, the difference should be EGL to put the camera buffer/stream to render display.