Utilizing Pegasus HW Accelerators

Please provide the following info (check/uncheck the boxes after clicking “+ Create Topic”):
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.0.8170
1.5.1.7815
1.5.0.7774
other

Host Machine Version
native Ubuntu 18.04
other

Hi All,

Is there a table or document which gives following information ?

  1. How many documented HW accelerators are there on Xavier (and Orin) ? ISP, PVA, Codecs, OpticalFlow, DisplayHead etc?
  2. Details about capability of each accelerator. What can it accelerate ? E.g. can ISP do debayer, scale and rotate in one go ? What would be data path of such an operation in ISP ? Will it read bayer data from DDR or will it directly receive it from CSI interface ? Will it write debayered output to DDR or and then read it back for scaling or rotation ?
  3. What are recommended nvidia SDK/APIs to use those accelerators ? E.g. I am guessing that in order to accelerate codec functionality, we should use NvMedia ?
  4. Are recommended APIs safety certified (ASIL-D) ?
  5. What is recommended image buffer format to be used with these accelerators ?
  6. Can the accelerator feed or receive data from GPU - if so what is recommended buffer format and APIs to facilitate this ?
  7. Is there a list of opencv functions / operations which can be accelerated by ISP/ PVA or other accelerators ?
  8. Are there any performance numbers which compares time required by an opencv function/algorithm on Xavier CPU/iGPU/accelerator-which-can-accelerate-that-function.

Ideally I would prefer this information tabulated for Xavier and Orin side by side.

Thanks.

PS : Moving to this forum, from - Utilizing Pegasus HW Accelerators

Dear @ai.trucker,
The devzone releases are not in the context of safety. Please contact your NVIDIA representative for more details.

You can use NVMedia/DW APIs to use the HW accelerators. NVmedia/CUDA/DW can support different data formats. Please check NVMedia and DW documentation for more details on supported functions/operations.
You can either use NVStreams or DW ImageStreamer to interop across NVmedia<->CUDA. This helps to exchange buffer across different HW accelerators.
We don’t have perf number comparing to OpenCV functions. You can check the algorithm/function and provide feedback. We will help you implement your algorithm efficiently by making use all available HW accelerators.

Could you check below links

  1. NVIDIA DRIVE AGX Developer Kit | NVIDIA Developer (check Tech specs)
  2. https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/archi_nvmedia.html#
  3. https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/NvMedia/nvmedia_understand.html#
  4. NVIDIA DriveWorks | NVIDIA Developer

Hi SivaRamaKrishnaNV,

  1. Another question is - are there nvidia samples / example codes (for pegasus) which can illustrate usage of each accelerator and NVStreams or DW ImageStreamer to interop across NVmedia<->CUDA ? Can you please point to those samples ?

  2. Are you suggesting that we talk to nvidia representative for details of data path that I am asking in #2 above ? I was not able the find these details in link #1 and #4.

Thanks.

Here are what you are looking for:

  • “NvMedia Sample Applications” document.
  • “NvSciStream Sample Application” document.
  • “Camera Calibration Sample” document.

Dear @ai.trucker,
I asked to contact nvidia representative if you are looking for safety releases. You can find the details of Xavier SoC HW accelerators in Tech specs section in link #1.