Using multiple Backends for nvbufsurfacetransform on Jetson

[patrick2]

JetPack 6.0 Orin64
Release 5.15.136-tegra
Orin AGX Development Kit 64GB
L4T 36.3.0
CUDA 12.2.140
cuDNN 8.9.4.25
TensorRT 10.3.0.30
VPI 3.2.5

Hello,

I’m currently developing an application to process camera input on NVIDIA Jetson AGX Orin. The application uses NvBufSurfTransform to convert camera frames from UYVY to RGBA format and performs further transforms like composition on these images. To speed up the processing after the image format conversion, I want to try setting NvBufSurfTransformConfigParams.compute_mode = NvBufSurfTransformCompute_GPU.

However, I’ve encountered an issue: the transformation from UYVY to RGBA fails when using the GPU, as this conversion is only supported on the VIC and session params are set globally.

I would like to know if it is possible to use different backends for NvBufSurfTransform, similar to how it is done in VPI. This would allow me to leverage the GPU for other transformations while still using the VIC for the initial UYVY to RGBA conversion.

Thanks in advance!

[carolyuu]

Hi,
Here are some suggestions for the common issues:

1. Performance

Please run the below command before benchmarking deep learning use case:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2. Installation

Installation guide of deep learning frameworks on Jetson:

• TensorFlow: https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html
• PyTorch: Installing PyTorch for Jetson Platform - NVIDIA Docs
  We also have containers that have frameworks preinstalled:
  Data Science, Machine Learning, AI, HPC Containers | NVIDIA NGC

3. Tutorial

Startup deep learning tutorial:

• Jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson
• TensorRT sample: Jetson/L4T/TRT Customized Example - eLinux.org

4. Report issue

If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.

Thanks!

[patrick2]

Thank you for your answer, but the problem is not deep learning related.

I just found out that even if multiple sessions are supported, composition using GPU is not possible yet. I tried to run NvBufSurfTransformMultiInputBufCompositeBlend() with NvBufSurfTransformCompute_GPU and it returned:

currently GPU based MultiInput Buffer composition operation is not supported

Is there any information if this is supported from Jetpack 6.1/L4T 36.4.0?

[DaneLLL]

Hi,
On Jetson platforms, most conversions are done on hardware VIC engine while calling NvBufSurface APIs. If you need GPU as backend, please use VPI interface.