Jetson Orin AGX DeepStream Version 7.0 JetPack Version 6
Hi,
I am working on setting up a deepstream pipeline with python on an 8K video stream.
I’ve set up a frame probe on the source pad of nvvideoconvert and pyds to retrieve the frame. Howerer, I have a couple of challenges:
I want to keep the buffer in NV12 format, and I want to to map the frame to the GPU for further use with CUDA ( with cupy ). I’ve looked the at the bindings from deepstream_python_apps and see the following:
This binding maps it to CPU and only supports RGB/RGBA.
This binding forwards the dataPtr, but only works with RGBA/RGB and on X86.
I am aware that i need to make a new binding for NV12 and Jetson, but what is your suggested approach for this?
the method of accessing NV12 data is different on x86 and jetson. please refer to this faq for how to aceess NV12 on Jetson. then deepstream_python_apps is opensource. you can modify it to customize. please refer this link for how to build and install.
/** Holds a pointer to allocated memory. Not valid for
\ref NVBUF_MEM_SURFACE_ARRAY or \ref NVBUF_MEM_HANDLE. */
void * dataPtr;
it depends on NvBufSurfaceMemType. As the comments shown in \opt\nvidia\deepstream\deepstream\sources\includes\nvbufsurface.h, if not NVBUF_MEM_SURFACE_ARRAY and NVBUF_MEM_HANDLE, dataPtr already is GPU data pointer. please refer to sample \opt\nvidia\deepstream\deepstream\sources\apps\sample_apps\deepstream-appsrc-cuda-test\deepstream_appsrc_cuda_test_app.c.
ls -l /opt/nvidia/deepstream/deepstream-7.0/sources/apps/sample_apps/
total 120
drwxr-xr-x 4 root root 4096 Oct 4 09:55 deepstream-3d-action-recognition
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-3d-depth-camera
drwxr-xr-x 6 root root 4096 Oct 4 09:55 deepstream-3d-lidar-sensor-fusion
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-app
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-appsrc-test
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-audio
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-can-orientation-app
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-dewarper-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-gst-metadata-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-image-decode-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-image-meta-test
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-infer-tensor-meta-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-ipc-test
drwxr-xr-x 6 root root 4096 Oct 4 09:55 deepstream-lidar-inference-app
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-mrcnn-test
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-nmos
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-nvdsanalytics-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-nvof-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-opencv-test
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-preprocess-test
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-segmentation-test
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-server
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-test1
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-test2
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-test3
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-test4
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-test5
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-testsr
drwxr-xr-x 3 root root 4096 Oct 4 09:55 deepstream-transfer-learning-app
drwxr-xr-x 2 root root 4096 Oct 4 09:55 deepstream-user-metadata-test
However, if I have NVBUF_MEM_SURFACE_ARRAY or NVBUF_MEM_HANDLE ( which are the preffered mem types for Jetson, I believe?) there is no way to map it to GPU?