[Drive OS 6.0.4 / Orin] How to access 4x GMSL Cameras in Python (SIPL) without GStreamer?

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

System Environment:

  • Hardware: NVIDIA DRIVE AGX Orin

  • Sensors: 4x AR0820 (8MP) via MAX96712 Deserializer (Group A)

  • OS: Drive OS 6.0.4 (Linux SDK) / Ubuntu 20.04

  • Kernel: 5.10.104-tegra

  • Python: 3.8.10

  • PyTorch: 1.12.0a0+nv22.06 (CUDA 11.4 verified)

Objective: I am trying to run Object Detection (YOLOv8) on 4x GMSL cameras simultaneously in Python with low latency.

Current Status (What Works): I have verified the hardware and low-level drivers using the pre-compiled C++ sample nvsipl_camera. The following command works perfectly and activates all 4 cameras:
Bash

sudo /opt/nvidia/drive-linux/samples/nvmedia/nvsipl/test/camera/nvsipl_camera \
--platform-config "F008A120RM0A_CPHY_x4" \
--link-enable-masks "0x1111 0x0000 0x0000 0x0000" \
--enableRawOutput \
--showfps

The Problem: I cannot access these streams from Python to run inference. I attempted to use OpenCV with GStreamer (nvsiplsrc), but it appears GStreamer plugins for SIPL are missing or not configured in this Drive OS build.

Failing Python Code:
Python

import cv2

# Standard GStreamer pipeline for SIPL
pipeline = (
    "nvsiplsrc platform-config=F008A120RM0A_CPHY_x4 selector-mask=0x0001 ! "
    "video/x-raw(memory:NVMM) ! "
    "nvvidconv ! video/x-raw, format=BGRx ! "
    "videoconvert ! video/x-raw, format=BGR ! "
    "appsink drop=1"
)

cap = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)

Error Output:

Plaintext

[ WARN:0] global ... open OpenCV | GStreamer warning: Error opening bin: no element "nvsiplsrc"

Investigation: When I try to check for the plugin, the command is not found:

Bash

cnmost@tegra-ubuntu:/$ gst-inspect-1.0
bash: gst-inspect-1.0: command not found

It seems the GStreamer SIPL integration is missing from my environment.

My Questions:

  1. Is there a direct Python API/Binding for NvMedia/SIPL available in Drive OS 6.0 that avoids GStreamer?

  2. If not, what is the recommended low-latency method to get frames from nvsipl C++ drivers into a Python memory buffer (e.g. PyTorch Tensor or NumPy array)?

  3. Are the GStreamer plugins (libgstnvsipl.so) supposed to be present in the standard Drive OS 6.0 docker/flash? If so, where are they located?

I am currently considering using Named Pipes (FIFO) to pipe raw YUV data from the C++ sample to Python, but I am looking for a cleaner, official solution.

Thanks for any support.

3

Dear @Mr.AlexIV ,
The GMSL cameras are accessible only via NvMedia SIPL . DW SAL uses NvMedia SIPL internally. There are no python bindings for DW or NvMedia.
Did you check DW object detector sample? You need to modify the sample to fit your usecase.
Gstreamer plugins are not part of DRIVE OS.

6

Hi, do I have this sample on DRIVE OS 6.0.4? I can’t find it

7

Dear @Mr.AlexIV ,
Is there any reason to use DRIVE OS 6.0.4? Can you use latest release DRIVE OS 6.0.10. For DRIVE OS 6.0.10 installation please see

For DW object detector sample details, please see DriveWorks SDK Reference: Basic Object Detector and Tracker Sample

8

I wanted to use the latest version, but I need to be a DRIVE AGX SDK Developer. However, even if I asked to join, my request has been denied. I sent a new one but it seems still pending. Without that, I can’t pull from docker

9

Dear @Mr.AlexIV ,
Could you please file new topic for DRIVE Developer program issue with screen shots. Thanks