Failed to set pipeline configuration

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.9.0.10816
other

Host Machine Version
native Ubuntu 18.04
other

NVMedia’s ./nvsipl_camera is able to detect camera, and print FPS, and write frames to .yuv file.
DriveWorks’s ./sample_camera is able to match camera name, serializer, and .nito file, but unable to set pipeline configuration.

[21-12-2022 11:19:39] Initialize DriveWorks SDK v4.0.0
[21-12-2022 11:19:39] Release build with GNU 7.3.1 from no-gitversion-build against Drive PDK v5.2.6.0
[21-12-2022 11:19:39] Rig::fromFile: Loading rig file: /usr/local/driveworks/data/samples/sensors/camera/camera/rig_ov2311p.json
[21-12-2022 11:19:39] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
[21-12-2022 11:19:39] No valid data file found for camera:sample0 in parameter string: camera-name=OV2311P,interface=csi-e,link=0,output-format=yuv,fifo-size=3,isp-mode=yuv420-uint8 (using configuration folder /usr/local/driveworks/data/samples/sensors/camera/camera/)
[21-12-2022 11:19:39] rig::DatabaseOverlayer: Failed to open a file ‘/tmp/car_vin’ - file likely does not exist - skipping overlay
onInitialize: creating camera.gmsl with params: camera-name=OV2311P,interface=csi-e,link=0,output-format=yuv,fifo-size=3,isp-mode=yuv420-uint8
[21-12-2022 11:19:39] SensorFactory::createSensor() → camera.gmsl, camera-name=OV2311P,interface=csi-e,link=0,output-format=yuv,fifo-size=3,isp-mode=yuv420-uint8
[21-12-2022 11:19:39] CameraBase: pool size set to 8
[21-12-2022 11:19:39] SensorFactory::createSensor() → camera.gmsl.master,
[21-12-2022 11:19:39] CameraMaster::parseDevBlock Getting device info list.
[21-12-2022 11:19:39] CameraClient, setting isp-mode to yuv420-uint8
[21-12-2022 11:19:39] CameraClient, isp-mode set to yuv420-uint8
[21-12-2022 11:19:39] devBlock: 1 Slave = 0 Interface = csi-e Camera_name = OV2311P Link = 0
[21-12-2022 11:19:39] Camera Match Name: OV2311P Description: OV2311, MAX9295 linkIndex: 4294967295 serInfo.Name: MAX9295
[21-12-2022 11:19:39] Client, Setting up information for camera ID 0
[21-12-2022 11:19:39] CameraClient, YUV camera detected, overriding all output types to produce processed only output
[21-12-2022 11:19:39] Client, successfully found info for camera ID 0 bound to id 0
[21-12-2022 11:19:40] CameraClient: using NITO found at /opt/nvidia/nvmedia/nit/OV2311P.nito
[21-12-2022 11:19:40] Initialize DriveWorks VisualizationSDK v4.0.0
[21-12-2022 11:19:40] Initialize DriveWorksGL SDK v4.0.0
onInitialize: Total cameras 1
onInitialize: getting image props 0
onInitialize: initilizing stream: 0
[21-12-2022 11:19:40] StreamConsumerGL: successfully initialized
[21-12-2022 11:19:40] StreamProducerCUDA: successfully initialized
Main: Starting master.
[21-12-2022 11:19:40] CameraGSMLMaster: starting…
[21-12-2022 11:19:40] SIPLMaster::SIPLMaster: Setting up master camera
resou…:1600x1300
resou…:1600x1300
CNvMMAX96712_9295_OV2311::SetSensorConnectionProperty
[21-12-2022 11:19:40] Platform:
[21-12-2022 11:19:40] Platform Config:
[21-12-2022 11:19:40] Description:
[21-12-2022 11:19:40] Number of device blocks: 1
[21-12-2022 11:19:40] Device Block : 0
[21-12-2022 11:19:40] csiPort: 6
[21-12-2022 11:19:40] i2cDevice: 1
[21-12-2022 11:19:40] Deserializer Name: MAX96712
[21-12-2022 11:19:40] Deserializer Description: Maxim 96712 Aggregator
[21-12-2022 11:19:40] Deserializer i2cAddress: 41
[21-12-2022 11:19:40] Simulator Mode: 0
[21-12-2022 11:19:40] Slave Mode: 0
[21-12-2022 11:19:40] Phy Mode: 0
[21-12-2022 11:19:40] Number of camera modules: 1
[21-12-2022 11:19:40] CameraModule index: 0
[21-12-2022 11:19:40] Name :OV2311P
[21-12-2022 11:19:40] Description: OV2311, MAX9295
[21-12-2022 11:19:40] Serializer name: MAX9295
[21-12-2022 11:19:40] Serializer description: Maxim 9295 Serializer
[21-12-2022 11:19:40] Serializer i2cAdress: 98
[21-12-2022 11:19:40] Sensor ID: 0
[21-12-2022 11:19:40] Sensor name: OV2311
[21-12-2022 11:19:40] Sensor description: Leopard camera built-in ISP (OV2311 + FPGA Output YUV)
[21-12-2022 11:19:40] Sensor i2cAddress: 96
[21-12-2022 11:19:40] Sensor isTPGEnabled: 0
[21-12-2022 11:19:40] Sensor isTriggerMode: 0
[21-12-2022 11:19:40] cfa: 4
[21-12-2022 11:19:40] embeddedTopLines: 0
[21-12-2022 11:19:40] embeddedBottomLines: 0
[21-12-2022 11:19:40] inputFormat: 0
[21-12-2022 11:19:40] height: 1300
[21-12-2022 11:19:40] width: 1600
[21-12-2022 11:19:40] fps: 60.0000000
[21-12-2022 11:19:40] Embedded Data: 0
[21-12-2022 11:19:40] CameraMaster available outputs: 8
[21-12-2022 11:19:40] CameraClient: setting pipeline config for camera ID 0
sample_camera: /dvs/git/dirty/git-master_linux/camera/fusa/sipl/src/core/pipelineMgr/spmgr/pipeline/CNvMSensorPipeline.cpp: 462: VerifyPipelineCfg: At least one output needs to be enabled
sample_camera: /dvs/git/dirty/git-master_linux/camera/fusa/sipl/src/core/CNvMCamera.cpp: 195: SetPipelineCfg: Invalid pipeline configuration
[21-12-2022 11:19:40] CameraClient: Failed to set Pipeline configuration for client with id 0, status 1
[21-12-2022 11:19:40] Driveworks exception thrown: DW_NVMEDIA_ERROR: CameraClient: Failed to set pipeline configuration!

terminate called after throwing an instance of ‘std::runtime_error’
what(): [2022-12-21 11:19:40] DW Error DW_NVMEDIA_ERROR executing DW function:
dwSensor_start(m_camera[i])
at samples/sensors/camera/camera/main.cpp:367
Aborted

2

According to https://developer.nvidia.com/drive/ecoystem-xavier#camera, the camera module you’re using isn’t supported.

3

Hi @VickNV ,

Thanks for the reply. Yes, I know it isn’t in that list of supported cameras.
However, we have the NvSIPL_camera NVMedia sample working with this camera and it is able to record raw footage.
Are you saying that this camera is capable of working with DriveOS but DriveWorks does not support it?

4

DRIVE OS / DriveWorks doesn’t support the custom camera module so hasn’t been verified with it.
Did you mean DRIVE OS 5.2.6 works with the custom module out of the box?

5

Yes,
DRIVE OS 5.2.6 out of the box can see the camera. We are able to do two things using the nvsipl camera test sample.:

  1. We can initiate everything and have it print the fps
./nvsipl_camera --platform-config=OV2311P_DPHY_x2 --link-enable-masks="0x0000 0x0000 0x1000 0x0000" --disableISP1Output --disableISP0Output --enableRawOutput --showfps -1
  1. We are able to make .yuv recordings with it
./nvsipl_camera --platform-config=OV2311P_DPHY_x2 --link-enable-masks="0x0000 0x0000 0x1000 0x0000" --disableISP1Output --disableISP0Output --enableRawOutput --showfps -1 --writeFrames 300 --skipFrames 50 --filedump-prefix 'camera-raw'

I can also share the outputs of those commands with verbosity level 4, if you wish.

Using the sipl_query sample of DriveWorks 4.0, we get the following:

nvidia@tegra-ubuntu:/usr/local/driveworks/tools/sensors$ ./sipl_query --camera=OV2311P --detail=tree
NvSIPL library version: 1.0.0
NvSIPL header version: 1.0.0

Platform Detection: 

Cameras: 
Name: OV2311P
Description: OV2311, MAX9295
Port index: 4294967295
Serializer name: MAX9295
Serializer description: Maxim 9295 Serializer
Serializer i2c address: b
EEPROM supported: 0
  Name: OV2311
  Sensor ID: 4294967295
  Description: Leopard camera built-in ISP (OV2311 + FPGA Output YUV)
  TPG Enabled: 0
  Trigger mode: 0
  Channel 0 Details: 
    format: 0
    cfa: 4
    embedded data: 0
    top lines: 0
    bottom lines: 0
    fps: 60
    width: 1600
    height: 1300

So even DriveWorks is able to recognize the camera, in the same platform configuration as the NVSIPL library of DRIVE OS 5.2.6 does.
According to the DriveWorks documentation (DriveWorks SDK Reference: Camera Sensor) the enum dwCameraType does show the OV2311 camera.

enum dwCameraType

Enum of available camera sensors.

Enumerator
DW_CAMERA_GENERIC 	       Generic video source, e.g. video file with raw or processed data
DW_CAMERA_GMSL_AR0231      GMSL AR0231 camera (rev-7)
DW_CAMERA_USB_GENERIC 	   Generic USB camera
DW_CAMERA_GMSL_AR0144 	   GMSL AR0144 camera
DW_CAMERA_GMSL_AR0138 	   GMSL AR0138 camera
DW_CAMERA_GMSL_AR0220 	   GMSL AR0220 camera
DW_CAMERA_GMSL_AR0820 	   GMSL AR0820 camera
DW_CAMERA_GMSL_MN34906     GMSL MN34906 camera
DW_CAMERA_GMSL_OV2311 	   GMSL OV2311 camera
DW_CAMERA_GMSL_IMX390 	   GMSL IMX390 camera
DW_CAMERA_GMSL_CUSTOM      GMSL custom camera defined using NvSIPL

Definition at line 117 of file Camera.h.
6

dug into your issue and found it’s due to DriveWorks 4.0 doesn’t support YUV.
Please check the following topic.

7

Hi,

Ok, this is becoming more clear to me now.

The camera we have also supports RAW output, not only YUV output. Can you confirm if the RAW output will work with DriveWorks 4.0, because the documentation seems to indicate that it will?

8

Yes, RAW is supported
You can go ahead, try the RAW output mode with nvmedia/DriveWorks and share the issue you see.
But as I said, this camera module isn’t in our support list. I doubt how much we can help on this module.