How to configure PX2 recorder for 2 cams without drop frames, and for 8 cams

Autonomous Vehicles DRIVE PX 2 DriveWorks
,
1

I use PX2 AutoCruise with DriveWorks SDK version 1.2.400 with 8 cams of type ar0231-rccb-bae-sf3324 connected.
I also have testet the setup with sample_camera_multiple_gmsl --selector-mask=0000111100
to verify that the B cams are running.
Now I want to use recorder to save the h264 data on my ext4 formatted usb drive.

I call recorder with the following rig file for 2 cameras out of the 8:

{
"rig": {
    "sensors": [
        {
            "name": "camera_capture_a",
            "nominalSensor2Rig": {
                "quaternion": [
                    0.5,
                    -0.5,
                    0.5,
                    -0.5
                ],
                "t": [
                    1.77,
                    0.0,
                    1.47
                ]
            },
            "parameter": "camera-type=ar0231-rccb-bae-sf3324,csi-port=a,camera-count=2,camera-mask=0011,format=h264,fifo-size=30,file-buffer-size=4194304",
          "properties": {
                "Model": "ftheta",
                "bw-poly": "0.0 0.00102669443003833 3.77865596590254e-08 -2.1812513600894e-11 4.76916107246175e-14",
                "cx": "960",
                "cy": "604",
                "height": "1208",
                "width": "1920"
            },
            "protocol": "camera.gmsl",
            "sensor2Rig": {
                "quaternion": [
                    0.5,
                    -0.5,
                    0.5,
                    -0.5
                ],
                "t": [
                    1.77,
                    0.0,
                    1.47
                ]
            }
        }
    ],
    "vehicle": {
        "valid": true,
        "value": {
            "COMMENT": "steeringCoefficient is not validated",
            "axlebaseFront": 1.582,
            "axlebaseRear": 1.575,
            "bumperFront": 0.912,
            "bumperRear": 1.109,
            "centerOfMassToRearAxle": 1.564,
            "frontCorneringStiffness": 30654.0,
            "height": 1.473,
            "inertia": 1780.8,
            "length": 4.872,
            "mass": 1779.4,
            "rearCorneringStiffness": 36407.0,
            "steeringCoefficient": 14.8,
            "wheelDiameter": 0.673,
            "wheelbase": 2.85,
            "width": 1.852,
            "widthWithMirrors": 2.121
        }
    }
},
"version": 2

}

This leads to the following output and question:

[15-7-2020 0:39:28] Initialize DriveWorks SDK v1.2.400
[15-7-2020 0:39:28] Release build with GNU 4.9.4 from v1.2.0-rc11-0-ga7f5475 against Vibrante PDK v5.0.10.3
[15-7-2020 0:39:28] Platform: Detected Drive PX2 - Tegra A
[15-7-2020 0:39:28] TimeSource: monotonic epoch time offset is 1594736413029575
[15-7-2020 0:39:28] TimeSource: PTP ioctl returned error. Synchronized time will not be available.
[15-7-2020 0:39:28] TimeSource: Could not detect valid PTP time source at ‘eth0’. Fallback to CLOCK_MONOTONIC.
[15-7-2020 0:39:28] Platform: number of GPU devices detected 1
[15-7-2020 0:39:28] Platform: currently selected GPU device integrated ID 0
[15-7-2020 0:39:28] SDK: Resources mounted from /usr/local/driveworks-1.2/data/resources
[15-7-2020 0:39:28] SDK: Create NvMediaDevice
[15-7-2020 0:39:28] SDK: Create NvMediaIPPManager
[15-7-2020 0:39:28] egl::Display: found 1 EGL devices
[15-7-2020 0:39:28] egl::Display: use drm device: drm-nvdc
[15-7-2020 0:39:28] SensorFactory::createSensor() → camera.gmsl, camera-type=ar0231-rccb-bae-sf3324,csi-port=a,camera-count=2,camera-mask=0011,format=h264,fifo-size=10,file-buffer-size=4194304
[15-7-2020 0:39:28] CameraGMSL: required FPS = 30, resolution = 1920x1208
***** aurix state *****
CAM_PWR_ON: 0x1
TEGRA_GMSL: TEGRA B
TEGRA_FPDL: TEGRA B
TEGRA_A_HDMI: MXM
TEGRA_B_HDMI: MXM
TEGRA_PCI: TEGRA B
FAN_STATE: UNKNOWN

nvmedia isc: TacpCamPowerControl: 241: Camera(s) are already powered on. Power cycling the camera(s)
***** aurix state *****
CAM_PWR_ON: 0x0
TEGRA_GMSL: UNKNOWN
TEGRA_FPDL: UNKNOWN
TEGRA_A_HDMI: UNKNOWN
TEGRA_B_HDMI: UNKNOWN
TEGRA_PCI: UNKNOWN
FAN_STATE: UNKNOWN

***** aurix state *****
CAM_PWR_ON: 0x1
TEGRA_GMSL: UNKNOWN
TEGRA_FPDL: UNKNOWN
TEGRA_A_HDMI: UNKNOWN
TEGRA_B_HDMI: UNKNOWN
TEGRA_PCI: UNKNOWN
FAN_STATE: UNKNOWN

[15-7-2020 0:39:29] CameraGMSL: default IPP event callback function set
[15-7-2020 0:39:29] CameraGMSL: serializer bitrate not specified. Using 8000000.
Rig: 1.json NewSink: /dev/null
[15-7-2020 0:39:29] SensorManager: started
Press s to start, just to see progress, q to quit.
[15-7-2020 0:39:30] CameraGMSL: Frame: 23 CaptureTimestamp: 29957120423 - ICP DROP in camera 1
[15-7-2020 0:39:30] CameraGMSL: Frame: 24 CaptureTimestamp: 29957153781 - ICP DROP in camera 1
[15-7-2020 0:39:30] CameraGMSL: Frame: 25 CaptureTimestamp: 29957187101 - ICP DROP in camera 1
[15-7-2020 0:39:30] CameraGMSL: Frame: 26 CaptureTimestamp: 29957220434 - ICP DROP in camera 1

Where do the drop frames come from?
What do I have to change in the rig file in order to record all 8 cameras?

btw: When I use this rig file with recorder-qtgui I get the failure status with an error message in the terminal saying:
/usr/local/driveworks/tools/capture/recorder-qtgui 1.json
QFont::setPointSizeF: Point size <= 0 (0.000000), must be greater than 0
QFont::setPointSizeF: Point size <= 0 (0.000000), must be greater than 0
recorder-qtgui: recorderSendCmd: failure!
Error: timeout, unable to stop recoder2 gracefully!

Could someone please give me a hint, what’s the issue? Cheers.

2

Hi @hermann.f,

what type of USB flash drive are you using?
please refer to the " Recording Capabilities" section on the “Tutorials” section in the documentation

to record mutiple cameras you should define a “sensor” block for each of them.
although it is possible to record 2(or 3 or 4) cameras of the same group in one sensor config block as you did, it is not recommended for several reasons, but mainly for the reason that every camera (in the same group or not) has it’s own configuration of properties.
please try that and see if you still have dropped frames on the recording.

can you please clarify: does the recording succeed, or is thi failure stopping you from recording?

3

Hi, the output of lsusb is:
lsusb
Bus 002 Device 003: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 04f2:1113 Chicony Electronics Co., Ltd
Bus 001 Device 003: ID 046d:c05a Logitech, Inc. M90/M100 Optical Mouse
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

For recording failure via recorder-qtgui- it shows a failure with the terminal output desribed above, so, it does not record at all : /

4

Hi hermann.fuerntratt,

Have you tried following the “basic recording” tutorial in the driveworks developer guide exactly? I see that you are using a camera-mask parameter in your rig file, but instead you should be using csi-port, camera-count, and siblingIndex with the recorder.

I’d suggest using this config and rig file as a baseline to confirm it is functional and then adapt from there.