Blotchy witch GMSL camera and driveworks API

Hardware Platform: DRIVE AGX Pegasus™ Developer Kit
Software Version: DRIVE Software 10
Host Machine Version: native Ubuntu 18.04
SDK Manager Version: 1.0.1.5538

Hi,
I’m trying to set up a video pipeline on pegasus, using driveworks sdk with a gmsl camera that came with the development kit.
After reading the frames from the sensor, I want to encode them to H265 and send them as raw UDP.
I’m using the camera_gmsl sample as a reference.
I have 2 questions:

  1. My problem is that the video coming out from this pipeline is not looking as expected. The video is blotchy and wavy, and the lightning is blinking fast.
    I’m running the sample itself to make sure it’s not my code that’s doing it. The sample is acting the same. But when running the camera_gmsl_raw sample, the video looks a lot better.
    I’m attaching 2 videos that demonstrate the abovesamples.tar.gz (2.9 MB) .
    Unfortunately I’m unable to mock the sample in my code, keeping the H265 encoding.
    This is my original code that worked fine but with blotchy video:
    // ... Initializing ...

    dwSensorParams params{};
    std::string parameterString = "fifo-size=3";
    parameterString += ",output-format=yuv";
    parameterString += ",camera-type=ar0231-rccb-bae-sf3324";
    parameterString += ",camera-group=a";
    params.parameters = parameterString.c_str();
    params.protocol = "camera.gmsl";
    CHECK_DW_ERROR(dwSAL_createSensor(&m_camera, params, m_sal));

    // ...

    std::string newParams = "";
    newParams += "format=h265";
    newParams += ",bitrate=1000000";
    newParams += ",framerate=30";
    newParams += ",type=user";
    m_serializerParams.parameters = newParams.c_str();
    CHECK_DW_ERROR(dwSensorSerializer_initialize(&m_serializer, &m_serializerParams, m_camera));

    // ... Processing ...

    if (m_frame)
    {
        CHECK_DW_ERROR(dwSensorCamera_returnFrame(&m_frame));
    }

    CHECK_DW_ERROR(dwSensorCamera_readFrame(&m_frame, 0, timeout, m_camera));
    CHECK_DW_ERROR(dwSensorSerPreformatted textializer_serializeCameraFrameAsync(m_frame, m_serializer));

And when I tried to change sensor’s output-format to raw, I get DW_INVALING_ARGUMENT from the dwSensorSerializer_initialize function.

  1. I couldn’t find in the SDK reference how to change the GOP of the encoder. Is that possible using the driveworks api?

Hi @eladg1,

Please share the command and steps about how to reproduce the issue with camera_gmsl sample. Thanks!

Why do you want to change? What do you want to change (something like NvMediaEncodeConfigH265::gopLength
in NvMedia)?

Hi VickNV,
thank you for the fast response!

I run the sample like that: ./sample_camera_gmsl from /usr/local/driveworks-2.2/bin/

I’m want to run a law latency video pipeline on a bad network that can sometimes yield high packet loss and jitter.
When the GOP is too high, and I’m losing some P frames, the video shows it. That’s why I want to keep the GOP as low as 15 to 30. The NVMedia API that you referenced is exactly what I’m looking but I’m not sure how to integrate it with Driveworks API. Is that event possible?

Thank you!

I want to highlight that IPP framework will be deprecated from the next release.
So I would suggest you put your efforts into sample_camera_gmsl_custom instead.
You can try if the issue is also reproducible on it.

We just checked our implementation. it is set as 16 now.

If you want to figure out, maybe you can use nvmvid_enc to manipulate the settings for your use cases. Thanks!