Run minipipeline with more than 2 camera

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
Impossible to add more than 2 camera on Multi screen camera.
Configuration:
minipipeline-new_from_support.zip (32.1 KB)

Error String

camIdx: 2 Port Status - isBufferAvailable: 0
camIdx: 2 Port Status - isBound: 1

No Camera id 2, 3, 4 are displayed. the limit should be 9.
Only 2 thread are launched ? why not more ? Which setting/config do I miss ?

[tid:38][dwCameraNodeImpl.hpp:204] readNextData: last output null, read a new frame  // Camera 0 ou 1
[tid:39][dwCameraNodeImpl.hpp:204] readNextData: last output null, read a new frame  // Camera 0 ou 1

image
image1845×997 81.5 KB

Code modification on VisualizationNodeImpl.cpp to investigate

dwStatus VisualizationNodeImpl::preProcessImage()
{
    uint8_t startIdx = 0;
    uint8_t endIdx   = 0;
    if (m_currentSensorScreen == static_cast<uint8_t>(SensorScreenSelect::SCREEN_SINGLE_CAMERA))
    {
        startIdx = m_params.masterCameraIndex;
        endIdx   = startIdx + 1; 
        DW_LOGD << "LucasCheckSetup0 Set for SCREEN_SINGLE_CAMERA): endIdx = " << endIdx << Logger::State::endl;
    }
    else if (m_currentSensorScreen == static_cast<uint8_t>(SensorScreenSelect::SCREEN_MULTI_CAMERA))
    {
        startIdx = 0;
        endIdx   = MAX_CAMERA_COUNT;
        DW_LOGD << "LucasCheckSetup1 Set for SCREEN_MULTI_CAMERA): endIdx = " << endIdx << Logger::State::endl;
    }

    DW_LOGD << "LucasDebug MAX_CAMERA_COUNT:" << MAX_CAMERA_COUNT << Logger::State::endl;

    for (uint8_t camIdx = startIdx; camIdx < endIdx; ++camIdx)
    {
        DW_LOGD << "camIdx: " << camIdx << "/" << endIdx << Logger::State::endl;
        if (m_params.camEnable[camIdx])
        {
            DW_LOGD << "camIdx: " << camIdx << "" << "m_params.camEnable[camIdx] = " << m_params.camEnable[camIdx] << Logger::State::endl;

            auto& port = NODE_GET_INPUT_ARRAY_PORT("IMAGE"_sv, camIdx);
            DW_LOGD << "camIdx: " << camIdx << " Port Status - isBufferAvailable: " << port.isBufferAvailable() << Logger::State::endl;
            DW_LOGD << "camIdx: " << camIdx << " Port Status - isBound: " << port.isBound() << Logger::State::endl;

            if (NODE_GET_INPUT_ARRAY_PORT("IMAGE"_sv, camIdx).isBufferAvailable())
            {
                DW_LOGD << "camIdx: " << camIdx << " Buffer is Available" << Logger::State::endl;
                // Copy convert RGBFP16 image to RGBA image
                dwImage_copyConvertAsync(m_cudaRgbaRenderImage[camIdx], *NODE_GET_INPUT_ARRAY_PORT("IMAGE"_sv, camIdx).getBuffer(), m_params.stream, m_ctx);
                dwStatus status = dwImageStreamerGL_producerSend(m_cudaRgbaRenderImage[camIdx], m_image2GL[camIdx]);
                if (status != DW_SUCCESS)
                {
                    if (status != DW_BUSY_WAITING)
                    {
                        DW_LOGE << "dwImageStreamerGL_producerSend failed " << status << Logger::State::endl;
                        return status;
                    }
                    else
                    {
                        DW_LOGW << "dwImageStreamerGL_producerSend busy" << Logger::State::endl;
                    }
                }
            }
        }
    }

Logs
Complete log:
LogFolder:

launcher.log (22.8 KB)
mainClient_0.log (15.6 MB)
schedule_manager_0.log (1.5 MB)
ssm.log (58.2 KB)
stm_master.log (5.7 KB)

Could someone help ?
Thanks in advance,
Lucas Lattion

3

Dear @lucas.lattion1 ,
You want to run minipipeline using multiple cameras? Is it working with out any issue with --offscreen=1 flag to confirm if the issue is only with display?
I will check internally if there is any limitation with minipipeline and update you..

6

Dear @SivaRamaKrishnaNV ,

Unfortunately the --offscreen=1 option flag doesn’t help.

#Unsuccessfull command
sudo ./run_mini_pipeline-new_from_support.sh --offscreen=1

I am always running on the target.

Could you please clarify if the code at dwCameraNodeImpl.hpp:204 is available somewhere? I’m trying to understand where exactly this limitation comes from.
Also, I noticed a reference to libdwframework_dwnodes.so.5.14. Could you confirm if this library is related to the issue?

Thanks,
Kind regards,
Lucas

7

ibdwframework_dwnodes.so.5.14 contains the implementation of all dw cgf nodes.

By default I notice display outputs single camera (Error in Minipipeline Application after adding custom nodes - #8 by SivaRamaKrishnaNV) . May I know where did you change to get 2nd cameras as well?

8

Is this still an issue to support? Any result can be shared?