Syncsensor with previewconsumerthread

hi,
i am editing syncsensor sample to display the 2 stream images by using preview consumer thread, but it

CUresult cuResult = cuEGLStreamConsumerConnect(&m_cuStreamLeft, m_leftEGLOutputStream->getEGLStream());
ScopedCudaEGLStreamFrameAcquire left(m_cuStreamLeft);
CUeglStreamConnection& m_connection;
CUstream m_stream;
CUgraphicsResource m_resource;
CUeglFrame m_frame;
CUresult r = cuEGLStreamConsumerAcquireFrame(&m_connection, &m_resource, &m_stream, -1);
cuGraphicsResourceGetMappedEglFrame(&m_frame, m_resource, 0, 0);
PRODUCER_PRINT("Launching consumer thread\n");
std::vector<EGLStreamKHR> eglStreams;
eglStreams.push_back(m_leftEGLOutputStream->getEGLStream());
eglStreams.push_back(m_rightEGLOutputStream->getEGLStream());
PreviewConsumerThread consumerThread(g_display.get(), eglStreams,
                                     PreviewConsumerThread::LAYOUT_SPLIT_VERTICAL,
                                     true /* Sync stream frames */);
PROPAGATE_ERROR(consumerThread.initialize());
consumerThread.setLineWidth(1);
consumerThread.setLineColor(1.0f, 0.0f, 0.0f);
// Wait until the consumer is connected to the streams.
PROPAGATE_ERROR(consumerThread.waitRunning());

below shows the error while running.

Capture Session: 0x55a8d27910
Stream Settings: 0x55a8d27950
EGL Stream Settings: 0x55a8d6ec80
PRODUCER: Creating left stream.
EGL Output Stream Left: 0x55a8d2a5a0
PRODUCER: Creating right stream.
EGL Output Stream Right: 0x55a8ce2590
PRODUCER: Launching StereoWorker
Initializing CUDA
CONSUMER: Connecting CUDA consumer to left stream
CONSUMER: Connecting CUDA consumer to right stream
CONSUMER: Waiting for Argus producer to connect to left stream.
Request: 0x55a8ce8a80
Source Settings: 0x55a8ce1a00
Left Stream Settings: 0x55a8ce1a40
Right Stream Settings: 0x55a8ce1a20
PRODUCER: Starting repeat capture requests.
CONSUMER: Waiting for Argus producer to connect to right stream.
CONSUMER: Streams connected, processing frames.
PRODUCER: Launching consumer thread
PREVIEW CONSUMER: Creating OpenGL context.
PREVIEW CONSUMER: Connecting to EGLStream(s).
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/PreviewConsumer.cpp, threadInitialize:123 Unable to connect GL as consumer
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/Thread.cpp, threadFunction:126 (propagating)
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/src/argus_syncsensor.cpp, threadExecute:202 (propagating)
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)

if i remove the preview consumer, then the syncsensor program as per normal.

can i just use this

CUeglFrame m_frame;

to feed to preview consumer thread?

please advise.

Hi,
Do you initialize multiple preview consumers? If there are tow sources, you would need to have two preview consumers.

Yes @DaneLLL
I passed in vector of egl stream pointer and using vertical split mode
It works (display 2 images from each camera in split screen) when I masked out the cudaworker thread for sync.
but this is not the solution, i need to be able to run sync and preview consumer together.
can i run cudaworker and preview consumer together?
but i am getting this error…

CONSUMER: Connecting CUDA consumer to left stream
CONSUMER: Connecting CUDA consumer to right stream
PREVIEW CONSUMER: Creating OpenGL context.
PREVIEW CONSUMER: Connecting to EGLStream(s).
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/PreviewConsumer.cpp, threadInitialize:123 Unable to connect GL as consumer
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/Thread.cpp, threadFunction:126 (propagating)
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/src/argus_syncsensor.cpp, threadExecute:202 (propagating)
Error generated. /home/nvidia/git/src/ros/src/camera/argus_camera_driver/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)

Hi,
Could you provide a patch for syncSensor so that we can reproduce the failure? This can help us do further investigation.

syncsensor_preview.zip (4.3 KB)

hi Dane,

the default is
#define ENABLE_CUDA 1
#define ENABLE_DISPLAY 0

you can confirm the preview is working fine by
#define ENABLE_CUDA 0
#define ENABLE_DISPLAY 1
it is working fine.

however, it is not working when,
#define ENABLE_CUDA 1
#define ENABLE_DISPLAY 1

Hi j2innet,

Is your patch can apply on default main.cpp?
I tried apply syncsensor_preview.patch on /usr/src/jetson_multimedia_api/argus/samples/syncSensor, but got errors.
Please check patch or provide full patch.
Thanks!

hi @carolyuu,
the jetpack4.2and 4.3 sample code is slightly difference.
i attach the original syncsensor that i modified. use this file to apply the patch should be correct.
thanks.

syncsensor.zip (4.7 KB)

Hi,
Have you tried JP4.3 or JP4.4? Would like to know if the issue is seen on newer release.