Capture_request and Consumer thread communications

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

In sample 13 of the Jetson MMAPI, to get images from the camera, there are two different loops. One loop for requesting the frames, and one for deliver (or consuming) the frames.
The request loop, repeats these commands:

ICaptureSession *iCaptureSession = interface_cast<ICaptureSession>(captureHolders[j].get()->getSession());
Request *request = captureHolders[j].get()->getRequest();
uint32_t frameId = iCaptureSession->capture(request);

And the consuming loop, repeats the below commands:

UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());
IFrame *iFrame = interface_cast<IFrame>(frame);

I need some explanation on the relationship between these two. I can understand that after requesting a frame, iFrameConsumer->acquireFrame() will wait to receive that frame. However I have noticed that sometimes the uint32_t frameId = iCaptureSession->capture(request); waits for some feedback (or signal) from iFrameConsumer which seems like a deadlock.

I am particularly asking this because I am trying to add a hardware trigger command to the code and I don’t know how to involve the trigger to these two loops.

4

hello Hasa,

BTW,
there’re configurable timeout variables for your hardware trigger use-case,
as you can see from gst-nvarguscamera/gstnvarguscamerasrc.cpp
for instance,
static const uint64_t DEFAULT_WAIT_FOR_EVENT_TIMEOUT = 3000000000;
static const uint64_t DEFAULT_ACQUIRE_FRAME_TIMEOUT = 5000000000;

5

Sorry I didn’t get your answer, Should I add it to the argus code? I am not using gstreamer.

6

hello Hasa,

ya… those were settings for nvarguscamerasrc plugins.
I assume you’re based-on JP-5.1.3/r35.5.0 release verison, you may refer to Topic 284939 to apply pre-built update to enable infinite timeout property.
please see-also developer guide to enable Infinite Timeout Support for verificaiton.

7

I got your point, but that timeOut limit is not my concern for this question. You can assume that I will send all the triggers fast enough to not go beyond the timeout.
My question is how should I match the number of hw triggers with the sw trigger (capture request)? Do I even need to match it?
I have already added some triggers to my code and can get the images, however I have a situation here:

At the begining of the program I need to send some hw trigger (say 40~50 triggers) and at the same time, the captureRequest loop is running in another thread. After a few hundred miliseconds I get one image. After one image, I can match the number of hw triggers with the received images. However after I finish triggering (hw), I try to break the loop of CaptureRequest and the loop of the Consumer, but the program always stuck at one of these two commands:

CaptureRequest: uint32_t frameId = iCaptureSession->capture(request);
Or
Consumer: UniqueObj<Frame> frame(iFrameConsumer->acquireFrame());

And it stays there until I get the timeout error. (Ideally it should pass those commands and proceed to the break command I have added.)

9

hello Hasa,

CaptureRequest
there’s queue buffer allocation, which by default configured as four buffers.
for instance, due to queue depth, VI driver side needs enqueue 4-times (it’s 4 HW trigger in your use-case), sending 1st frame to user-space until the capture queue is filled.

Consumer
it’s a thread to acquire frames, it’s shall also break when you given a stop request.

10

Thank you.

So the number of triggers I send must be always a multiplication of 4 (4,8,12, …) before closing the camera?!

11

hello Hasa,

the minimum HW trigger is 4 to fill the queue. VI outputting frames when queue is full.

12

Sorry I need some more clarification on this:

  • What is VI?
  • The number of the initial Triggers to get the first frame is not constant in my test. Somtimes I need to send 10 Triggers to start the camera working, sometime, up to 100 triggers for the start! Which confuses me which frame is for which trigger.
  • I also noticed that the pointer of the EGLstream frame in the consumer, is a combination of 14 different pointers that will get filled and overwritten one after another. Is this the same queue that you mentioned?
13

hello Hasa,

VI stands for Video Input, please refer to Camera Architecture Stack as see-also.

this is based-on single thread, you may running with v4l2 IOCTL to test it.
for instance,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1

15

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.