Multiple callbacks from NvMediaIPPPipelineSingleCapture

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

When using the NVMEDIA_IPP_COMPONENT_FILE_READER, calling NvMediaIPPPipelineSingleCapture causes the pipeline to call the NvMediaIPPFileReaderComponentConfig imageGroupReadCallback multiple times. It appears to have the same effect as if I just started the pipeline in a free run mode. I would expect the callback to get called only once per time I call NvMediaIPPPipelineSingleCapture.

Below is sample code for how I setup the pipeline. Is there another setting I need to set to make sure that triggered mode only happens once when NvMediaIPPPipelineSingleCapture is called?

symptoms:

First image is in the queue and is processed correctly.
Reader call back is called before another image is ready to be processed if queue is empty then I return NVMEDIA_STATUS_OK
It appears an empty image is processed through the ISP and the output image is blank.
Because save and load are expensive, the number of blank images is the size of the configured image pool.

If I return NVMEDIA_STATUS_ERROR when there are no images to process then it seems to stop the entire image pipeline.

How can I setup the pipeline so that I will guarantee that I will only get 1 callback per NvMediaIPPPipelineSingleCapture call?

ext_img_device_ = ExtImgDevInit(params);
manager_ = NvMediaIPPManagerCreate(NVMEDIA_IPP_VERSION_INFO, ext_img_device_);
pipeline_ = NvMediaIPPPipelineCreate(manager_);

// setup triggered mode.
bool triggered = NVMEDIA_TRUE;
constexpr size_t PROP_SIZE = 2;
NvMediaIPPPipelineProperty properties[PROP_SIZE];
properties[0].type = NVMEDIA_IPP_PIPELINE_PROPERTY_TRIGGER_BASED_CAPTURE;
properties[0].value = &triggered;
uint32_t frame_delay = 0;
// Set this to indicate take settings immediately, but didn't have any noticeable effect.
properties[1].type = NVMEDIA_IPP_PIPELINE_PROPERTY_SETTINGS_DELAY;
properties[1].value = &frame_delay;
CHECK_NVMEDIA_STATUS_OK(
    NvMediaIPPPipelineSetProperties(pipeline_, PROP_SIZE, properties));

components = file_reader, isp, acp, isc, output;
// Create Components
for each component { NvMediaIPPComponentCreateNew(...); }

// Attach Components
for each component { NvMediaIPPComponentAttach(...); }

// Process images
for (image in images) {
  image = load_image();
  in_queue_.push(image);
  // For some reason this triggers continuous reader callbacks
  NvMediaIPPPipelineSingleCapture(...);
  out_queue_.wait_for_image();
  save_image(out_queue_.pop());
}

// Images are pushed to the reader callback image queue and processed in the ISP.
...


// When all images are done being processed cleanup
// Stop Pipeline
NvMediaIPPPipelineStop(...);

// Destroy Components
for each component { NvMediaIPPComponentDestroy(...); }

NvMediaIPPPipelineDestroy(pipeline_);
NvMediaIPPManagerDestroy(manager_);
ExtImgDevDeinit(ext_img_device_);
2

Any thoughts on why I’m getting multiple callbacks?

3

Dear daden55vvg,

Trigger mode is only supported for capture component and not for file reader component.

How can I setup the pipeline so that I will guarantee that I will only get 1 callback per NvMediaIPPPipelineSingleCapture call?
This is not supported in NvMedia IPP.

Reading the code above, my understanding is that intention here is to process one image at a time.
This could be done by synchronizing the callback with in_queue_.push().

4

That’s a bummer that it isn’t supported. This means testing triggered pipelines can’t be done with saved golden image files. Can I open a feature request to support this mode?

Reading the code above, my understanding is that intention here is to process one image at a time.
Kind of. The main goal is the pipeline shouldn’t run unless there are images in the queue ready to be processed. Pipelining multiple images is ok if it improves throughput.

This could be done by synchronizing the callback with in_queue_.push().
Please clarify synchronizing. Initially I blocked in the reader callback until in_queue_.size() > 0 , which worked but was clunky. Is this the recommended approach?

5

Dear daden55vvg,

Yes, it’s okay. The file reader thread needs to wait until there is an image ready to be processed. It doesn’t matter where the actual wait is, whether it is inside NvMedia IPP or reader callback.

6

Thanks for clarifying. How can I open a feature request to support triggered pipelines in the file reader pipeline?

7

Dear daden55vvg,

Unfortunately there is no plan to add any features to NvMediaIPP(as it is going to be replaced by SIPL framework). Thanks.