Dear Forum,
I was successfully creating single session for stereo camera with dual sensors. And the timestamps seem match each other (even I am not sure if their data was really associated with that timestamp). However, I could not get the slave/second sensor working with the mode different than default. Looking at the implementation of the argus samples, there’s no specificity to distiguish the sensor when calling the setSensorMode(), for example from jetson_multimedia_api/argus/samples/syncStereoRawReprocess/main.cpp :
/**
* Set up ac, sensor and isp settings to reprocess the raw buffer.
* For this sample, default settings are used, if needed settings can be read
* from metadata of raw buffers.
*/
ISourceSettings *iSourceSettings =
interface_cast<ISourceSettings>(iRequest->getSourceSettings());
APP_PRINT("setSensorMode: %d (%dx%d), bitDepth %d output bitDepth %d\n",
options.sensorModeIndex,
moduleInfo[i].iSensorMode[0]->getResolution().width(),
moduleInfo[i].iSensorMode[0]->getResolution().height(),
moduleInfo[i].iSensorMode[0]->getInputBitDepth(),
moduleInfo[i].iSensorMode[0]->getOutputBitDepth());
if (iSourceSettings->setSensorMode(moduleInfo[i].sensorMode[0]) != STATUS_OK)
ORIGINATE_ERROR("Failed to set sensor mode in source settings");
And when the second camera runs at different mode (different resolution and fps), it is difficult to find matched timestamps btw them.
Could you advice, please?
Best Regards,
Khang
Hello @ShaneCCC, @JerryChang,
This is important to trade off btw the resolution and the % of CPU used by ffmpeg for h264 encoding during recording (knowing that there’s h264_nvenc option for HW accelerated encoding with Xavier NX but this has not worked with Orin family yet). Could you help to escalate the question, please ?
hello khang.l4es,
please try calling repeat() again to update your capture settings, repeat() is a convenience method that will queue a request whenever the request queue is empty and the camera is ready to accept new requests.
for instance,
iAutoSettings-> setSensorMode();
iSession->repeat();
Hi @JerryChang,
Indeed, there’s been a repeat() call after configuring multiple of capture settings including setSensorMode() :
//3. Set sensor mode to the source
h->iSourceSettings = interface_cast<ISourceSettings>(interface_cast<IRequest>(h->capRequest)->getSourceSettings());
ISourceSettings* isrcSettings = static_cast<ISourceSettings*>(h->iSourceSettings);
status = isrcSettings->setSensorMode(sensorModes[m_sensor_mode]);
if (Argus::STATUS_OK != status) {
ArgusProvider::changeState(mPort,ARGUS_CAMERA_STATE::OFF);
return ARGUS_STATE::INVALID_SOURCE_CONFIGURATION;
}
// 2. set frame duration
status = isrcSettings->setFrameDurationRange(Argus::Range<uint64_t>(ONE_SECOND_NANOS/mConfig.mFPS,ONE_SECOND_NANOS/mConfig.mFPS));
fps = mConfig.mFPS;
// Get the digital ISP gain range and store it
Argus::IAutoControlSettings* ac = Argus::interface_cast<Argus::IAutoControlSettings>(interface_cast<IRequest>(h->capRequest)->getAutoControlSettings());
if (ac)
{
Range<float> ispDigitalGainRange = ac->getIspDigitalGainRange();
digital_gain_limits_range.first = ispDigitalGainRange.min();
digital_gain_limits_range.second = ispDigitalGainRange.max();
digital_gain_current_range.first = digital_gain_limits_range.first;
digital_gain_current_range.second = digital_gain_limits_range.second;
ac->getToneMapCurve(RGBChannel::RGB_CHANNEL_R,&default_tone_mapping_lut_red);
ac->getToneMapCurve(RGBChannel::RGB_CHANNEL_G,&default_tone_mapping_lut_green);
ac->getToneMapCurve(RGBChannel::RGB_CHANNEL_B,&default_tone_mapping_lut_blue);
}
//3. Set Exposure range
ISensorMode *iSensorMode = interface_cast<ISensorMode>(sensorModes[m_sensor_mode]);
if (!iSensorMode)
{
ArgusProvider::changeState(mPort,ARGUS_CAMERA_STATE::OFF);
return ARGUS_STATE::INVALID_SOURCE_CONFIGURATION;
}
Range<uint64_t> limitExposureTimeRange = iSensorMode->getExposureTimeRange();
exposure_limits_range.first = limitExposureTimeRange.min()/1000ULL; //in us
exposure_limits_range.second = std::min((unsigned long long)(ONE_SECOND_MICROS)/(fps),(unsigned long long)(limitExposureTimeRange.max()/1000ULL)); //Limit if FPS is too quick for DTS max exp (EXPMAX = 100% of FPS as most)
exposure_current_range.first = exposure_limits_range.first;
exposure_current_range.second = exposure_limits_range.second;
if (exposure_limits_range.second!=limitExposureTimeRange.max()/1000ULL)
setAutomaticExposure();
Range<float> sensorModeAnalogGainRange = iSensorMode->getAnalogGainRange();
analog_gain_limits_range.first = sensorModeAnalogGainRange.min();
analog_gain_limits_range.second = sensorModeAnalogGainRange.max();
analog_gain_current_range.first = analog_gain_limits_range.first;
analog_gain_current_range.second = analog_gain_limits_range.second;
//4. Set Clip Rect (if set)
// configure stream settings
auto iOutStreamSettings = interface_cast<IStreamSettings>(interface_cast<IRequest>(h->capRequest)->getStreamSettings(h->mStream.get()));
if (!iOutStreamSettings) {
ArgusProvider::changeState(mPort,ARGUS_CAMERA_STATE::OFF);
return ARGUS_STATE::INVALID_SOURCE_CONFIGURATION;
}
// set stream resolution
status = iOutStreamSettings->setSourceClipRect(Argus::Rectangle<float>(0,0,1.f,1.f));
if (Argus::STATUS_OK != status) {
return ARGUS_STATE::INVALID_SOURCE_CONFIGURATION;
}
/// Create the "stream-on" request --> in Repeat mode, acquireFrame will wait until new frame is available
Argus::Status sts = interface_cast<ICaptureSession>(h->mCaptureSession)->repeat(h->capRequest.get());
if (sts!=Argus::STATUS_OK)
{
ArgusProvider::changeState(mPort,ARGUS_CAMERA_STATE::OFF);
return ARGUS_STATE::CANNOT_START_STREAM_REQUEST;
}
And the mCaptureSession is the single capture session shared by dual sensor devices.
Suppose the single session mode set all sensor to same configuration. Why do you use single session for different sensor mode?
Hi @ShaneCCC,
I do not want to use single session for different sensor mode on different sensor devices. I want to switch to other sensor mode (other than the default sensor mode) for having lower resolution, higher frame-rate, and also binning mode (implemented in the sensor’s driver in other mode) and this should apply to both sensor devices in my stereo configuration.
Currently, if I switch to other sensor mode, only the first (leader) sensor device takes into account of new sensor mode, the other sensor device (follower) is still with the default sensor mode which is unexpected. By consequence, their frames arrive at different timing and it is difficult to find matching timestamp btw them.