11_camera_object_identification frame count issue

Hi Guys,

I am trying to understand the working of 11_camera_object_identification code in tegra multimedia api. I am unable to get the right number of frames for the CAPTURE_TIME and DEFAULT_FPS specified.

I have provided the following CAPTURE_TIME and DEFAULT_FPS :

// Constant configuration.
#define MAX_PENDING_FRAMES      (3)
#define DEFAULT_FPS             (30)

// Configurations
static int          CAPTURE_TIME = 150; // In seconds.
static bool         DO_STAT = false;

static int nFrames = 0;

I have commented out the opencv processing call just to understand the frames outputted by camera and received by Consumer Thread. Following is the code for the callback :

bool ConsumerThread::converterCapturePlaneDqCallback(
    struct v4l2_buffer *v4l2_buf,
    NvBuffer * buffer,
    NvBuffer * shared_buffer,
    void *arg)
{
    ConsumerThread *thiz = (ConsumerThread*)arg;
    camera_caffe_context *p_ctx = thiz->m_pContext;
    int e;

    if (!v4l2_buf)
    {
        REPORT_ERROR("Failed to dequeue buffer from conv capture plane");
        thiz->abort();
        return false;
    }

    if (v4l2_buf->m.planes[0].bytesused == 0)
    {
        return false;
    }

    //thiz->writeFrameToOpencvConsumer(p_ctx, buffer);

    nFrames++;

    cout<<"Current Frame Number : " << nFrames << endl; 


    e = thiz->m_ImageConverter->capture_plane.qBuffer(*v4l2_buf, NULL);
    if (e < 0)
        ORIGINATE_ERROR("qBuffer failed");

    return true;
}

If I understand correctly, for the input parameters I have given I should be getting 30 * 150 = 4500 frames. However, when I run the program I get much lesser frames. Following dump is an example :

CONSUMER: acquireFd 1828717907 (1 frames)
Current Frame Number : 4446
CONSUMER: releaseFd 1828717907 (1 frames)
CONSUMER: Done.
PRODUCER: Done -- exiting.
PowerServiceCore:handleRequests: timePassed = 2010

(NvOdmDevice) Error NotInitialized: V4L2Device not powered on (in dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function setControlVal(), line 378)
(NvOdmDevice) Error NotInitialized: V4L2Device not powered on (in dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function setControlVal(), line 378)
opencv_handler_close with 0x493490 is called

Why are the number of frames lesser than 4500 even though there is no processing done using the frames ? How does this need to be comprehended. Please help me out.

Thanks.

Hi lamegeorge,
Do yo run with onboard camera(ov5693)?
Do you observe the same in running
~/tegra_multimedia_api/samples/09_camera_jpeg_capture
~/tegra_multimedia_api/samples/10_camera_recording

Hi DaneLLL,

  1. I am using Sony IMX274.

  2. I ran ~/tegra_multimedia_api/samples/09_camera_jpeg_capture and ~/tegra_multimedia_api/samples/10_camera_recording and I am getting around the same number of frames for the same time as used earlier.

Thanks.

Hi lamegeorge,
Please check the timestamp interval:

iStreamSettings->setMetadataEnable(true);
// Print out some capture metadata from the frame.
        IArgusCaptureMetadata *iArgusCaptureMetadata = interface_cast<IArgusCaptureMetadata>(frame);
        if (!iArgusCaptureMetadata)
            ORIGINATE_ERROR("Failed to get IArgusCaptureMetadata interface.");
        CaptureMetadata *metadata = iArgusCaptureMetadata->getMetadata();
        ICaptureMetadata *iMetadata = interface_cast<ICaptureMetadata>(metadata);
        if (!iMetadata)
            ORIGINATE_ERROR("Failed to get ICaptureMetadata interface.");
        CONSUMER_PRINT("\tSensor Timestamp interval: %llu\n",
                       static_cast<unsigned long long>(iMetadata->getSensorTimestamp()) - lastTimestamp);
        lastTimestamp = static_cast<unsigned long long>(iMetadata->getSensorTimestamp());

For 30fps it should be 33.33ms. For capturing 4446 frames in 150 seconds, it is ~33.73ms.

Hi DaneLLL,

I tried including the changes suggested by you. However I get an error mainly due to setMetadataEnable call. Kindly let me know how to use the same.

Error is as follows :

main.cpp: In function ‘bool ArgusSamples::execute()’:
main.cpp:448:22: error: ‘class Argus::IOutputStreamSettings’ has no member named ‘setMetadataEnable’
     iStreamSettings->setMetadataEnable(true);
                      ^
Makefile:60: recipe for target 'main.o' failed
make: *** [main.o] Error 1

Thanks.

Hi lanmegeorge, setMetadataEnable is new in r28.1. You can ignore it on r24.2.1

Hi DaneLLL,

Thanks for your response. I tried your suggestion using r28.1 and I seemed to get few instants of frame drop when I run @ 30 fps 1080p without any processing :

The output showing 1 instant of frame drop is given below :

CONSUMER: 	Sensor Timestamp interval: 33296000
CONSUMER: 	Sensor Timestamp interval: 33283000
CONSUMER: 	Sensor Timestamp interval: 66571000
CONSUMER: 	Sensor Timestamp interval: 33291000
CONSUMER: 	Sensor Timestamp interval: 33282000

Kindly let me know your thoughts on this and suggestions on how to proceed forward.

Thanks.

hello lamegeorge,

please run with below command and sharing the messages to let us confirm which sensor init mode you’re using.

v4l2-ctl -V --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=imx274.raw  -d /dev/video0

Hi JerryChang,

Please find the output of the suggested command below :

Format Video Capture:
	Width/Height      : 1920/1080
	Pixel Format      : 'RG10'
	Field             : None
	Bytes per Line    : 3840
	Size Image        : 4147200
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             :

Hope this helps you to figure out the problem.

Thanks.

hello lamegeorge,

seems you’re working with correct sensor output resolution, you should check sensor driver about the frame rate.
may i know who’s your sensor vendor? you should contact them for further suggestion.
thanks

Hi JerryChang,

The sensor vendor is Leopard Imaging. Camera is Sony IMX274.

Thanks.

Hi Lamegeorge,

You said you got frame drop with 30fps configuration, it may be because the sensor outputing is not exactly 60fps.

We have modified the imx274 sensor driver on R28.1, please contact with Leopard Imaging to get the new driver patch.