[ARGUS] Proper Way to Deallocate and Release Resources

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I am working on an extremely simple program that uses Argus to capture from an IMX377 almost exactly how the OneShot example works. In fact, I essentially copy and pasted it. In the same program I am also capturing from a Boson YUV Thermal Camera over USB. I know that Argus does not play nice with the YUV camera so while I am creating Argus cameraProvider [Argus::UniqueObjArgus::CameraProvider(Argus::CameraProvider::create())] I disable the YUV USB camera. I then reenable it and access it using <linux/videodev2.h> and <sys/ioctl.h> in a c++ library.

I am able to run my program and capture both RGB and Thermal images no problem. The program executes correctly and terminates correctly. But if I try to run the program again, it gives me a ‘Timeout error’ when I make the v4l2 int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout) call. This error does NOT occur when I completely disable Argus resource acquisition.

So my main question is:
What is the proper way of deallocating Argus resources and destroying Argus Objects? I am currently doing the following:

Camera_RGB::~Camera_RGB()
{
    iSession->stopRepeat();
    iSession->waitForIdle();
    iStream->disconnect();

    request.reset();
    request.release();

    captureSession.reset();

    cameraProvider.reset();
    cameraProvider.release();
}

And your help/insight is incredibly appreciated!

2

It’s better to provide more detail information to analysis.

3

@ShaneCCC I will post the code that I am referring to just below this. Essentially though I am using the Argus OneShot example program and it works fine by itself, but it doesn’t play well when I am also using a YUV Format USB camera - FLIR Boson. So I remove and add the usb camera from the kernal modules using modprobe when I am creating the Argus::CameraProvider so that Argus does not throw an error.

Here is my code for Argus

Camera_RGB::Camera_RGB(const std::string &_cameraName)
{
	m_cameraName 	 		 = _cameraName;
	m_numberOfFrames 		 = DEFAULT_NUMBER_OF_FRAMES;
	m_numberOfFramesCaptured = 0;
}

Camera_RGB::~Camera_RGB()
{
    iSession->stopRepeat();
    iSession->waitForIdle();
    iStream->disconnect();

    request.reset();
    request.release();

    captureSession.reset();

	// Shut down Argus.
    cameraProvider.reset();
    cameraProvider.release();
}

	// Camera_Interface inheritted virtual functions
void Camera_RGB::initialize()
{
	// Argus has a bug when trying to access the USB camera that will cause a crash
	// Here we disables the USB Thermal Boson camera so we avoid this issue
	util::removeThermalCameraModule();

    /*
     * Set up Argus API Framework, identify available camera devices, and create
     * a capture session for the first available device
     */
    cameraProvider = Argus::UniqueObj<Argus::CameraProvider>(Argus::CameraProvider::create());

    iCameraProvider = Argus::interface_cast<Argus::ICameraProvider>(cameraProvider);
    EXIT_IF_NULL(iCameraProvider, "Cannot get core camera provider interface");
    //printf("Argus Version: %s\n", iCameraProvider->getVersion().c_str());

    Argus::Status status = iCameraProvider->getCameraDevices(&m_cameraDevices);
    EXIT_IF_NOT_OK(status, "Failed to get camera devices");

     // Creates the stream between the Argus camera image capturing
     // sub-system (producer) and the image acquisition code (consumer).  A consumer object is
     // created from the stream to be used to request the image frame.  A successfully submitted
     // capture request activates the stream's functionality to eventually make a frame available
     // for acquisition.
    captureSession = Argus::UniqueObj<Argus::CaptureSession>(iCameraProvider->createCaptureSession(m_cameraDevices[DEFAULT_CAMERA_INDEX], &m_status));

    iSession = Argus::interface_cast<Argus::ICaptureSession>(captureSession);
    EXIT_IF_NULL(iSession, "Cannot get Capture Session Interface");

    streamSettings = Argus::UniqueObj<Argus::OutputStreamSettings>(iSession->createOutputStreamSettings());

    iStreamSettings = Argus::interface_cast<Argus::IOutputStreamSettings>(streamSettings);
        EXIT_IF_NULL(iStreamSettings, "Cannot get OutputStreamSettings Interface");
    iStreamSettings->setMode(Argus::STREAM_MODE_MAILBOX);
    iStreamSettings->setPixelFormat(Argus::PIXEL_FMT_YCbCr_420_888);
    iStreamSettings->setResolution(Argus::Size2D<uint32_t>(m_frameWidth, m_frameHeight));
    iStreamSettings->setMetadataEnable(true);

    stream = Argus::UniqueObj<Argus::OutputStream>(iSession->createOutputStream(streamSettings.get()));

    iStream = Argus::interface_cast<Argus::IStream>(stream);
        EXIT_IF_NULL(iStream, "Cannot get OutputStream Interface");

    consumer = Argus::UniqueObj<EGLStream::FrameConsumer>(EGLStream::FrameConsumer::create(stream.get()));

    iFrameConsumer = Argus::interface_cast<EGLStream::IFrameConsumer>(consumer);
        EXIT_IF_NULL(iFrameConsumer, "Failed to initialize Consumer");

    request = Argus::UniqueObj<Argus::Request>(iSession->createRequest(Argus::CAPTURE_INTENT_STILL_CAPTURE));
    
    iRequest = Argus::interface_cast<Argus::IRequest>(request);
        EXIT_IF_NULL(iRequest, "Failed to get capture request interface");

    // add the thermal camera back to the kernal module list so we can use it to capture images
    util::addThermalCameraModule();
}

void Camera_RGB::captureFrames()
{   
    m_status = iRequest->enableOutputStream(stream.get());
		EXIT_IF_NOT_OK(m_status, "Failed to enable stream in capture request");

    uint32_t requestId = iSession->capture(request.get());
    	EXIT_IF_NULL(requestId, "Failed to submit capture request");



for(int i = 0; i < m_numberOfFrames; i++)
    {
	    // Acquire a frame generated by the capture request, get the image from the frame
	    // and create a .JPG file of the captured image	   
	    Argus::UniqueObj<EGLStream::Frame> frame(iFrameConsumer->acquireFrame(FIVE_SECONDS_IN_NANOSECONDS, &m_status));
	       EXIT_IF_NOT_OK(m_status,"[Argus] unable to acquire frame.\n");

	    EGLStream::IFrame *iFrame = Argus::interface_cast<EGLStream::IFrame>(frame);
	       EXIT_IF_NULL(iFrame, "Failed to get IFrame interface");

	    EGLStream::Image *image = iFrame->getImage();
	       EXIT_IF_NULL(image, "Failed to get Image from iFrame->getImage()");

	    EGLStream::IImageJPEG *iImageJPEG = Argus::interface_cast<EGLStream::IImageJPEG>(image);
	       EXIT_IF_NULL(iImageJPEG, "Failed to get ImageJPEG Interface");

	    std::string saveName = "test0" + std::to_string(m_numberOfFramesCaptured) + std::to_string(i) + "-rgb.jpg";
	    m_status = iImageJPEG->writeJPEG(saveName.c_str());
	       EXIT_IF_NOT_OK(m_status, "Failed to write JPEG");
        std::cout << "[Argus] Saved RGB image: " << saveName << std::endl;

	    // Increment the this objects total image capture count
	    m_numberOfFramesCaptured++;
	}
    iRequest->disableOutputStream(stream.get());
    iSession->stopRepeat();
    iSession->waitForIdle();
}

And the code for the Flir Boson Camera uses Linux’s V4L2 since the Flir shows up as a video source /dev/video3.
Hopefully this helps give some more detail.

EDIT

Here is the output from dmesg. I get the NvMap warning whenever I use the Argus Object. The things I want to highlight from the bunch of data below is this error.

[62718.172993] TestHardwareInt[9459]: unhandled level 2 translation fault (11) at 0x000027b5, esr 0x92000046

and this warning

[  507.672090] nvmap_alloc_handle: PID 2450: TestHardwareInt: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant.

[62459.533474] CPU: 1 PID: 9154 Comm: TestHardwareInt Tainted: G        W       4.4.38 #1
[62459.541433] Hardware name: quill (DT)
[62459.545141] task: ffffffc1ea82cb00 ti: ffffffc07aa5c000 task.ti: ffffffc07aa5c000
[62459.552645] PC is at 0x7f9bfba338
[62459.555982] LR is at 0x7f9bfba3f4
[62459.559318] pc : [<0000007f9bfba338>] lr : [<0000007f9bfba3f4>] pstate: 80000000
[62459.566751] sp : 0000007ff062e6e0
[62459.570091] x29: 0000007ff062e6f0 x28: 0000000000000000 
[62459.575444] x27: 0000000000000000 x26: 0000000000000000 
[62459.580793] x25: 0000000000000000 x24: 0000000000000000 
[62459.586156] x23: 0000000000000000 x22: 0000000000000000 
[62459.591513] x21: 0000000000000000 x20: 00000000004a3220 
[62459.596867] x19: 00000000004a4000 x18: 0000000000000020 
[62459.602242] x17: 0000007f9c044fc8 x16: 0000007f9bfba324 
[62459.607590] x15: 0000007f9c077cc0 x14: 0000000000000000 
[62459.612941] x13: 2f6372732f796172 x12: 782d656369766564 
[62459.618304] x11: 0000007ff062e690 x10: 0000000000000088 
[62459.623654] x9 : ffffff80ffffffc8 x8 : 0000000000000040 
[62459.629016] x7 : 7f7f7f7f7f7f7f7f x6 : 00000000004a551b 
[62459.634372] x5 : 0000000000000000 x4 : 0000000000000000 
[62459.639725] x3 : 0000000000000000 x2 : 0000000000000001 
[62459.645088] x1 : 0000000000000001 x0 : 0000000000002710 

[62459.651957] vdso base = 0x7f9c076000
[62682.008296] TestHardwareInt[9400]: unhandled level 2 translation fault (11) at 0x000027b5, esr 0x92000046
[62682.017936] pgd = ffffffc07a144000
[62682.021365] [000027b5] *pgd=00000000fb1f1003, *pud=00000000fb1f1003, *pmd=0000000000000000

[62682.031254] CPU: 2 PID: 9400 Comm: TestHardwareInt Tainted: G        W       4.4.38 #1
[62682.039245] Hardware name: quill (DT)
[62682.042981] task: ffffffc07a12be80 ti: ffffffc07aa54000 task.ti: ffffffc07aa54000
[62682.050515] PC is at 0x7f91398338
[62682.053859] LR is at 0x7f913983f4
[62682.057195] pc : [<0000007f91398338>] lr : [<0000007f913983f4>] pstate: 80000000
[62682.064653] sp : 0000007fd4185c80
[62682.067970] x29: 0000007fd4185c90 x28: 0000000000000000 
[62682.073343] x27: 0000000000000000 x26: 0000000000000000 
[62682.078706] x25: 0000000000000000 x24: 0000000000000000 
[62682.084078] x23: 0000000000000000 x22: 0000000000000000 
[62682.089471] x21: 0000000000000000 x20: 00000000004a3220 
[62682.094824] x19: 00000000004a4000 x18: 0000000000000020 
[62682.100180] x17: 0000007f91422fc8 x16: 0000007f91398324 
[62682.105547] x15: 0000007f91455cc0 x14: 0000000000000000 
[62682.110896] x13: 2f6372732f796172 x12: 782d656369766564 
[62682.116267] x11: 0000007fd4185c30 x10: 0000000000000088 
[62682.121668] x9 : ffffff80ffffffc8 x8 : 0000000000000040 
[62682.127025] x7 : 7f7f7f7f7f7f7f7f x6 : 00000000004a551b 
[62682.132464] x5 : 0000000000000000 x4 : 0000000000000000 
[62682.137834] x3 : 0000000000000000 x2 : 0000000000000001 
[62682.143185] x1 : 0000000000000001 x0 : 0000000000002710 

[62682.150049] vdso base = 0x7f91454000
[62718.172993] TestHardwareInt[9459]: unhandled level 2 translation fault (11) at 0x000027b5, esr 0x92000046
[62718.182617] pgd = ffffffc1e3d16000
[62718.186058] [000027b5] *pgd=000000026cefd003, *pud=000000026cefd003, *pmd=0000000000000000

[62718.196061] CPU: 2 PID: 9459 Comm: TestHardwareInt Tainted: G        W       4.4.38 #1
[62718.204013] Hardware name: quill (DT)
[62718.207726] task: ffffffc1ea82e400 ti: ffffffc07ab58000 task.ti: ffffffc07ab58000
[62718.215256] PC is at 0x7f9fdb2338
[62718.218588] LR is at 0x7f9fdb23f4
[62718.221946] pc : [<0000007f9fdb2338>] lr : [<0000007f9fdb23f4>] pstate: 80000000
[62718.229414] sp : 0000007fc8429880
[62718.232752] x29: 0000007fc8429890 x28: 0000000000000000 
[62718.238106] x27: 0000000000000000 x26: 0000000000000000 
[62718.243470] x25: 0000000000000000 x24: 0000000000000000 
[62718.248827] x23: 0000000000000000 x22: 0000000000000000 
[62718.254183] x21: 0000000000000000 x20: 00000000004a3220 
[62718.259549] x19: 00000000004a4000 x18: 0000000000000020 
[62718.264900] x17: 0000007f9fe3cfc8 x16: 0000007f9fdb2324 
[62718.270264] x15: 0000007f9fe6fcc0 x14: 0000000000000000 
[62718.275637] x13: 646e61207a483030 x12: 30382073756c7020 
[62718.281001] x11: 0000007fc8429830 x10: 0000000000000088 
[62718.286359] x9 : ffffff80ffffffc8 x8 : 0000000000000040 
[62718.291729] x7 : 7f7f7f7f7f7f7f7f x6 : 00000000004a551b 
[62718.297089] x5 : 0000000000000000 x4 : 0000000000000000 
[62718.302467] x3 : 0000000000000000 x2 : 0000000000000001 
[62718.307847] x1 : 0000000000000001 x0 : 0000000000002710 

[62718.314700] vdso base = 0x7f9fe6e000
[62776.256824] usbcore: deregistering interface driver uvcvideo
[62777.227780] uvcvideo: Found UVC 1.00 device Boson (09cb:4007)
[62777.235422] usbcore: registered new interface driver uvcvideo
[62777.241470] USB Video Class driver (1.1.1)
[62777.919294] tegra_mipi_cal 3990000.mipical: Mipi cal timeout,val:9881, lanes:300000
[62777.927013] tegra_mipi_cal 3990000.mipical: MIPI_CAL_CTRL                  0x04 0x2a000010
[62777.935321] tegra_mipi_cal 3990000.mipical: CIL_MIPI_CAL_STATUS            0x0c 0x00009881
[62777.943631] tegra_mipi_cal 3990000.mipical: CIL_MIPI_CAL_STATUS_2          0x10 0x00000000
[62777.951933] tegra_mipi_cal 3990000.mipical: CILA_MIPI_CAL_CONFIG           0x18 0x00200000
[62777.960220] tegra_mipi_cal 3990000.mipical: CILB_MIPI_CAL_CONFIG           0x1c 0x00200000
[62777.968503] tegra_mipi_cal 3990000.mipical: CILC_MIPI_CAL_CONFIG           0x20 0x00000000
[62777.976788] tegra_mipi_cal 3990000.mipical: CILD_MIPI_CAL_CONFIG           0x24 0x00000000
[62777.985064] tegra_mipi_cal 3990000.mipical: CILE_MIPI_CAL_CONFIG           0x28 0x00000000
[62777.993348] tegra_mipi_cal 3990000.mipical: CILF_MIPI_CAL_CONFIG           0x2c 0x00000000
[62778.001633] tegra_mipi_cal 3990000.mipical: DSIA_MIPI_CAL_CONFIG           0x3c 0x00000200
[62778.009906] tegra_mipi_cal 3990000.mipical: DSIB_MIPI_CAL_CONFIG           0x40 0x00000200
[62778.018168] tegra_mipi_cal 3990000.mipical: DSIC_MIPI_CAL_CONFIG           0x44 0x00000200
[62778.026427] tegra_mipi_cal 3990000.mipical: DSID_MIPI_CAL_CONFIG           0x48 0x00000200
[62778.034688] tegra_mipi_cal 3990000.mipical: MIPI_BIAS_PAD_CFG0             0x5c 0x00000000
[62778.042948] tegra_mipi_cal 3990000.mipical: MIPI_BIAS_PAD_CFG1             0x60 0x00000000
[62778.051210] tegra_mipi_cal 3990000.mipical: MIPI_BIAS_PAD_CFG2             0x64 0x00010010
[62778.059467] tegra_mipi_cal 3990000.mipical: DSIA_MIPI_CAL_CONFIG_2         0x68 0x00000002
[62778.067798] tegra_mipi_cal 3990000.mipical: DSIB_MIPI_CAL_CONFIG_2         0x6c 0x00000002
[62778.076340] tegra_mipi_cal 3990000.mipical: DSIC_MIPI_CAL_CONFIG_2         0x74 0x00000002
[62778.084717] tegra_mipi_cal 3990000.mipical: DSID_MIPI_CAL_CONFIG_2         0x78 0x00000002
[62779.210011] nvmap_alloc_handle: PID 9530: TestHardwareInt: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant. 
[62785.489972] usbcore: deregistering interface driver uvcvideo
[62785.908543] uvcvideo: Found UVC 1.00 device Boson (09cb:4007)
[62785.916410] usbcore: registered new interface driver uvcvideo
[62785.922177] USB Video Class driver (1.1.1)
[62788.024179] nvmap_alloc_handle: PID 9626: TestHardwareInt: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant. 
[62833.411427] usbcore: deregistering interface driver uvcvideo
[62833.808975] uvcvideo: Found UVC 1.00 device Boson (09cb:4007)
[62833.823268] usbcore: registered new interface driver uvcvideo
[62833.829082] USB Video Class driver (1.1.1)
[62835.501054] nvmap_alloc_handle: PID 9706: TestHardwareInt: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant.
4

@pwalter
Please upgrade to J4.2 this problem should be fixed by J4.2(r32.1)

5

@ShaneCCC
Thank you for that pointer! This means refreshing the Jetson correct? I can’t just download the latest release of L4T?

6

Flashing is required…this is in fact what would add the latest release of L4T. SDK Manager 4.2 is a front end to L4T R32.1. CUDA would be upgraded to version 10.