Getting image to a pointer from frame without memory copy

[anugrah]

Hi,
I’m new to Nvidia Jetson platform and Argus.
I have to get image from camera sensor (MIPI CSI) and paint it to gtk window.
Previously the painting was done in qt where the multicamera sample is modified to create an nvbuffer and copy the contents in to a file descriptor from which it would be read to a pointer variable pdata as follows:

UniqueObj frame(iFrameConsumer-> acquireFrame());

         iFrame = interface_cast< IFrame>(frame);
         //std::cout<<iFrame<<std::endl;
         image = iFrame->getImage();

          iNativeBuffer = interface_cast<NV::IImageNativeBuffer>(image);


         if(!fd18)
         {
         fd18 = iNativeBuffer->createNvBuffer(STREAM_SIZE, NvBufferColorFormat_ARGB32, NvBufferLayout_Pitch);
         }
         else if (iNativeBuffer->copyToNvBuffer(fd18) != STATUS_OK)
         {
                    //    ORIGINATE_ERROR("Failed to copy frame to NvBuffer.");
         }

         void *pdata = NULL;
         NvBufferParams params;
         NvBufferGetParams(fd18, &params);
         NvBufferMemMap(fd18, 0, NvBufferMem_Read, &pdata);
         NvBufferMemSyncForCpu(fd18, 0, &pdata);
         //std::cout<<params.pitch[0]<<std::endl;
         int b1 = params.pitch[2];
         //std::cout<<"This is the pitch1"<<b1<<std::endl;


         sync_obj->setData((unsigned char*)pdata);

My question is how would I get the image frame to a pointer (which i can pass to opencv mat or use for any other purpose) without copying the frame to file descriptor and then copying the fd to pdata - which is not at all efficient.

I know Iam a noob and any help would be apprecieated.

Thanks

[JerryChang]

hello anugrah,

may I know what’s the actual use-case, please share some details of your capture frame usage.
thanks

[anugrah]

Hi,
My use case is to capture image from a CSI/MIPI camera sensor, use argus to perform some hardware isp functions (Edge enhance, exposure white balance etc) and transfer the frame to an ope cv mat for doing other operations (such as color conversion etc) and to finally stream it to a window (GTK).
After the previously mentioned code segment, the pointer is send to another thread using a signal for doing the post processing using opencv. I have to implement this in a new platform (GTK) while previously qt was used.

Browsing through the forums I came to know there was an object identification sample using open cv (11_camera_object_identification) but for some reason its removed from the installation.

I would like to know if there is some sample I could refer to capture frame using argus and transfer it to open cv mat efficiently with as little memcpy as possible.

Thanks in advance

[JerryChang]

hello anugrah,

please see-also this topic, Topic 83012, there’s a patch to modify the sample for using cv::Mat

[anugrah]

Hi Jerry,
Sorry for the late reply.
The patch you mentioned works.
Thankyou,
Regards
Anugrah

[anugrah]

I had another problem and would be great if you could give some insight to it.

Im working with the multicamera sample and implemented the opencv mat part successfully.
Now when I try to run a GTK window in the same main program, it hangs.

• Tried argus in a separate thread: not working

• Tried GTK in separate thread: not working

Without threading, one of the process (ArgusSamples::execute() or gtk::app->run() ) blocks the other, depending on which comes first.

heres is the rough code:

#includes

//GTK part

cv::Mat display_img;
GtkWidget *window;
GtkWidget *button;
GtkWidget *halign;
void * gtkThread(void * args){
    window = gtk_window_new(GTK_WINDOW_TOPLEVEL);
    gtk_window_set_title(GTK_WINDOW(window), "Mnemonic");
    gtk_window_set_default_size(GTK_WINDOW(window), 300, 200);
    gtk_container_set_border_width(GTK_CONTAINER(window), 15);
    button = gtk_button_new_with_mnemonic("_Button");
    halign = gtk_alignment_new(0, 0, 0, 0);
    gtk_container_add(GTK_CONTAINER(halign), button);
    gtk_container_add(GTK_CONTAINER(window), halign);
    gtk_widget_show_all(window);
    g_signal_connect(G_OBJECT(window), "destroy",
                     G_CALLBACK(gtk_main_quit), NULL);
    gtk_main();
}

//Argus sample multicamera classes and functions with patch applied

//main loop

int main(int argc, char ** argv)
{
// if (!parseCmdline(argc, argv))
// {
// printHelp();
// return EXIT_FAILURE;
// }
XInitThreads();
gtk_init(&argc, &argv);
pthread_t t3;
pthread_create(&t3, NULL, &gtkThread, NULL);
bool ex = 0;
ex = ArgusSamples::execute();
while(1)
{
//if(!ex)
// ex = ArgusSamples::execute();
printf(“In while\n”);
}

My question is does argus blocks other threads while in main loop?
if yes is there is any way to run gtk thread inside the main?

Any comments would be appreciated
Thanks

[JerryChang]

hello anugrah,

this topic has been marked as solved, please file a new topic to follow-up the issue,
thanks