Hello
actually i am building a following pipeline
PGIE → SGIE1 → preprocess → SGIE2 → custom post processing.
Now i have used a cv::Mat (2x3 Matrix) object in preprocess plugin and now
i want to use that Matrix again in custom post processing of SGIE2 so how can i obtain it.
I also want the full frame that was input for PGIE at the custom post processing of SGIE2 because i want to add some custom image on that so how can i obtain the full input frame at the custom post processing part.
my PGIE is face detection (instance segmentation) model so i am already using object_meta to show keypoints and my SGIE2 is face swap model which generates images so how can i use them and the matrix to perform inverse cv operations and then attach the face image to the full frame.
i tried it but i was not getting the data in the gie_processing_done_buf_prob function of the deepstream-app.c (where i was already attaching the kps of my PGIE) instead when i was accessing the data i was getting garbage values
yes face is swapped but the input and output images of the model are just facial area cropped images instead of full frame that’s why i have to post process the output image using cv and attach it on the initial full frame.
so i wanted to know HOW and WHERE can i access all these following three things at the same place so that i can do the post processing:
my swap models facial swapped output image
matrix that i used in the preprocess plugin for affine transformation (i need this to perform inverse warp affine)
You need to parse that from the output of your model. Please refer to the sources\apps\sample_apps\deepstream-infer-tensor-meta-test to learn how to parse the output tensor data.
Please learn how to use our NvDsUserMeta to transport messages from our sources\apps\sample_apps\deepstream-test4
You can get that from any gstbuffer, like pgie_src_pad_buffer_probein our sources\apps\sample_apps\deepstream-image-meta-test demo.
hey thanks for your help i am able to get the swapped full frame image but i am not getting all faces swapped at the same time
following is the code that i am using by modifying the deepstream-app deepstream-app_mod.txt (16.2 KB)
now i should get all the faces swapped in the frame but i am getting only single face swapped each time
My l_frame loop iterates once, l_obj twice (one for kps and one for bbox by another model), l_user_meta once for the current swapped image generated by my face swap model
so i wanted to know if there is something missing or is there any logical issue present…just want to tell you that i am getting proper bboxes and proper kps and the norm_crop2 function i am using because for blending purpose i needed the image that was given to my swap model as input but i couldn’t obtain it even as user meta so i performed the same operation here and i verified that the images are same
yes my model is working properly as initially in custom parser i was saving all the images but now i am directly taking the output tensor meta in the deepstream-app.c file and also my model does not need any sepcific post processing just scaling of pixel values to 0-255.
You need to make sure that the outputs of your model includes everything you need. If not, you need to check the inputs and outputs of the model in our source code yourself.
hello thanks for your help!!
i was able to send and receive the matrix using objc_meta->misc_obj_info and then as you said i got the full frame image from the buffer and atleast i got the model outputs using output tensor meta flag…and actually there was a small logical error because of which i was not getting all the matrices and that too i solved and now i am getting all the faces.