How to use the cv2.imshow() function with DeepStream SDK 5.0

Accelerated Computing Intelligent Video Analytics DeepStream SDK
#1

How to use the cv2.imshow() function with DeepStream SDK 5.0

I’ve heard that you can now access image data directly from DeepStream SDK 5.0.
Via pyds.get_nvds_buf_surface() we have confirmed that the image can be handled by OpenCV.

                n_frame=pyds.get_nvds_buf_surface(hash(gst_buffer),frame_meta.batch_id)
                frame_image=np.array(n_frame,copy=True,order='C')
                #covert the array into cv2 default color format
                print("ffs")
                frame_image=cv2.cvtColor(frame_image,cv2.COLOR_RGBA2BGRA)
                frame_image=draw_bounding_boxes(frame_image,obj_meta,obj_meta.confidence)

But I want the result to appear on the screen after processing in OpenCV.
So I want to use cv2.imshow(), but it stops at the cv2.waitKey(1) part.
Is there a way to display the results of processing in OpenCV?

Translated with www.DeepL.com/Translator (free version)

#3

Hi @natukage405,
please refer to https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/blob/master/apps/deepstream-imagedata-multistream/deepstream_imagedata-multistream.py#L121

#5

I’m making it with that as a reference.
I’m having trouble because it doesn’t show up when I run cv2.imshow(“Test”,frame_image) after get_nvds_buf_surfac()

#6

Hi natukage405, we are looking into it, and will get back to you.

#7

Hi @natukage405,
can you save the image with cv2.imwrite()?
With below chnage, I can save the image as png and view it with display.
Would check if it’s your display issue or others.

Thanks!

@@ -123,6 +126,7 @@ def tiler_sink_pad_buffer_probe(pad,info,u_data):
                     frame_image=np.array(n_frame,copy=True,order='C')
                     #covert the array into cv2 default color format
                     frame_image=cv2.cvtColor(frame_image,cv2.COLOR_RGBA2BGRA)
+                    cv2.imwrite("test.png", frame_image)

                 save_image = True
                 frame_image=draw_bounding_boxes(frame_image,obj_meta,obj_meta.confidence)
#8

I can confirm that the image can be saved with cv2.imwrite() and that I can reference the saved image.
But what I want to do is to display the image using cv2.imshow().
You need to investigate.

#11

I have the same problem as natukage405, I want to display real-time frame by using cv2.imshow() but when I use it in the code (in deepstream_test1_usbcam in python examples), it does not work. I need the cv2.imshow() output for using it in another GUI.
I appreciate it if you could help me with this issue.

#12

This is still on our list to investigate. Thanks for your patience.

#14

Hi @zhliunycm2 @mchi , this is an issue for my use case too, kindly clarify if there is any resolution for the same.

#15

Hi @rsnk96
Why can’t use the display DeepStream currently provides?
Can you elabrate your use case and why this is must-have ?

#17

Because they might need to make some transformations. I think the point is showing a custom image (~custom overlay, shrink etc.). I was running into this same issue. Do you have any solution yet? Thank you.

#18

Hi benkelaci,

Please help to open a new topic for your issue. Thanks

#19

My problem is completely the same as the original topic, but you do not have any good solution (cv2.imshow with deepstream), just a workaround (that is sometimes ok, sometimes not).

#20

Hi @benkelaci,
We have render plugin, may I know why imshow() is needed? What’s the use case?

#21

e.g.: a custom drawing/texts on the screen. The default deepstream drawing is not optimal in many cases (especially if there are overlapped objects. E.g.: if I would add text that shows average of the last few object count etc. Can I use custom drawing on render plugin?