deepstream in python inquires

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

hi,

i have two inquires about deepstream in python.

first: how do i get original frame info such as, bbox and label to do something with it?

second: how do i use deepstream in python to just read video file or rtsp stream without inference and capture those frames to do something with the frames. i want to make use of the accelerated video decoding for mult stream.

thanks

2

First:
Can you refer to osd_sink_pad_buffer_probe in samples. frame info is in batch_meta → frame_meta → obj_meta

Second:
Why do you not need nvinfer? You have your own inference code ?

3

thanks for the response.

yes, i have my own inference and tracking code.

Currently, i am using opencv for two streams on jetson tx2, which is not fast enough. so i need to make use of the gstreamer nvidia acceleration plugins.

4

Second: Can you refer to python/apps/deepstream-test3 which support multiple streams.
I suggest you use full python-deepstream with nvinfer (I think it’s not hard), but not only use decoder part. We will think about how to get YUV buffer from python API.

5

I agree with you, it is not hard to make it run. but my use case is a bit complicated and its hard to integrate it with deepstream when i dont fully understand how it works.

perhaps you can help me?

generally speaking, i have two cameras one is responsible for tracking objects and another zoomed in camera for detecting features about those objects.i have set the zoomed in area as a region of interest on the tracking camera so that when a specific object is within the ROI i give a queue to the zoomed in camera to pass frames to the feature detector and do its job.

how is it possible to customize deepstream to do this? if it is not possible in python deepstream yet, please tell how i can just use the decoder to capture frames.

thanks

6

I think one camera is OK ? Who provide original object bbox for tracking?

Deepstream also has nvtracker.
And PGIE (detection) can filter object according to roi “roi-top-offset=0” “roi-bottom-offset=0”

You can check samples/configs/deepstream-app/config_infer_primary.txt
./sources/gst-plugins/gst-nvinfer/gstnvinfer_meta_utils.cpp → attach_metadata_detector → roiTopOffset

7

second camera is special camera with specific config to be able to obtain close and clear footage under different circumstances and it is not fit for tracking so i am stuck with two cameras setup for best results.

i understand that deepstream has its own tracker i have tested test-app-2 and it looks nice. i am willing to use it if only i know how to capture the output and further process it to do something with it. like for example pass to the second camera model that i currently have object-x with id y within the region of interest.

when ever i am trying to get the batch and frame meta i have some location on the memory printed and i cant find a way to figure the real data.

8

The alpha release of pyds doesn’t support image data access so the user can’t get decoded frames yet. For now they can try this: Use NVIDIA DeepStream to Accelerate H.264 Video Stream Decoding | by zong fan | Medium

9

i have seen this medium thread before, i could not pull more than 15 fps for video decoding only, which is much slower than opencv.

can you point to how i can get decoded frames in deepstream c++

thanks

10

Please check https://devtalk.nvidia.com/default/topic/1061492/deepstream-sdk/deepstream-sdk-faq/ → 2. How to get original NV12 frame buffer

11

Hi @ChrisDing,
How can I get opencv image (numpy array) in function - deepstream-test3:

tiler_src_pad_buffer_probe(pad,info,u_data)?

1 Like
13

Please help to open a new topic for your issue. Thanks

1 Like