NVIDIA GPU Driver Version - driver version - 450.51 with cuda - 11.0, cudnn - 8.1.0
Issue Type - questions
Requirement details -
Currently in our inference pipeline, we are restarting the camera whenever we get any frame none. Is this feature available in deepstream to restart the camera if we are not able to read any frame (if camera goes down).
We have pre-processing and post processing part for our detection pipeline. Can we customize this part in Deepstream.
How can we pass the output from nvinfer( frames and predictions) to python script to further process the results (Is there will be any latency issues in later stages).
How can we convert the Deepstream supported format(RGBA/BGRx) to normal RGB/BGR without any change in image quality
What kind of pre-processing and post-processing? What kind of model are you using?Detector, classifier, segmentation or anything else? What is the input and output of the model?
What kind of pre-processing and post-processing? --We are doing Image preprocessing, normalization and transformation in the pre-processing part. Post processing includes converting bounding box coordinates from YOLO format to original, NMS and filtering top predictions per class(Class ID, bounding box, confidence).
What kind of model are you using? Detector, classifier, segmentation or anything else? - Detection, YOLOv5
What is the input and output of the model? - Input : Tensor image, Output : Tensor output of size (1, 16125, 106) â 106 (4 bbox, confidence and 101 confidence per class).
If there is only format change from RGBA/BGRx to RGB/BGR, there is no image quality change with or without deepstream. - there is change in colour for original image when converted from RGBA to RGB( we are using cv2.COLOR_RGBA2RGB)
4.It is just format change, the color will not be changed. With deepstream, all video and images buffers are HW buffers, and the color transformation is done by HW. There is no NV deepstream component can output RGB/BGR data.
deepstream_python.py (14.0 KB)
I am able to run pipeline for multiple RTSP sources. But when I tried to convert the frames from Gst Buffer to numpy array using the following lines:
n_frame=pyds.get_nvds_buf_surface(hash(gst_buffer),frame_meta.batch_id)
frame_image=np.array(n_frame,copy=True,order=âCâ)
frame=cv2.cvtColor(frame_image,cv2.COLOR_RGBA2BGRA)
I got the following error:
Traceback (most recent call last):
File âdeepstream_python.pyâ, line 81, in tiler_src_pad_buffer_probe
n_frame = pyds.get_nvds_buf_surface(hash(gst_buffer), frame_meta.batch_id)
RuntimeError: get_nvds_buf_Surface: Currently we only support RGBA color Format
I tried to put the nvvideoconvert and capsfilter before tiler to convert the frames to RGBA.
nvvidconv1 = Gst.ElementFactory.make(ânvvideoconvertâ, âconvertor1â)
if not nvvidconv1:
sys.stderr.write(" Unable to create nvvidconv1 \n")
print(âCreating filter1 \n â)
caps1 = Gst.Caps.from_string(âvideo/x-raw(memory:NVMM), format=RGBAâ)
filter1 = Gst.ElementFactory.make(âcapsfilterâ, âfilter1â)
if not filter1:
sys.stderr.write(â Unable to get the caps filter1 \nâ)
filter1.set_property(âcapsâ, caps1)
But still got the same error.
You are using cv2.cvtColor(frame_image,cv2.COLOR_RGBA2BGRA) in tiler_src_pad_buffer_probe, and tiler_src_pad_buffer_probe is probed in nvstreammux src pad. Why do you think âto put the nvvideoconvert and capsfilter before tiler to convert the frames to RGBA.â can help to convert the data in nvstreammux src pad to RGBA?
deepstream_test.py (16.4 KB) @Fiona.Chen We are trying to setup up a pipeline where we want numpy frames from Deepstream after Decoding and aggregation.
We tried the following script for that( We are able to convert Gst-Buffers to numpy ) at later stage we want to pass these frames to our detection pipeline. So we just want a simple function which will take the gst-Buffers as input and return the numpy frames for detection pipeline. (If you have any better solution for this please suggest).
Currently I want to disable the output stream in the script. How to do that?
What is your detection pipeline? A separated process different to deepstream app?
There is already a detector in deepstream_test.py, do you mean you will need a second detector?
2. Do you mean you donât want to display the output?
@Fiona.Chen
What is your detection pipeline? A separated process different to deepstream app?- Yes we are not using the nvinfer from deepstream. We are passing the frames(numpy) to our custom detection pipeline.
2. Do you mean you donât want to display the output? - Yes we donât want to display the output we just want the frames.
So you just use deepstream to generate tiling frames(no inference is needed in deepstream) and want to send the frames to another process? If so, you donât need to use deepstream.
Just remove nvegltransform and nveglglessink, replace with fakesink will work.
So you just use deepstream to generate tiling frames(no inference is needed in deepstream) and want to send the frames to another process? If so, you donât need to use deepstream. - We want the GPU accelerated decoding for RTSP source thatâs why we thought of using deepstream for decoding.
Just remove nvegltransform and nveglglessink, replace with fakesink will work. â Our platform is not Jetson so we are not using nvegltransform and we already replaced the nveglglessink to fakesink in the script(output display is still there).
Back to the original question. We have no special suggestion for this because deepstream has no restriction for any way to transferring your data. The numpy data is just a normal python data, has nothing to do with deepstream, you can use them as you like.
If there is only fakesink in the pipeline, there will be no display.
Can you elaborate this with details? What is the scenario? Deepstream is a SDK, it is not an application. What do you mean to ârun the Deepstreamâ? For inference several frames in a video or something else?
Suppose I want to do the inference in deepstream-python. So it will do the inference until we terminate manually or there is any error in pipeline.
If we want to run the inference for letâs say 10000 frames. Is there any way to terminate the process after certain number of frames ?
2. If we are using deepstream-python how to restart the camera in between process( do we need to initialize the all the source elements again if we encounter any of the RTSP source is not working) ?
You can stop the rtspsrc rtspsrc by change the element state to NULL. And when you want the stream to start again, you can change the state to PLAYING again. No need to initialize if the ip camera is always working. BTW. Since there are very complicated threads management in gstreamer, you need to maintain the pipeline and element states very carefully in your application. And gstreamer python is not easy for debugging since all working parts are implemented in c/c++, it is better to use c/c++ instead to use python.
These are very basic gstreamer development skills, you can refer to Gstreamer resources for more information and samples.
When we changed the source state from PLAYING â NULL the pipeline stopped for source but when we changed it back to PLAYING the pipeline is unable to initialize the source again.
When we changed the source state from PLAYING â PAUSED the pipeline didnât stop for source(we are still getting the frames)