Thanks, so good.
My last question is that:
I run the one sample of deepstream-app for 8 rtsp streams, and show this results:
Q1- Average FPS for each stream is about 27, that’s mean the nano process each stream with 27 fps at the same time?
Q2- It runs one model for all streams, It captures one batch (included 8 frames of all streams) and then inference on one batch? If so, do process one batch at the same time or do loop for each frame of batch?
Q1- Average FPS for each stream is about 27, that’s mean the nano process each stream with 27 fps at the same time? ===> yes
Q2- It runs one model for all streams, It captures one batch (included 8 frames of all streams) and then inference on one batch? If so, do process one batch at the same time or do loop for each frame of batch? ==> one inference shot processes one batch(8 frames) at the same time.
Thanks for your quick answers.
Q1- Is is possible to run custom trained model like face recognition with deep stream 5.0 DP?
Q2- I run this sample, and I want to don’t show me tiled window, I want to do process in background with showing anything, How do O do?
Q1- Is is possible to run custom trained model like face recognition with deep stream 5.0 DP?
Yes, DeepStream wraps TensorRT and Triton, so any model that can run with TensorRT or Triton can run on DeepStream. Normally, using TensorRT can get better performance.
I want to do process in background with showing anything
I don’t get your point, could you clarify?
I run this code, I want to use only decoded frames to do for processing, but this code also shows visualization, but I want to disable visualization.
you can change
sink = Gst.ElementFactory.make(“nveglglessink”, “nvvideo-renderer”)
to
sink = Gst.ElementFactory.make(“fakesink”, “nvvideo-no-renderer”)
Thanks a lot.
Q1 - If I want to use the decoded frames in the custom python app, How do I do?
In this function of deepstream-imagedata-multistream, the decoded frames can be captured with frame_image and I want to put the frames into queue and use queue in the processing units. and I also modified this function. But when I put the queue part of codes in this function, running this decoder is broke. why this happen? because I can’t use the queue in this part?, if so How I capture the frames of streams and use them in the processing units?
def tiler_sink_pad_buffer_probe(pad,info,u_data):
frame_number=0
num_rects=0
gst_buffer = info.get_buffer()
if not gst_buffer:
print("Unable to get GstBuffer ")
return
batch_meta = pyds.gst_buffer_get_nvds_batch_meta(hash(gst_buffer))
l_frame = batch_meta.frame_meta_list
while l_frame is not None:
try:
frame_meta = pyds.NvDsFrameMeta.cast(l_frame.data)
except StopIteration:
break
frame_number=frame_meta.frame_num
n_frame=pyds.get_nvds_buf_surface(hash(gst_buffer),frame_meta.batch_id)
frame_image=np.array(n_frame,copy=True,order='C')
frame_image=cv2.cvtColor(frame_image,cv2.COLOR_RGBA2RGB)
if not q.full():
q.put(frame_image)
else:
q.get(frame_image)
fps_streams["stream{0}".format(frame_meta.pad_index)].get_fps()
try:
l_frame=l_frame.next
except StopIteration:
break
return Gst.PadProbeReturn.OK
Q2 - Because my input frame rate in high than output frame rate. i.e input rate = 25 FPS and outptu rate = 5 fps, because of I want to capture every 5th frame, and nvv4l2decoder has the drop-frame-interval options, I want to know this sample code used nvv4l2decoder, or uridecodebin ? How I can use drop-frame-interval option in this sample code?
what failure? can you provide the fialure log and narrow down which code cause this failure?
Ok, I will show the logs.
1- If I want to debug every element output, like nvinfer for detection, How do I do? I want to get output of nvvinfer like the frame_image data of streamermux, Is it possible?
2- What’s difference between programming and inference like this sample codes and python custom programming? this element of deepstream plugins speed up the inference performance?Is so, whereas this element of deepstream plugins like nvinfer / streamermux / sink/ … are run in sequential state?
3- The deepstream plugins are run as asyncio?
For #1, yes, you can check the source code under /opt/nvidia/deepstream/deepstream-5.0/sources/gst-plugins/gst-nvinfer/, e.g. the meta code - gstnvinfer_meta_utils.cpp
For #2,
What’s difference between programming and inference like this sample codes and python custom programming?
Don’t understand this question.
this element of deepstream plugins speed up the inference performance?
which element? do you mean nvinfer? If you do, the answre is YES
Is so, whereas this element of deepstream plugins like nvinfer / streamermux / sink/ … are run in sequential state?
For one frame data, yes.
For different frames, they can run in parallel.
For#3, GStreamer is an asynchronous framework
Thanks a lot.
For one frame data, yes.
For different frames, they can run in parallel.
1- What’s mean about different frames? your mean is multi-stream?
I want to right understand about the Gstreamer. please correct my mistake if there.
2- Suppose my program or system has 5 steps, and these steps want to work together as pipeline, then I implement the codes of each steps in c++/python, then gsteamer link these steps and run these steps in asynchronous/pipeline? If so, What’s advantage of this way with Gsteamer? assume in the python async, we can implement each part of steps then connect each steps of them together, and to work together as pipeline? right?
3- Gsteamer work like python asyncio but in c++?
4- Is it possible to implement custom gsteamer plugins? only support c++? or also python? Is it possible to convert a python function or module to gstreamer plugin and run in gstreamer pipeline?
5- what’s difference between implement a system with gstreamer plugins and python async implements?
1- What’s mean about different frames? your mean is multi-stream?
No, the different frame in the same stream, e.g. nvinfer process frame#n, decoding can process frame#(n+1), so they can run in paralle,
Regarding gstreamer itself questions, you could search gstreamer related reference doc and sample on Google.
Hi @mchi, Thanks so much to spend a lot time for guidance me.
1- I investigate the gstnvinfer_meta_utils.cpp, I want to know, Is it possible to change and modify these codes?
2- nvinfer plugin use gstnvinfer_meta_utils.cpp? I want to know If I want to little modify the codes, I have to change only this file code?
3- Is it possible to skip the some classes of object detection? Suppose I want to use PeopleNet and this model has person/face/bag detection and I want to skip the bag class, How do I do? I have to modify gstnvinfer_meta_utils.cpp?
4- Is it possible to add some other task to nvinfer plugin? suppose I want to add face recognition model after peoplenet model, Is it possible? If so, I also need to modify gstnvinfer_meta_utils.cpp?
5- In this picture, they said deep stream is for both developer and enterprises, I want to know how we can the modify and add custom models to deep stream, really it is possible? In opinion, the deep stream is hard code and close-source, I want to know when this SDK is close-source, How we can modify the codes?
Deepstream offers the flexibility for rapid prototyping to full production level solutions and greater flexibility by allowing you to choose your inference path. With native integration to NVIDIA Triton Inference Server, you can deploy models in native frameworks such as PyTorch and TensorFlow for inference or achieve the best possible performance using NVIDIA TensorRT for high throughput inference with options for multi-GPU, multi-stream and batching support.
6- Is it possible to use custom detection or any models in deep stream?
you can deploy models in native frameworks such as PyTorch and TensorFlow for inference
7- It’s adaptive only to Triton or also TensorRT?
best possible performance using NVIDIA TensorRT for high throughput inference
8- When TensorRT gives more performance, Why need to use Teriton?
what’s your use case?
Peoplenet
there is Peoplenet tlt DS sample in DS5.0.
Hi all, @mchi,
Q1- Is it possible to run custom models such as detection trained with tensorflow/keras/pytoch framework on deepstream? Or the SDK currently support only TLT models?
Q2- Converting the TLT model (.etlt) to TensorRT engine with tlt-convertor, How I can specify the model convrted to GPU/DLAs? with trtexec we can set –useDLACore=0 or --useDLACore=1 --allowGPUFallback=true, but with the tlt-converter we have:
tlt-converter model.etlt \
-k key \
-o output_cov/Sigmoid,output_bbox/BiasAdd \
-d 3,544,960 \
-i nchw \
-m 1 \
-t fp16 \
-e model.engine
How we can specify the model to be converted to GPU or DLAs?
yes, you can convert it to onnx model and use it in DeepStream with dsnvinfer (TensorRT).
or you can use the model trianed by tf, direcetly with dsnvinferserver (Triton)
TLT DLA support will be added in next release. @Morganh to confirm
@LoveNvidia
For your Q2, please set as below bold font in deepstream config file.
[property]
gpu-id=0
net-scale-factor=1.0
offsets=103.939;116.779;123.68
model-color-format=1
labelfile-path=label.txt
tlt-encoded-model=your.etlt
tlt-model-key=your_keyenable-dla = 1 or 0
use-dla-core = 1 or 0
Q1- Why I convert the etlt model with tlt-converter, that logs some layers with DLA and some layer with GPU, I want to know with currently version of TLT(2.0) , converting etlt to tensortrt egine file doesn’t use DLA? only use GPU?
Q2- Is it possible to convert the etlt model to tensortrt engine file with trtexec with capability DLA support?
Q3- The TensorRT is preferred than Triton In terms of performance?