I looking forward to connect with deepstream with tritonserver(nvinferserver) directly without using grpc or http/rest. I already created ensemble the models in “model_repository”. Wondering how to write like a “deepstream-test3 config_triton_infer_primary_peoplenet.txt” base on my ensemble models. How can I achieve it. please be advice.
model_repository
face_detection_pre
face_detection
face_detection_post
ensemble_face_detection(pre/detection/post)
In “ensemble_face_detection” include “face detection_pre”,“face_detection” and “face_detection_post” these are written by python backend. I want to connect “ensemble_face_detection” with deepstream+triton same container.
Software version : Deepstream SDK 6.3
Container version : nvcr.io/nvidia/deepstream 6.3-triton-multiarch
Does it support for python backend?When i look readme file inside of that you mention only explain about Triton C++ backend.Pls correct me if i miss something.
DeepStream nvinferserver plugin leverages Triton to do inference. Triton is opensource and supports Python backend. Please refer to python ensemble bakend sample.
I read sample link that is not connect with nvinferserver plugin, use via tritongrpcclient.InferenceServerClient(grpc).if not support to nvinferserver directly on python backend, how to link deepstream pipeline with tritongrpcclient?pls
Please read nvinferserver introdcution first. nvinferserver is a Gtreamer plugin which leverages Triton to do inference. Triton supports many kinds of backend including Python backend.
please refer to deepstream_test1_rtsp_in_rtsp_out.py for how to use nvinfersrver.
please refer to nvdsinferserver_custom_impl_ensemble for how to use ensemble. It is TRT backend. there is no Python ensemble backend sample in DeepStream.
Please refer to other user’s implements. this code is a python backend sample. this code is a python ensemble backend sample.