HTTP Error while accessing Model API from Deepstream - Triton Server

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Deepstream Triton Docker: nvcr.io/nvidia/deepstream:5.0.1-20.09-triton

Serving Triton Models Repository : trtserver --model-repository=/models/ --log-verbose=1 --strict-model-config=False

Note : In Triton server the command to serve models is
tritonserver --model-repository=/models/ --strict-model-config=False --log-verbose=1 --strict-model-config=False

and in deepstream its “trtserver”

Issue : HTTP error when trying to access localhost docker apis from triton-perf container
APIs loaded and running Logs :

I1102 07:27:58.614903 522 grpc_server.cc:1939] Started GRPCService at 0.0.0.0:8001
I1102 07:27:58.614922 522 http_server.cc:1411] Starting HTTPService at 0.0.0.0:8000
I1102 07:27:58.656984 522 http_server.cc:1426] Starting Metrics Service at 0.0.0.0:8002

Trying to access Model API Endpoints:
Command : perf_client -u localhost:8000 -m model_name -b 1 --concurrency 1
Error Output : error: failed to get model metadata: failed to parse the request JSON buffer: The document is empty. at 0

When I check the triton Logs, I get these outputs:

I1102 07:28:12.073094 522 http_server.cc:456] HTTP request: 0 /v2/models/model_name
I1102 07:28:12.073159 522 http_server.cc:509] HTTP error: 0 /v2/models/model_name - 400

I don’t have these issues when I use triton server. Why is this issue arising with Deepstream-Triton? How can I resolve this?

Thanks.

2

Additional things to confirm have you exposed your DS-TRT container ports?
If you are trying by perf client then have you used --net=host while running the container?

4

Perf Client Command

sudo docker run -it --net=host -v /local_dir:/mount nvcr.io/nvidia/tritonserver:20.06-py3-clientsdk

DS-Triton docker command

nvidia-docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -v /local_dir:/mount -e DISPLAY=$DISPLAY -p8000:8000 -p8001:8001 -p8002:8002 nvcr.io/nvidia/deepstream:5.0.1-20.09-triton

5

Internal bug tracking id : 200673911

6

It’s about perf_client and tritonserver(trtserver) API-version mismatched.
deepstream:5.0.1-20.09-triton is based on nvcr.io/nvidia/tritonserver:20.03-py3 under API-v1. Suggest perf_client stay on same version
docker pull nvcr.io/nvidia/tritonserver:20.03-py3-clientsdk

nvcr.io/nvidia/tritonserver:20.06-py3 is a beta version for API-v2(not stable). It’s not suggested to use for deepstream.