HTTP Error while accessing Model API from Deepstream - Triton Server

Deepstream Triton Docker:

Serving Triton Models Repository : trtserver --model-repository=/models/ --log-verbose=1 --strict-model-config=False

Note : In Triton server the command to serve models is
tritonserver --model-repository=/models/ --strict-model-config=False --log-verbose=1 --strict-model-config=False

and in deepstream its “trtserver”

Issue : HTTP error when trying to access localhost docker apis from triton-perf container
APIs loaded and running Logs :

I1102 07:27:58.614903 522] Started GRPCService at
I1102 07:27:58.614922 522] Starting HTTPService at
I1102 07:27:58.656984 522] Starting Metrics Service at

Trying to access Model API Endpoints:
Command : perf_client -u localhost:8000 -m model_name -b 1 --concurrency 1
Error Output : error: failed to get model metadata: failed to parse the request JSON buffer: The document is empty. at 0

When I check the triton Logs, I get these outputs:

I1102 07:28:12.073094 522] HTTP request: 0 /v2/models/model_name
I1102 07:28:12.073159 522] HTTP error: 0 /v2/models/model_name - 400

I don’t have these issues when I use triton server. Why is this issue arising with Deepstream-Triton? How can I resolve this?


Additional things to confirm have you exposed your DS-TRT container ports?
If you are trying by perf client then have you used --net=host while running the container?

Perf Client Command

sudo docker run -it --net=host -v /local_dir:/mount

DS-Triton docker command

nvidia-docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -v /local_dir:/mount -e DISPLAY=$DISPLAY -p8000:8000 -p8001:8001 -p8002:8002

Internal bug tracking id : 200673911

It’s about perf_client and tritonserver(trtserver) API-version mismatched.
deepstream:5.0.1-20.09-triton is based on under API-v1. Suggest perf_client stay on same version
docker pull is a beta version for API-v2(not stable). It’s not suggested to use for deepstream.