• Hardware Platform (GPU)
• DeepStream Version 6.0/6.1
• TensorRT Version 8.2.5-1
• NVIDIA GPU Driver Version 515.43.04/ Cuda 11.7
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue? Use an external Triton server for inference.
This has been tested on multiple versions of Cuda/Drivers.
As far as I am aware, currently the bug we are facing is due to switching to the grpc url for an external triton server. This only happens on the models that have a reshape in the config.pbtxt, the reshape adds an extra dimension which is needed to satisfy the triton server, otherwise it won’t accept the config. I believe this then confuses the Deepstream Triton client, as it attempts to preprocess the input shape to match the aforementioned reshape, when the model is only expecting 3 dimensions for input.
Classifier error:
ERROR: infer_grpc_client.cpp:342 inference failed with error: unexpected shape for input ‘INPUT__0’ for model ‘jigsaw_contraction’. Expected [3,448,448], got [1,3,448,448]
Detector error:
python3: infer_cuda_utils.cpp:86: nvdsinferserver::CudaTensorBuf::CudaTensorBuf(const nvdsinferserver::InferDims&, nvdsinferserver::InferDataType, int, const string&, nvdsinferserver::InferMemType, int, bool): Assertion `!hasWildcard(dims)’ failed.
This error does not appear within the default Triton server set-up inside of Deepstream.
The error also does not appear when I use my own client to communicate with the server and set the shape myself.
For example:
service_pb2.ModelInferRequest().InferInputTensor().shape.extend([3, 448, 448])