Unable to run Inference on Tensorflow Hub models that do not support batching with Triton InferServer

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.0
• TensorRT Version 8.0.1
• NVIDIA GPU Driver Version (valid for GPU only) 440.33.01

**• Steps to reproduce the issue **

  • Create a container from the image : ‘nvcr.io/nvidia/deepstream:6.0-triton’ using the below command sudo docker run -it --gpus all -p 8554:8554 -w /opt/nvidia/deepstream/deepstream-6.0
  • Unzip the attached folder in the current working directory (deepstream-6.0).
  • Go to efficientdet folder.
  • Run the following command python3 main.py file:///opt/nvidia/deepstream/deepstream-6.0/efficientdet/data/sample_720p.h264
  • The screenshot of the error is attached.
  • Note : We are facing this problem with tensorflow models that do not support batching, whereas with models that support dynamic batching this problem disappears.
    efficientdet.zip (45.2 MB)

Any update regarding this issue?

Sorry for the late response, we will do the investigation to have the update soon. Thanks

Ok. Awaiting for your response!

Please refer to NVIDIA-AI-IOT/deepstream_tao_apps: Sample apps to demonstrate how to deploy models trained with TAO on DeepStream (github.com)

Thanks for the update.
But we want to deploy any TensorFlow 2 hub Model ( Not only efficientdet ) using triton-inferserver ( nvinferserver ). We observed Models that don’t support batching cannot be deployed with triton-inference.
We would appreciate any help in this regard. Thanks!

Any updates ?