Triton server ONNX support

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) T4
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.0
• NVIDIA GPU Driver Version (valid for GPU only) 11.4
• Issue Type( questions, new requirements, bugs)

Is there a way , i can have pre-complied custom lib parsers of ONNX models for the inference ?

using “custom lib parsers” to do inference ?

yes … and can i have single parser loaded to server all the ONNX models

the inference pipeline is nornally:

pre-process → onnx inference with onnx backend → post-processing

for most of the model, we can just use the onnx backend in Triton server for the inference
DeepStream nvinferser includes some pre-process.
for post-processing, it’s normally model specific

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.