One program with multi pipeline，nvinferserver always infer failed

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

**• Hardware Platform GPU
**• DeepStream Version 6.2
**• CUDA Version: 12.0
**• NVIDIA GPU Driver Version 525.105.17

my program will create multi pipeline with nvinferserver plugin（load the same model).
buf if any one of the pipelines exit , other pipeline request will be fail.

Here is the log:

ERROR: infer_trtis_server.cpp:72 Triton: TrtServerRequest failed to create request, triton_err_str:Unavailable, err_msg:Request for unknown model: ‘20230808_enc’ version 1 is not at ready state
ERROR: infer_trtis_server.cpp:1112 Triton failed to create request for model: 20230808_enc version:1
ERROR: infer_trtis_backend.cpp:582 Triton run failed to create request for model: 20230808_enc
ERROR: infer_trtis_backend.cpp:514 Triton failed to run inference on model 20230808_enc, nvinfer error:NVDSINFER_TRITON_ERROR
0:00:27.639015275 2231654 0x7f2d60055670 WARN nvinferserver gstnvinferserver.cpp:568:gst_nvinfer_server_push_buffer: error: inference failed with unique-id:1
2023-08-18 03:42:09,949 ERROR [default] ERROR from element primary-gie1: inference failed with unique-id:1

2023-08-18 03:42:09,949 ERROR [default] Error details: gstnvinferserver.cpp(568): gst_nvinfer_server_push_buffer (): /GstPipeline:pipeline_LX1000000aaa/GstNvInferServer:primary-gie1

the config block

triton {
model_name: “58_20230808_enc”
version: 1

  model_repo {
   root: "/opt/trt_repo"
   strict_model_config: true
  }
}

}

How can I fix it !

3

can you elaborate on the application? what is the whole media pipeline? can you use deepstream-test1 to reproduce this issue?

4

There is no update from you for a period, assuming this is not an issue any more. Hence we are closing this topic. If need further support, please open a new one. Thanks.
Is this still an DeepStream issue to support? Thanks
when one pipeline quitted, the model will be unloaded, but the other pipeline is still using this model, so there was a model not at ready state error. nvinferserver plugin and triton are opensource. you can check the code if interested.
you can let the two pipeline quit at the same time. to be specific, when bus_call receive the first pipeline’s EOS, you can save the loop’s pointer. after receiving the second pipeline’s EOS, g_main_loop_quit two loops.