Hi, I am using onnx language model in trition, my config file content name: “textmodel.onnx”, platform: “onnxruntime_onnx”, backend: “onnxruntime”, version_policy: { latest: { num_versions: 1 } }, max_batch_size: 500000, dynamic_batching { preferred_batch_size: [ 500000 ] max_queue_dela…

Trition exiting after every batch inference on cpu

nadeemm July 27, 2021, 12:58am 2

Could you provide some information about your system configuration ?

Topic		Replies	Views
TRTIS randomly quit without any error message Triton Inference Server (archived)	0	582	May 12, 2020
Triton Server CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered Triton Inference Server (archived)	0	1208	June 12, 2020
Batch Inference Wrong in Python API TensorRT	15	3633	October 12, 2021
ResourceExhaustedError: Running TF-TRT integration on Jetson AGX Jetson AGX Xavier	10	1217	October 18, 2021
Cuda Error in executeInternal: 700 (an illegal memory access was encountered) Jetson AGX Xavier tensorrt	10	6257	December 2, 2021
Triton server crushed occasionally Triton Inference Server (archived) tensorrt	1	512	September 1, 2020
Issue while Serving the model using Triton Server Triton Inference Server (archived) inference-server-triton	0	508	November 18, 2020
TensorRT running inference with batch size > 1 TensorRT tensorrt	8	3784	January 26, 2021
Run out of memory when creating TensorRT engine from onnx model Jetson Xavier NX tensorrt	7	2886	October 18, 2021
batch inference with TensorRT TensorRT	4	1726	October 12, 2021