Description
I am devloping a C/C++ program to infer images. It uses in-process C API to call triton API. I need add a string in infer request. Below is the related to to add string.
std::vector<int64_t> models_shape({1});
FAIL_IF_ERR(
TRITONSERVER_InferenceRequestAddInput(
irequest, “model_list”, TRITONSERVER_TYPE_BYTES, &models_shape[0], models_shape.size()),
“setting model list meta-data for the request”);
FAIL_IF_ERR(
TRITONSERVER_InferenceRequestAppendInputData(
irequest, “model_list”, (void )req->models_data, req->models_data_size, TRITONSERVER_MEMORY_CPU,
0 / memory_type_id */),
“assigning model_list data”);
But I got error in infer response callback:
infer response error: Failed to process the request(s) for model instance ‘model_main_0_0’, message: error: unpack_from requires a buffer of at least 16 bytes for unpacking 4 bytes at offset 12 (actual buffer size is 14)
Before I call TRITONSERVER_InferenceRequestAppendInputData, I print out the input data with binary format as below:
00 00 00 0a 6d 6f 64 65 6c 5f 6d 61 69 6e
It should be correct. The total data size is 14, the first 4 bytes is the string len 10, followed the string “model_main”.
I dont know why I got the error: requires a buffer of at least 16 bytes for unpacking 4 bytes at offset 12
My triton version is r24.05.
Environment
TensorRT Version: 10.0.1.6
GPU Type: RTX3090
Nvidia Driver Version: 550.120
CUDA Version: 12.4
CUDNN Version: 9.1.0
Operating System + Version: Ubuntu 22.04.5 LTS
Python Version (if applicable): python 3.10.12
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered