Deepstream - Failed to register CUDA shared memory

dbdnjswns2 · December 20, 2023, 11:59pm

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) : GPU
• DeepStream Version : 6.1
• JetPack Version (valid for Jetson only)
• TensorRT Version : 8.4.1
• NVIDIA GPU Driver Version (valid for GPU only) : 535.129.03
• Issue Type( questions, new requirements, bugs) : questions
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) :
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
deepstream으로 다른 docker에서 실행시킨 triton으로 inference를 실행하고자 합니다.
python의 multi process로 여러 프로세스로 inference를 실행하고자 합니다.
그런데 enable_cuda_buffer_sharing을 true로 설정하고 진행하는데 계속 아래와 같은 에러가 발생하는데 이유가 뭐야?
ERROR: infer_grpc_client.cpp:223 Failed to register CUDA shared memory.
ERROR: infer_grpc_client.cpp:311 Failed to set inference input: shared memory region ‘inbuf_0x7faaf00018e0’ already in manager
ERROR: infer_grpc_backend.cpp:140 gRPC backend run failed to create request for model: yolov8_16batch_dynamic
ERROR: infer_grpc_backend.cpp:195 TritonSimple failed to run inference on model yolov8_16batch_dynamic, nvinfer error:NVDSINFER_TRITON_ERROR

yingliu · December 21, 2023, 2:38am

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
• The pipeline being used

junshengy · December 25, 2023, 6:52am

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

How did you start docker? Can you share the specific command line?
Which apps/pipelines are you running? Can you share the steps to reproduce?
Please use English

As an additional supplement, is it consistent with this question?

github.com/triton-inference-server/server

error: failed to register input shared memory region: failed to register CUDA shared memory region 'input_data1'

opened 12:43PM - 02 Dec 21 UTC

closed 12:26AM - 22 Dec 21 UTC

simon5u

bug

**Description** Our PC has two GPUs and we use tritonserver 21.02 to host the y…olo5 models with cuda shared memory on these two GPUs. We run the tritonserver, eveything is working fine and the GPU memory consumption is around 5GB each. Then, we start to run our client code using two processes because we need to support 20 cameras, each process will handle 10 cameras with hardware decoding. We set the CUDA_VISIBLE_DEVICES=0 for the first process, and 1 for the second process. We are having the problem "error: failed to register input shared memory region: failed to register CUDA shared memory region 'input_data1'" when we run the second process. We dont have any problem on running the first process. We think that the problem was caused by the cuda shared memory. Any advice on this issue? **Triton Information** nvcr.io/nvidia/deepstream:5.1-21.02-triton Are you using the Triton container or did you build it yourself? We pull the original docker. **To Reproduce** Run the tritonserver to host the yolo5 model with cuda shared memory and using two GPUs. Then, we run two client processes with CUDA_VISIBLE_DEVICES=0 or 1. First process is running well with bbox output from yolo5. Second process will crash and show a log "error: failed to register input shared memory region: failed to register CUDA shared memory region 'input_data1'" Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well). Model is yolo5 from https://github.com/ultralytics/yolov5/releases. Input is a 2D image and output is an object bbox. **Expected behavior** We are expecting a way to direct the client process to the triton model hosted in a particular gpu.

system · January 8, 2024, 12:55pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
CUDA shared memory registration failed when requesting recognition from deepstream to an external triton server. to occur DeepStream SDK	6	355	April 23, 2024
Error when using Triton Server for Inference on deepstream-imagedata-example DeepStream SDK	21	1740	October 12, 2021
DeepStream Triton gRPC example does not run with Deepstream Triton Docker images DeepStream SDK	12	1071	January 17, 2023
Deepstream Triton Inference Server Error, Segmentation fault (core dumped) DeepStream SDK	6	1190	January 20, 2023
"nvdsinferserver.config.TritonGrpcParams" has no field named "enable_cuda_buffer_sharing" DeepStream SDK	6	330	June 30, 2023
Triton server logs DeepStream SDK	7	5044	May 16, 2022
Is it possible to deploy the deepstream_parallel_inference_app in the docker container of deepstream-5.1？ DeepStream SDK docker	23	596	March 8, 2023
Triton Inference through docker DeepStream SDK	6	1397	March 16, 2022
Error when using ensemble model with deepstream-5.1 : failed to get input buffer in CPU memory DeepStream SDK inference-server-triton	7	1191	September 4, 2021
Docker fails to register cuda shared memory TensorRT tensorrt , cuda , docker	3	1067	July 14, 2022

Deepstream - Failed to register CUDA shared memory

Related topics