Good Day!, I am trying to run object detection inference on multiple camera sources by utilizing Pebble’s ProcessPool. The pipeline/concept I am using has been tested and performs as expected on x86 servers. However, I seem to run into some issues on the JetsonTX2. I have tested python’s multiproces…

Hi, Based on the output message: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method" This error is related to API usage since it complains about the duplicate initialization call. Have you tried the sample on another device (ex…

Hello @AastaLLL , Thank You for responding. Have you tried the sample on another device (ex. x86 system) before? Answer: Yes I have tried it on x86 system and it functions as expected. I am actually adapting this concept from our implementation on x86 system to the Jetson platform.

Hello @AastaLLL . Just some additional context as this issue covers 2 errors. The first error covers Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the ‘spawn’ start method The second error is a primary roadblock as loading the model without multipr…

Hi, Thanks for the feedback. Have you tried the single process use case? Does it work?

Yes @AastaLLL . We are able to run single process use case without ProcessPool cc @venketramana1

@AastaLLL As remarked by @rehan2 . We have tested for single process use case without the use of ProcessPool. The sample script reproduces both cases whereby the function performs as expected outside of ProcessPool, but fails when used with ProcessPool.

Thanks for the sharing. We are going to reproduce this internally. Will share more information with you later.

Hi, We can reproduce this issue in our environment and are working on it. Will share more information with you later. Thanks.

Hi, Based on the below doc, please use torch.multiprocessing for the multiprocess usage. https://pytorch.org/docs/stable/notes/multiprocessing.html#cuda-in-multiprocessing Thanks.

Thanks for the feedback. Good to know you found a way to solve this.

Multiprocessing on Jetson

Robotics & Edge Computing Jetson Systems Jetson TX2

venketramana1 October 1, 2021, 5:55am 13

Good Day @AastaLLL, Thank You for the feedback and suggestion. We manage to solve the problem by reverting TRTorch from v0.4.0 to v0.2.0. We need to use Pebble ProcessPool for our implementation.

Multiprocessing PyTorch inference with TensorRT on Jetson Orin NX devices

Topic		Replies	Views
Multiprocessing PyTorch inference with TensorRT on Jetson Orin NX devices Jetson Orin NX tensorrt , cuda , pytorch , cudnn	1	764	April 17, 2024
Error in cuda when trying to inference via multiprocessing TensorRT	2	1787	November 14, 2021
Tensorrt inference with pytorch tensor(data_ptr) TensorRT tensorrt , cuda , pytorch	2	2026	June 11, 2021
'operation not supported' of spawn method in pytorch multiprocessing on Jetson Xavier NX Jetson Xavier NX pytorch	6	1083	November 1, 2022
Decoupling loading from inference models on multiprocessing Jetson TX1 jetson-inference	2	1161	January 24, 2022
How to use TensorRT in python multiprocessing environment? TensorRT cuda	1	1363	February 17, 2021
Python multi-process TensorRT inference resulting in `Error Code 1: Cudnn (CUDNN_STATUS_MAPPING_ERROR)` Jetson Xavier NX tensorrt , jetson-inference , python , cudnn	3	525	May 20, 2024
Python + torch + cuda in multiprocess application freeze Jetson AGX Orin pytorch	7	115	March 30, 2026
Yolov7 inferencing using multiprocess and tensorrt TensorRT	1	1222	April 28, 2023
Running Multiple TensorRT Engines On Jetson AGX Jetson AGX Xavier tensorrt , yolo	1	2611	June 24, 2021

Multiprocessing on Jetson

Related topics