Python 3.8 RAM owerflow and loading issues

[AastaLLL]

Hi,

It looks like you are facing some issue related to the CUDA context.
In general, you should store/restore the CUDA context when switching the tasks for each thread.

A sample for CUDA with thread can be found here:

How to use TensorRT by the multi-threading package of python

Hi, The sample is ready. The workflow looks like this: Master: create TensorRT engine and buffer, store the created CUDA context. Thread1: restore master’s CUDA context, run inference, destroy its CUDA context. Master: destroy the stored CUDA context. The sample is created with TensorRT default MNIST model. 1. Prepare data $ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --saveEngine=mnist.trt $ cd /usr/src/tensorrt/data/mnist/ $ sudo pip3 install pillow $ pyt…

Please let me know if I misunderstood your question.
Thanks.