How to make context on DLA?

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hello,

I am trying to do inference fully on DLA in a multi-thread pipeline python. All the layers of my model run on DLA.

In first place, I tried:

cfx = cuda.Device(0).make_context()
#do inference
cfx.pop()

but I had the following error:

[TRT] [E] 1: [eglUtils.cpp::~CudaProducer::208] Error Code 1: Cuda Runtime (invalid device context)

I usually use it to run my TensorRT model on GPU, I guess it does not work on DLA? Then, when I tried to change the configuration by removing the cuda.Device(0), I had the follow error:

pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?

Can you explained how to configure the DLA to do inference ?

Thanks.

2

In addition, I’m using the Xavier NX with JetPack4.6.4 and tensorrt 8.0.1.6-1+cuda10.2 with cuda 10.2.460-1.

Thanks

4

Hi,

Could you share a complete source to check the issue?
Thanks

5

Hello,

Thank you for your reply. Unfortunately, I can’t share the code or model. However, I can tell you that:

  • a first context is already created for a tensorRT model running on GPU. It works.
  • we want to create another context for a tensorRT model running exclusively on DLA.

Usually, when I create another context for tensorRT model running on GPU, I used the following lines of code to create the context and read the model:

import tensorrt as trt
from pycuda import driver as cuda

self.cfx = cuda.Device(0).make_context()
trt_logger = trt.Logger(trt.Logger.Severity.ERROR)
trt.init_libnvinfer_plugins(trt_logger, '')
        
with open("model_gpu.trt", "rb") as f:
            
    with trt.Runtime(trt_logger) as runtime:
                
        if dla_core is not None:
            runtime.DLA_core = dla_core
                
        self.engine = runtime.deserialize_cuda_engine(f.read())
        
self.context = self.engine.create_execution_context()

It works when I used a GPU model. But now, when i do inference with my DLA model, I had the following error :

[TRT] [E] 1: [eglUtils.cpp::~CudaProducer::208] Error Code 1: Cuda Runtime (invalid device context)

Maybe it’s because of my model do not run on GPU anymore?

Then, I tried to change the creation of the context:

trt_logger = trt.Logger(trt.Logger.Severity.ERROR)
trt_runtime = trt.Runtime(trt_logger)
trt.BuilderFlag.GPU_FALLBACK = False
trt.IBuilderConfig.default_device_type = trt.DeviceType.DLA
trt.IBuilderConfig.DLA_core = 0
        
with open("model_dla.trt", 'rb') as f:
    engine_data = f.read()
            
self.engine = trt_runtime.deserialize_cuda_engine(engine_data)
self.context = self.engine.create_execution_context()

And I have the following error:

pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?

Thanks

6

I’m closing this topic due to there is no update from you for a period, assuming this issue was resolved.
If still need the support, please open a new topic. Thanks

Hi,

We want to check it further.
Could you share which JetPack/TensorRT version you use?

Is it JetPack 5.1.2 with TensorRT 8.5?
More, is it possible to share a simple reproducible source that can reproduce the error with us?

Thanks.

1 Like
8
9

Hi,
Can you try running your model with trtexec command, and share the “”–verbose"" log in case if the issue persist

You can refer below link for all the supported operators list, in case any operator is not supported you need to create a custom plugin to support that operation

Also, request you to share your model and script if not shared already so that we can help you better.

Meanwhile, for some common errors and queries please refer to below link:

Thanks!

1 Like