Create multiple contexts for DLA

Hi, I am now using DLA on jetson xavier nx to do DNN inference. I would like to create multiple contexts(i.e. load multiple engines) for a DLA. However I have found that I can only create 3 or 4(depend on engine size) and then it reports error. I am using jetpack4.6.

NvMapMemAllocInternalTagged: 1074810371 error 12
NvMapMemHandleAlloc: error 12
Module_id 33 Severity 2 : NVMEDIA_DLA 2650
Module_id 33 Severity 2 : Runtime: loadBare failed. error: 0x000006
ERROR: 1: [nvdlaUtils.cpp::deserialize::164] Error Code 1: DLA (NvMediaDlaLoadLoadable : load loadable failed.)

I have found similar issues in the following links:

Issues when using DLA with TensorRT 7.1.3 compared to TensorRT 6.0.1
How to know when to stop generating new DLA context? - #7 by yjkim2

It seems that the DLA can only support 4 contexts since jetpack 4.3. But jetpack 4.3 allows create up to 8 contexts.

So I wonder any latter jetpack versions fixed this issue? If not, is AGX Xavier or AGX Orin support more than 4 DLA contexts ?

Thank you very much for answering this long question!

Hi,

You can find this information below:

https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#dla-supp-lay-rest

The following restrictions apply to all layers while running on DLA:

  • TensorRT may split a network into multiple DLA loadables if any intermediate layers cannot run on DLA and GPUFallback is enabled. Otherwise, TensorRT can emit an error and fallback. For more information, refer to GPU Fallback Mode.
  • At most, 16 DLA loadables can be in use concurrently, per core, due to hardware and software memory limitations.

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.