Using DLA hardware with DNN APIs

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

DRIVE OS Version: 6.0.10

Issue Description:

We designed a network with fully supported DLA layers as per the TensorRT 8.6.13 Developer Guide
succesfully exported the network to onnx with fixed I/O formats.

I then natively compiled the network to run on DLA Hardware using the following command with flattened I/O (dla_linear) -

LoadableBuiltbyTRTLinear.log (95.6 KB)

I then used this native .loadable compiled plan directly with both processor formats via the application layer - DW_PROCESSOR_CUDLA and DW_PROCESSOR_DLA_0

Application output for both types compiled -

benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$ ./dla_lanenetDLA0 --image=/home/benchdev2/DLALaneNet/1.png   --model=/home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable  --device=dla --iterations=10000 --verbose=1
[08-06-2026 23:39:09] Platform: Detected Drive Orin P3710
[08-06-2026 23:39:09] Adding variable DW_Base:DW_Version
[08-06-2026 23:39:09] Added variable DW_Base:DW_Version
[08-06-2026 23:39:09] Platform: number of GPU devices detected 1
[08-06-2026 23:39:09] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_12_04, Arch: ga10b
[08-06-2026 23:39:09] Platform: currently selected GPU device integrated ID 0
[08-06-2026 23:39:09] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:39:09] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:39:09] Context::mountResourceCandidateDataPath resource FAILED to mount from './resources': VirtualFileSystem: Failed to mount './resources/resources.pak'
[08-06-2026 23:39:09] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/bin/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/bin/data/resources.pak'
[08-06-2026 23:39:09] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:39:09] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:39:09] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:39:09] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:39:09] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:39:09] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:39:09] SDK: No resources(.pak) mounted, some modules will not function properly
[08-06-2026 23:39:09] [08-06-2026 23:39:09] Initialize DriveWorks SDK v5.20.24
[08-06-2026 23:39:09] [08-06-2026 23:39:09] Release build with GNU 9.3.0 from buildbrain-branch-0-gb4c0b405b15 against Drive PDK v6.0.10.0
[cfg   ] device     : dla (DLA_0)
[cfg   ] dla-engine : 0
[cfg   ] image      : /home/benchdev2/DLALaneNet/1.png
[cfg   ] model      : /home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable
[cfg   ] output     : /home/benchdev2/DLALaneNet/dla_preview.png
[cfg   ] iterations : 10000
[cfg   ] verbose    : yes
[cfg   ] verify     : run `tegrastats` in another terminal; GR3D (GPU) stays idle if on DLA
[08-06-2026 23:39:09] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[08-06-2026 23:39:09] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[08-06-2026 23:39:09] Loaded engine size: 23 MiB
[08-06-2026 23:39:09] 1: [runtime.cpp::parsePlan::314] Error Code 1: Serialization (Serialization assertion plan->header.magicTag == rt::kPLAN_MAGIC_TAG failed.)
[08-06-2026 23:39:09] Driveworks exception thrown: DW_DNN_INVALID_MODEL: DNN: Unable to load model.

ERROR: [2026-06-09 00:39:09] DW Error DW_DNN_INVALID_MODEL executing DW function:
 dwDNN_initializeTensorRTFromFileWithEngineId( &dnn, modelPath.c_str(), nullptr, processor, dlaEngine, ctx)
 at /usr/local/driveworks/samples/src/DLALaneNet/main.cpp:426
[08-06-2026 23:39:09] [08-06-2026 23:39:09] Releasing Driveworks SDK Context
benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$ ./dla_lanenetCuDLA --image=/home/benchdev2/DLALaneNet/1.png   --model=/home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable  --device=dla --iterations=10000 --verbose=1
[08-06-2026 23:39:20] Platform: Detected Drive Orin P3710
[08-06-2026 23:39:20] Adding variable DW_Base:DW_Version
[08-06-2026 23:39:20] Added variable DW_Base:DW_Version
[08-06-2026 23:39:20] Platform: number of GPU devices detected 1
[08-06-2026 23:39:20] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_12_04, Arch: ga10b
[08-06-2026 23:39:20] Platform: currently selected GPU device integrated ID 0
[08-06-2026 23:39:21] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:39:21] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:39:21] Context::mountResourceCandidateDataPath resource FAILED to mount from './resources': VirtualFileSystem: Failed to mount './resources/resources.pak'
[08-06-2026 23:39:21] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/bin/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/bin/data/resources.pak'
[08-06-2026 23:39:21] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:39:21] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:39:21] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:39:21] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:39:21] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:39:21] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:39:21] SDK: No resources(.pak) mounted, some modules will not function properly
[08-06-2026 23:39:21] [08-06-2026 23:39:21] Initialize DriveWorks SDK v5.20.24
[08-06-2026 23:39:21] [08-06-2026 23:39:21] Release build with GNU 9.3.0 from buildbrain-branch-0-gb4c0b405b15 against Drive PDK v6.0.10.0
[cfg   ] device     : dla (DLA_0)
[cfg   ] dla-engine : 0
[cfg   ] image      : /home/benchdev2/DLALaneNet/1.png
[cfg   ] model      : /home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable
[cfg   ] output     : /home/benchdev2/DLALaneNet/dla_preview.png
[cfg   ] iterations : 10000
[cfg   ] verbose    : yes
[cfg   ] verify     : run `tegrastats` in another terminal; GR3D (GPU) stays idle if on DLA
[08-06-2026 23:39:21] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[08-06-2026 23:39:21] DNN: TensorRT model file has wrong magic number. Please ensure that the model has been created by TensorRT_optimization tool in DriveWorks. The model might be incompatible.
[08-06-2026 23:39:21] Driveworks exception thrown: DW_INVALID_ARGUMENT: DNN: Invalid processor type.

ERROR: [2026-06-09 00:39:21] DW Error DW_INVALID_ARGUMENT executing DW function:
 dwDNN_initializeTensorRTFromFileWithEngineId( &dnn, modelPath.c_str(), nullptr, processor, dlaEngine, ctx)
 at /usr/local/driveworks/samples/src/DLALaneNet/main.cpp:426
[08-06-2026 23:39:21] [08-06-2026 23:39:21] Releasing Driveworks SDK Context
benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$

I see that model is invalid to run with DNN APis.

For next steps I used only the tensorRT_optimzation tool produce a model compiled for DLA -

Note: here the tool doesnt take dla_linear as the I/O formats so I used chw16 which is default onnx model’s I/O formats.

benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$ ./tensorRT_optimization --modelType=onnx --onnxFile=/home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx  --inputIOFormats=fp16:chw16 --outputIOFormats=fp16:chw16  --useDLA=1  --out=/home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin
[08-06-2026 23:44:38] DNNGenerator: Initializing TensorRT generation on model /home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx.
[08-06-2026 23:44:38] DNNGenerator: Input "input": 1x3x512x1024
[08-06-2026 23:44:38] DNNGenerator: Output "logits": 1x2x512x1024
[08-06-2026 23:44:38] DNNGenerator: Precision is not specified for the DLA model. Setting precision to FP16.
[08-06-2026 23:44:56] DNNGenerator: Serializing extra DNN '/home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin.no_header' for profiling with trtexec.
[08-06-2026 23:44:56] DNNGenerator: Skipping validation due to non-chw input TensorFormat(s).
[08-06-2026 23:44:56] [08-06-2026 23:44:56] Releasing Driveworks SDK Context
benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$

We use this network built by tool to be directly used with both type of applications -

benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$ ./dla_lanenetDLA0 --image=/home/benchdev2/DLALaneNet/1.png   --model=/home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin  --device=dla --iterations=10000 --verbose=1
[08-06-2026 23:47:43] Platform: Detected Drive Orin P3710
[08-06-2026 23:47:43] Adding variable DW_Base:DW_Version
[08-06-2026 23:47:43] Added variable DW_Base:DW_Version
[08-06-2026 23:47:43] Platform: number of GPU devices detected 1
[08-06-2026 23:47:43] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_12_04, Arch: ga10b
[08-06-2026 23:47:43] Platform: currently selected GPU device integrated ID 0
[08-06-2026 23:47:43] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:47:43] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:47:43] Context::mountResourceCandidateDataPath resource FAILED to mount from './resources': VirtualFileSystem: Failed to mount './resources/resources.pak'
[08-06-2026 23:47:43] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/bin/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/bin/data/resources.pak'
[08-06-2026 23:47:43] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:47:43] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:47:43] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:47:43] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:47:43] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:47:43] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:47:43] SDK: No resources(.pak) mounted, some modules will not function properly
[08-06-2026 23:47:43] [08-06-2026 23:47:43] Initialize DriveWorks SDK v5.20.24
[08-06-2026 23:47:43] [08-06-2026 23:47:43] Release build with GNU 9.3.0 from buildbrain-branch-0-gb4c0b405b15 against Drive PDK v6.0.10.0
[cfg   ] device     : dla (DLA_0)
[cfg   ] dla-engine : 0
[cfg   ] image      : /home/benchdev2/DLALaneNet/1.png
[cfg   ] model      : /home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin
[cfg   ] output     : /home/benchdev2/DLALaneNet/dla_preview.png
[cfg   ] iterations : 10000
[cfg   ] verbose    : yes
[cfg   ] verify     : run `tegrastats` in another terminal; GR3D (GPU) stays idle if on DLA
[08-06-2026 23:47:43] DNN: Initializing network on cuDLA 0.
[08-06-2026 23:47:43] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:47:43] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:47:43] ========== DLA Device 0 Attributes ==========
[08-06-2026 23:47:43]            UVA supported = 0
[08-06-2026 23:47:43]            Device version = 2
[08-06-2026 23:47:43]            Device 0 Check PASSED
[08-06-2026 23:47:43] Driveworks exception thrown: DW_INVALID_ARGUMENT: cudlaModuleLoadFromMemory : load loadable failed. 7

ERROR: [2026-06-09 00:47:43] DW Error DW_INVALID_ARGUMENT executing DW function:
 dwDNN_initializeTensorRTFromFileWithEngineId( &dnn, modelPath.c_str(), nullptr, processor, dlaEngine, ctx)
 at /usr/local/driveworks/samples/src/DLALaneNet/main.cpp:426
[08-06-2026 23:47:43] [08-06-2026 23:47:43] Releasing Driveworks SDK Context
benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$ ./dla_lanenetCuDLA --image=/home/benchdev2/DLALaneNet/1.png   --model=/home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin  --device=dla --iterations=10000 --verbose=1
[08-06-2026 23:48:00] Platform: Detected Drive Orin P3710
[08-06-2026 23:48:00] Adding variable DW_Base:DW_Version
[08-06-2026 23:48:00] Added variable DW_Base:DW_Version
[08-06-2026 23:48:00] Platform: number of GPU devices detected 1
[08-06-2026 23:48:00] Platform: currently selected GPU device 0, Resource Data Dir: trt_08_06_12_04, Arch: ga10b
[08-06-2026 23:48:00] Platform: currently selected GPU device integrated ID 0
[08-06-2026 23:48:00] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:48:00] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:48:00] Context::mountResourceCandidateDataPath resource FAILED to mount from './resources': VirtualFileSystem: Failed to mount './resources/resources.pak'
[08-06-2026 23:48:00] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/bin/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/bin/data/resources.pak'
[08-06-2026 23:48:00] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:48:00] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:48:00] Context::findDataRootInPathWalk data/DATA_ROOT found at: /usr/local/driveworks-5.20/data
[08-06-2026 23:48:00] Context::mountResourceCandidateDataPath resource FAILED to mount from '/usr/local/driveworks-5.20/data': VirtualFileSystem: Failed to mount '/usr/local/driveworks-5.20/data/resources.pak'
[08-06-2026 23:48:00] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:48:00] Context::findResourcesPackageInPathWalk: Could not find ./resources/resources.pak in upto 7 parent directories from /usr/local/driveworks-5.20/targets/aarch64-Linux/lib/libdw_base.so.5.20
[08-06-2026 23:48:00] SDK: No resources(.pak) mounted, some modules will not function properly
[08-06-2026 23:48:00] [08-06-2026 23:48:00] Initialize DriveWorks SDK v5.20.24
[08-06-2026 23:48:00] [08-06-2026 23:48:00] Release build with GNU 9.3.0 from buildbrain-branch-0-gb4c0b405b15 against Drive PDK v6.0.10.0
[cfg   ] device     : dla (DLA_0)
[cfg   ] dla-engine : 0
[cfg   ] image      : /home/benchdev2/DLALaneNet/1.png
[cfg   ] model      : /home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin
[cfg   ] output     : /home/benchdev2/DLALaneNet/dla_preview.png
[cfg   ] iterations : 10000
[cfg   ] verbose    : yes
[cfg   ] verify     : run `tegrastats` in another terminal; GR3D (GPU) stays idle if on DLA
[08-06-2026 23:48:01] DNN: Initializing network on cuDLA 0.
[08-06-2026 23:48:01] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:48:01] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:48:01] ========== DLA Device 0 Attributes ==========
[08-06-2026 23:48:01]            UVA supported = 0
[08-06-2026 23:48:01]            Device version = 2
[08-06-2026 23:48:01]            Device 0 Check PASSED
[08-06-2026 23:48:01] Driveworks exception thrown: DW_INVALID_ARGUMENT: cudlaModuleLoadFromMemory : load loadable failed. 7

ERROR: [2026-06-09 00:48:01] DW Error DW_INVALID_ARGUMENT executing DW function:
 dwDNN_initializeTensorRTFromFileWithEngineId( &dnn, modelPath.c_str(), nullptr, processor, dlaEngine, ctx)
 at /usr/local/driveworks/samples/src/DLALaneNet/main.cpp:426
[08-06-2026 23:48:01] [08-06-2026 23:48:01] Releasing Driveworks SDK Context
benchdev2@tegra-ubuntu:/usr/local/driveworks/bin$

I see that DLA engine is invoked from the verbose logging however it doesnt load the model.
Please also note when the I/O formats arent passed to the tool, it does compile the network but fails to serialize it - so passing it to the application is vain.

For the next steps -
I now wrap the initally natively compiled loadable by trtexec by passing it to the tool like this -
*
Note :* here I dont pass I/O formats as trtexec compiled network already has fixed the I/O in linear format.

benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$ ./tensorRT_optimization --modelType=onnx --onnxFile=/home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx --dlaLoadable=/home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable  --useDLA=1  --out=/home/benchdev2/DLALaneNet/lanenet_wrappedbyTool.dla.bin
[08-06-2026 23:52:40] DNNGenerator: Initializing TensorRT generation on model /home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx.
[08-06-2026 23:52:41] DNNGenerator: Input "input": 1x3x512x1024
[08-06-2026 23:52:41] DNNGenerator: Output "logits": 1x2x512x1024
[08-06-2026 23:52:41] DNNGenerator: DLA loadable is already built!
[08-06-2026 23:52:41] DNN: Initializing network on cuDLA 0.
[08-06-2026 23:52:41] CUDLAEngine:getDLACount: CUDLA version is = 1004000
[08-06-2026 23:52:41] CUDLAEngine:getDLACount: Number of DLA devices = 2
[08-06-2026 23:52:41] ========== DLA Device 0 Attributes ==========
[08-06-2026 23:52:41]            UVA supported = 0
[08-06-2026 23:52:41]            Device version = 2
[08-06-2026 23:52:41]            Device 0 Check PASSED
[08-06-2026 23:52:41] CUDLALoadable: it is DLA Header!
[08-06-2026 23:52:41] TENSOR DESCRIPTOR 
[08-06-2026 23:52:41] 	TENSOR NAME : input'
[08-06-2026 23:52:41] 	size: 3145728
[08-06-2026 23:52:41] 	dims: [1, 3, 512, 1024]
[08-06-2026 23:52:41] 	data fmt: NCHW
[08-06-2026 23:52:41] 	data type: HALF
[08-06-2026 23:52:41] 	data category: PLANAR
[08-06-2026 23:52:41] 	pixel fmt: 5
[08-06-2026 23:52:41] 	pixel mapping: 0
[08-06-2026 23:52:41] 	stride[0]: 2
[08-06-2026 23:52:41] 	stride[1]: 2048
[08-06-2026 23:52:41] 	stride[2]: 1048576
[08-06-2026 23:52:41] 	stride[3]: 1048576
[08-06-2026 23:52:41] TENSOR DESCRIPTOR 
[08-06-2026 23:52:41] 	TENSOR NAME : logits'
[08-06-2026 23:52:41] 	size: 2097152
[08-06-2026 23:52:41] 	dims: [1, 2, 512, 1024]
[08-06-2026 23:52:41] 	data fmt: NCHW
[08-06-2026 23:52:41] 	data type: HALF
[08-06-2026 23:52:41] 	data category: PLANAR
[08-06-2026 23:52:41] 	pixel fmt: 5
[08-06-2026 23:52:41] 	pixel mapping: 0
[08-06-2026 23:52:41] 	stride[0]: 2
[08-06-2026 23:52:41] 	stride[1]: 2048
[08-06-2026 23:52:41] 	stride[2]: 1048576
[08-06-2026 23:52:41] 	stride[3]: 1048576
[08-06-2026 23:52:41] DNN: Metadata json file could not be found. Metadata has been filled with default values. Please place <network_filename>.json in the same directory as the network file if custom metadata is needed.
[08-06-2026 23:52:41] [08-06-2026 23:52:41] Releasing Driveworks SDK Context
Error: DW_DNN_INVALID_MODEL: DNN: Unexpected input blob.
benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$

I see that the tool outputs blob mismatch like above - and this same Error is produced when the network is passed to both applications.

For next steps - I then tried rebuilding the network with explicit I/O being fp16:chw16

LoadableBuiltbyTRTfp16chw16.log (95.6 KB)

I then re-wrap this loadable with the tensorRT_optimization in order of hypothesis that this might mitigate the blob mismatch error -

However the tool it unable to wrap this loadable in any manner -

benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$ ./tensorRT_optimization --modelType=onnx --onnxFile=/home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx  --inputIOFormats=fp16:chw16 --outputIOFormats=fp16:chw16  --dlaLoadable=/home/benchdev/DLALaneNet/lanenetTRTBuiltfp16chw16.loadable --useDLA=1  --out=/home/benchdev2/DLALaneNet/lanenet_wrappedbyToolfp16chw16.dla.bin
[09-06-2026 00:06:56] DNNGenerator: Initializing TensorRT generation on model /home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx.
[09-06-2026 00:06:56] DNNGenerator: Input "input": 1x3x512x1024
[09-06-2026 00:06:56] DNNGenerator: Output "logits": 1x2x512x1024
[09-06-2026 00:06:56] DNNGenerator: DLA loadable is already built!
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)
benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$ ./tensorRT_optimization --modelType=onnx --onnxFile=/home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx   --dlaLoadable=/home/benchdev/DLALaneNet/lanenetTRTBuiltfp16chw16.loadable --useDLA=1  --out=/home/benchdev2/DLALaneNet/lanenet_wrappedbyToolfp16chw16.dla.bin
[09-06-2026 00:07:21] DNNGenerator: Initializing TensorRT generation on model /home/benchdev2/DLALaneNet/dla_lanenet_latest.onnx.
[09-06-2026 00:07:21] DNNGenerator: Input "input": 1x3x512x1024
[09-06-2026 00:07:21] DNNGenerator: Output "logits": 1x2x512x1024
[09-06-2026 00:07:21] DNNGenerator: DLA loadable is already built!
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)
benchdev2@tegra-ubuntu:/usr/local/driveworks/tools/dnn$

Can we get a clue how to proceeed with this model ?
PS - I tried natively testing CuDLA APIs outside of DNN APIs and the tool, the file runs appropriately without spiking any GR3D-FREQ in tegrastats - I infer this means the model runs on DLA with cuDLA APIs?

5

Dear @SivaRamaKrishnaNV any insights on this - this is a clean thread from cudlaModuleLoadFromMemory : load loadable failed. 7 - #10 by atharv.sharma
Me and my colleague have originated the issues from this post.

6

Just updating this thread with some more findings @SivaRamaKrishnaNV - I used a prober application to check if cuDLA accepts the model outside of DNN APIs on both the files (native TRT DLA built, and DW-DNN tensorrt_optimsation tool)

What it seems like both ways there’s blob mismatch when I tried to use cudla API directly to load from memory for both compiled files like this :

snippet -

cudlaDevHandle dev{};

    cudlaStatus s = cudlaCreateDevice(static_cast<uint64_t>(dlaDevice), &dev, flags);
if (s != cudlaSuccess) {
std::cout << "  cudlaCreateDevice FAILED: " << statusName(s) << " (" << int(s) << ")\n";
return false;
    }

std::cout << "  cudlaCreateDevice OK\n";


cudlaModule mod{};

s = cudlaModuleLoadFromMemory(dev, blob.data(), blob.size(), &mod, 0);

I then output this vector’s first 4 fields trying to see what both the compilers are trying to pack like this in part of main()

snippet -

std::vector<uint8_t> blob;
    
try {
blob = readFile(modelPath);
    } catch (const std::exception& e) {
std::cerr << "ERROR: " << e.what() << "\n";
return EXIT_FAILURE;
    }
std::cout << "file: " << modelPath << " (" << blob.size() << " bytes)\n";

if (blob.size() >= 4)

std::printf("Fields[0..3]: %02x %02x %02x %02x\n",

blob[0], blob[1], blob[2], blob[3]);

I use only cudlaMode as CUDLA_STANDALONE in the probing application to check this.
Suprisingly - there’s a difference in the blob field values,


benchdev2@tegra-ubuntu:~$ ./dlaProberTest --model=/home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable
cuDLA runtime version: 1004000
probing DLA core 0
file: /home/benchdev2/DLALaneNet/lanenetTRTBuiltLinear.loadable (24638168 bytes)
Field[0..3]: 20 00 00 00

--- mode=standalone (CUDLA_STANDALONE) device=0 ---
cudlaCreateDevice OK
cudlaModuleLoadFromMemory -> cudlaSuccess (integer 0)
tensors: 1 in / 1 out
in[0]  'input''  3145728 B
out[0] 'logits''  2097152 B
RESULT: LOAD OK

benchdev2@tegra-ubuntu:~$ ./dlaProberTest --model=/home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin
cuDLA runtime version: 1004000
probing DLA core 0
file: /home/benchdev2/DLALaneNet/lanenet_builtbyTool.dla.bin (24637090 bytes)
Field[0..3]: c8 3e ab 04

--- mode=standalone (CUDLA_STANDALONE) device=0 ---
cudlaCreateDevice OK
cudlaModuleLoadFromMemory -> cudlaErrorUmd (integer 7)
RESULT: LOAD FAILED

Does this mean that cuDLA harness itself is natively rejecting the model compiled by Tool in general, no whatter what DNN API is used to parse the model or initialize it?

7

That specific error `cudlaErrorUmd in documentation for cudla.h enumeration for cudlaStatus point to this -

  /**

     * This indicates that there was an error in the

     * DLA runtime for the API call.

     */

cudlaErrorUmd                                          = 7,
8

Dear @SivaRamaKrishnaNV Do you have any information on the source code for the tool by the way? or is it closed source?

9

tensorRT_optimization source code is not available
I am investigating the issue and update you.

11

@SivaRamaKrishnaNV any updates regrading this?

12

It appears like a bug in cudla libs. The suggested path is to generate DLA model from ONNX using tensorRT_optimzation tool and load using DNN APIs. I am yet to get an update from engineering team to see if any patch lib can be provided. Note that DOS 6.0.10 is last devzone release for Orin.

13

This what we tried to do, but cuDLA rejects loading the model produced by tensorRT_optimization tool. Please also note that cuDLA APIs can load and run inference in standalone mode with native
cudlaModuleLoadFromMemory() and cudlaSubmitTask() API.

14

It is noticed that tensorRT_optimization tool generated DLA model does not seems to work with DNN APIs. Please use cuDLA APIs directly to run model on DLA.