Problem building TensorRT engines for DLA core

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I get this error when building YOLOv4 TensorRT engines for DLA core on Jetson Xavier NX. What does this error message mean??

[TensorRT] ERROR: …/builder/cudnnBuilderGraph.cpp (448) - Assertion Error in getDefinition: 0 (mIndex >= 0 && “symbol is not concrete”)

Environment

TensorRT Version: 7.1.3.4
GPU Type: Jetson Xavier NX
Nvidia Driver Version: JetPack-4.4
CUDA Version: 10.2
CUDNN Version: 8.0
Operating System + Version: 4 4.9.140-tegra
Python Version (if applicable): 3.6
Baremetal or Container (if container which image + tag): baremetal

Relevant Files

Steps To Reproduce

I use python API to build YOLOv4 TensorRT engine for DLA core, and get the error.

            builder.max_batch_size = MAX_BATCH_SIZE                                       
            config = builder.create_builder_config()                                      
            config.max_workspace_size = 1 << 30                                           
            config.set_flag(trt.BuilderFlag.GPU_FALLBACK)                                 
            config.set_flag(trt.BuilderFlag.FP16)                                         
            profile = builder.create_optimization_profile()                               
            profile.set_shape(                                                            
                '000_net',                          # input tensor name                   
                (MAX_BATCH_SIZE, 3, net_h, net_w),  # min shape                           
                (MAX_BATCH_SIZE, 3, net_h, net_w),  # opt shape                           
                (MAX_BATCH_SIZE, 3, net_h, net_w))  # max shape                           
            config.add_optimization_profile(profile)                                      
            if do_int8:                                                                                             
                config.set_flag(trt.BuilderFlag.INT8)                                     
                config.int8_calibrator = YOLOEntropyCalibrator(...)                                 
                config.set_calibration_profile(profile)
            config.default_device_type = trt.DeviceType.DLA
            config.DLA_core = 0
            config.set_flag(trt.BuilderFlag.STRICT_TYPES)
            print('Using DLA core 0.')
            engine = builder.build_engine(network, config)

Error:

[TensorRT] ERROR: …/builder/cudnnBuilderGraph.cpp (448) - Assertion Error in getDefinition: 0 (mIndex >= 0 && “symbol is not concrete”)

2

Hi @jkjung13,
Request you to share your model, so that i can help you better.

Thanks!

3

I put the ONNX file here: yolov4-tiny-416.onnx

In addition to TensorRT python API, I’m also able to reproduce the error with “trtexec” as follows. (The same command works OK if “–useDLACore=0” is removed.)

$ /usr/src/tensorrt/bin/trtexec \
      --onnx=yolov4-tiny-416.onnx \
      --fp16 --int8 --useDLACore=0 \
      --allowGPUFallback \
      --workspace=1024 --warmUp=2 \
      --dumpProfile --verbose
......
[09/29/2020-17:08:38] [E] [TRT] ../builder/cudnnBuilder2.cpp (1757) - Assertion Error in operator(): 0 (et.region->getType() == RegionType::kNVM)
[09/29/2020-17:08:38] [E] Engine creation failed
[09/29/2020-17:08:38] [E] Engine set up failed

Note that the error message is different from my original post, but it does fail…

By the way, the ONNX file was created from the official darknet YOLOv4-Tiny model:

4

@AakankshaS Would you be able to reproduce the error with the ONNX model?

Or could you help to look it up internally and see what these error messages mean (and how to resolve them)? Thanks.

[TensorRT] ERROR: …/builder/cudnnBuilderGraph.cpp (448) - Assertion Error in getDefinition: 0 (mIndex >= 0 && “symbol is not concrete”)

[TRT] …/builder/cudnnBuilder2.cpp (1757) - Assertion Error in operator(): 0 (et.region->getType() == RegionType::kNVM)

5

Hi @jkjung13,
Apologies for delayed response.

This error usually means that scales are missing for some tensors.
However, i am checkin with the team, and will update you soon on the issue.

Thanks!

6

@AakankshaS Noted. But I’m still puzzled since I’m able to build the same INT8 TensorRT engine for GPU. That indicates TensorRT is able to figure out scales all tensors in the model, right? It only fails when I build the engine for DLA core.

Anyway, thanks. And I’m eager to hearing your update soon.

8

Hi! Is there any progress on this? I have the exact same error with trtexec when using DLACore on Jetson NX.

9
1 Like