[TensorRT Error] DLA validation failed

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hello,
I have a network layer that can be run on DLA (which has been validated with the function config->canRunOnDLA(layer)) and it also has been INT8 quantized, but when I declare it as running on DLA with the API config->setDeviceType( layer, DeviceType::kDLA), it still returns “DLA validation failed” error when building engine.
Here is my code for this part:

        config->setFlag(BuilderFlag::kGPU_FALLBACK);
        config->setDLACore(0);
        ILayer* layer = network->getLayer(10);
        if(config->canRunOnDLA(layer)) {
            config->setDeviceType(layer, DeviceType::kDLA);
        }

and the error:

ERROR: 4: [network.cpp::validate::2789] Error Code 4: Internal Error (DLA validation failed)
ERROR: 2: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

Additionally, this should not be a problem with DLA or TensorRT itself. I can successfully build an engine running on DLA using trtexec.

Here is my source code file:
YolotoTensorRT.txt (7.8 KB)

my onnx model is yolov5_trimmed_qat.

Looking forward to your responses!

2

Hi,
Here are some suggestions for the common issues:

1. Performance

Please run the below command before benchmarking deep learning use case:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2. Installation

Installation guide of deep learning frameworks on Jetson:

3. Tutorial

Startup deep learning tutorial:

4. Report issue

If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.

Thanks!

4

Hi,

Could you enable the verbose log and share the output with us?
Thanks.

5

How to output verbose if the source code does not build the engine successfully?

The attachment is the verbose log output using trtexec instead of my source code:
trtexec_yolov5.log (315.8 KB)

But I want to have some network layers running on the DLA, not the whole model, so I can’t use trtexec, but need to use the TensorRT API to write the corresponding code. But currently running into the problem I described above.

6

Hi,

You can enable it via setting the logger to VERBOSE level.

https://docs.nvidia.com/deeplearning/tensorrt/api/c_api/classnvinfer1_1_1_i_logger.html

class Logger : public nvinfer1::ILogger
{
    void log(Severity severity, const char* msg) noexcept override
    {
        if (severity <= Severity:: kVERBOSE) std::cout << msg << std::endl;
    }
} logger;

...
nvinfer1::IBuilder* builder = nvinfer1::createInferBuilder(logger);
nvinfer1::IBuilderConfig* config = builder->createBuilderConfig();
...

Thanks.

7

Thanks,

Following attached is the full output:
can_run_on_DLA.log (58.5 KB)

The log returns following error:
Network built for DLA requires kENTROPY_CALIBRATION_2 calibrator.

But my calibrator uses the type it says, here is my calibrator construct:

class MyCalibrator : public IInt8EntropyCalibrator2
{
private:
    int         nCalibration {0};
    int         nElement {0};
    size_t      bufferSize {0};
    int         nBatch {0};
    int         iBatch {0};
    float      *pData {nullptr};
    void      *bufferD {nullptr};
    cnpy::NpyArray array;
    Dims32      dim;
    std::string cacheFile {""};

public:
    MyCalibrator(const std::string &calibrationDataFile, const int nCalibration, const Dims32 inputShape, const std::string &cacheFile);
    ~MyCalibrator() noexcept;
    int32_t     getBatchSize() const noexcept;
    bool        getBatch(void *bindings[], char const *names[], int32_t nbBindings) noexcept;
    void const *readCalibrationCache(std::size_t &length) noexcept;
    void        writeCalibrationCache(void const *ptr, std::size_t length) noexcept;
};
8

Hi,

Do you want to apply calibration with DLA output?
Suppose you can do this with GPU and use the same cache for DLA inference.

Could you help to turn off the calibration to see if it can work?

Thanks.

9

I finally found out that this is because after I changed my header file to inherit Calibrator for DLA, since the header file is not in the target file’s dependencies, the target file won’t compile the header file changes again, and the target file will still use Calibrator which is not applicable to DLA.
My solution: add the header file into makefile (or My solution is to add the header file to the makefile (or cmake) file and recompile the engine to use Calibrator for DLA.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.