Enable DLA Mode For A Layer During Network Creation

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
#1

Hi,

I tried to enable dla mode for a layer following the example [10.1.2. Example 2: Enable DLA Mode For A Layer During Network Creation], Since class nvinfer1::ICudaEngine has no member named ‘setDLACore’, I replaced engine->setDLACore(0) with builder->setDLACore(0). But it seems no layer run on DLA.

By the way, is there any sample code to do this?

#3

Hi,

Do you check this via the VERBOSE output from TensorRT?
If not, could you share how do you confirm this?

To run a layer on the DLA, please check if DLA can support the layer type first.
https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-713/developer-guide/index.html#dla_layers

if(canRunOnDLA(conv1))
{
    config->setFlag(BuilderFlag::kFP16); or config->setFlag(BuilderFlag::kINT8);
    builder->setDeviceType(conv1, DeviceType::kDLA); 
}

Thanks.

#4

yes, the verbose output show as bellow:

[0]: (Unnamed Layer* 0) [Convolution] set to DLA
   0: (Unnamed Layer* 0) [Convolution], 1
[1]: (Unnamed Layer* 1) [Convolution] set to DLA
   1: (Unnamed Layer* 1) [Convolution], 1
[2]: (Unnamed Layer* 2) [Convolution] set to DLA
   2: (Unnamed Layer* 2) [Convolution], 1
[3]: (Unnamed Layer* 3) [Convolution] set to DLA
   3: (Unnamed Layer* 3) [Convolution], 1
[TRT Info]
[TRT Info] --------------- Layers running on DLA:
[TRT Info]
[TRT Info] --------------- Layers running on GPU:
[TRT Info] (Unnamed Layer* 0) [Convolution], (Unnamed Layer* 1) [Convolution], (Unnamed Layer* 2) [Convolution], (Unnamed Layer* 3) [Convolution],
[TRT Info] Detected 1 inputs and 1 output network tensors.

and the code is:

config_->setFlag(nvinfer1::BuilderFlag::kFP16);
for (int i = 0; i < network->getNbLayers(); ++i) {
    auto layer = network->getLayer(i);
    auto layer_name = layer->getName();
    if (builder_->canRunOnDLA(layer)) {
        printf("[%d]: %s set to DLA\n", i, layer_name);
        layer->setPrecision(nvinfer1::DataType::kHALF);
        builder_->setDeviceType(layer, nvinfer1::DeviceType::kDLA);
        builder_->setDLACore(0);
    }
    int device_type = int(builder_->getDeviceType(layer));
    printf("%4d: %s, %d\n", i, layer_name, device_type);
}

Thanks

#5

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Do you also set the DLA core when launching the TensorRT engine?

engine->setDLACore(0)

Thanks