Enable DLA Mode For A Layer During Network Creation


I tried to enable dla mode for a layer following the example [10.1.2. Example 2: Enable DLA Mode For A Layer During Network Creation], Since class nvinfer1::ICudaEngine has no member named ‘setDLACore’, I replaced engine->setDLACore(0) with builder->setDLACore(0). But it seems no layer run on DLA.

By the way, is there any sample code to do this?


Do you check this via the VERBOSE output from TensorRT?
If not, could you share how do you confirm this?

To run a layer on the DLA, please check if DLA can support the layer type first.

    config->setFlag(BuilderFlag::kFP16); or config->setFlag(BuilderFlag::kINT8);
    builder->setDeviceType(conv1, DeviceType::kDLA); 


yes, the verbose output show as bellow:

[0]: (Unnamed Layer* 0) [Convolution] set to DLA
   0: (Unnamed Layer* 0) [Convolution], 1
[1]: (Unnamed Layer* 1) [Convolution] set to DLA
   1: (Unnamed Layer* 1) [Convolution], 1
[2]: (Unnamed Layer* 2) [Convolution] set to DLA
   2: (Unnamed Layer* 2) [Convolution], 1
[3]: (Unnamed Layer* 3) [Convolution] set to DLA
   3: (Unnamed Layer* 3) [Convolution], 1
[TRT Info]
[TRT Info] --------------- Layers running on DLA:
[TRT Info]
[TRT Info] --------------- Layers running on GPU:
[TRT Info] (Unnamed Layer* 0) [Convolution], (Unnamed Layer* 1) [Convolution], (Unnamed Layer* 2) [Convolution], (Unnamed Layer* 3) [Convolution],
[TRT Info] Detected 1 inputs and 1 output network tensors.

and the code is:

for (int i = 0; i < network->getNbLayers(); ++i) {
    auto layer = network->getLayer(i);
    auto layer_name = layer->getName();
    if (builder_->canRunOnDLA(layer)) {
        printf("[%d]: %s set to DLA\n", i, layer_name);
        builder_->setDeviceType(layer, nvinfer1::DeviceType::kDLA);
    int device_type = int(builder_->getDeviceType(layer));
    printf("%4d: %s, %d\n", i, layer_name, device_type);


There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.


Do you also set the DLA core when launching the TensorRT engine?