Build tensorrt engine use DLA from onnx with trtexec on agx xavier jetpack 4.4 failed

My onnx model is SMOKE, DLA34DCN network
build cmd :
trtexec --onnx=model_test.onnx --saveEngine=model_test.engine --fp16 --useDLACore=0 --allowGPUFallback
I got errors like this:
[10/20/2022-15:17:09] [E] [TRT] …/builder/cudnnBuilderGraphNodes.cpp (207) - Assertion Error in getSupportedFormats: 0 (kCudaElementWiseSupportsBool && “Cuda backend does not support kEQUAL/kGREATER/kLESS”)
[10/20/2022-15:17:09] [E] Engine creation failed
[10/20/2022-15:17:09] [E] Engine set up failed
system info::
Jetpack:4.4 [L4T 32.4.3]
Cuda:10.2
TensorRT: 7.1.3.0
build trt use GPU can be success:
trtexec --onnx=model_test.onnx --saveEngine=model_test.onnx.trt --fp16

Hi,

Based on the error, TensorRT fails when dealing with a boolean layer.
Please noted that DLA only supports FP16 and INT8 input, so ideally the layer should fall back to GPU.

Would you mind testing the model with our latest JetPack 5.0.2?
TensorRT 8 has better support in boolean input and might help with your issue.

Thanks.

Thanks, I test with JetPack 5.0.2 successed.
Then we removed boolean layer,and build with Jetack 4.4 also failed:
trtexec --onnx=model_smoke.onnx --saveEngine=model_smoke.onnx.trt --fp16 --useDLACore=0 --allowGPUFallback

[10/21/2022-10:31:57] [I] [TRT] --------------- Layers running on DLA:
[10/21/2022-10:31:57] [I] [TRT] {(Unnamed Layer* 0) [Convolution],(Unnamed Layer* 1) [Scale],(Unnamed Layer* 2) [Activation],(Unnamed Layer* 3) [Convolution],(Unnamed Layer* 4) [Scale],(Unnamed Layer* 5) [Activation],(Unnamed Layer* 6) [Convolution],(Unnamed Layer* 7) [Scale],(Unnamed Layer* 8) [Activation],(Unnamed Layer* 9) [Pooling],(Unnamed Layer* 10) [Convolution],(Unnamed Layer* 11) [Scale],(Unnamed Layer* 12) [Convolution],(Unnamed Layer* 13) [Scale],(Unnamed Layer* 14) [Activation],(Unnamed Layer* 15) [Convolution],(Unnamed Layer* 16) [Scale],(Unnamed Layer* 17) [ElementWise],(Unnamed Layer* 18) [Activation],(Unnamed Layer* 19) [Convolution],(Unnamed Layer* 20) [Scale],(Unnamed Layer* 21) [Activation],(Unnamed Layer* 22) [Convolution],(Unnamed Layer* 23) [Scale],(Unnamed Layer* 24) [ElementWise],(Unnamed Layer* 25) [Activation],(Unnamed Layer* 26) [Concatenation],(Unnamed Layer* 27) [Convolution],(Unnamed Layer* 28) [Scale],(Unnamed Layer* 29) [Activation],(Unnamed Layer* 30) [Pooling],(Unnamed Layer* 31) [Pooling],(Unnamed Layer* 32) [Convolution],(Unnamed Layer* 33) [Scale],(Unnamed Layer* 34) [Convolution],(Unnamed Layer* 35) [Scale],(Unnamed Layer* 36) [Activation],(Unnamed Layer* 37) [Convolution],(Unnamed Layer* 38) [Scale],(Unnamed Layer* 39) [ElementWise],(Unnamed Layer* 40) [Activation],(Unnamed Layer* 41) [Convolution],(Unnamed Layer* 42) [Scale],(Unnamed Layer* 43) [Activation],(Unnamed Layer* 44) [Convolution],(Unnamed Layer* 45) [Scale],(Unnamed Layer* 46) [ElementWise],(Unnamed Layer* 47) [Activation],(Unnamed Layer* 48) [Concatenation],(Unnamed Layer* 49) [Convolution],(Unnamed Layer* 50) [Scale],(Unnamed Layer* 51) [Activation],(Unnamed Layer* 52) [Convolution],(Unnamed Layer* 53) [Scale],(Unnamed Layer* 54) [Activation],(Unnamed Layer* 55) [Convolution],(Unnamed Layer* 56) [Scale],(Unnamed Layer* 57) [ElementWise],(Unnamed Layer* 58) [Activation],(Unnamed Layer* 59) [Convolution],(Unnamed Layer* 60) [Scale],(Unnamed Layer* 61) [Activation],(Unnamed Layer* 62) [Convolution],(Unnamed Layer* 63) [Scale],(Unnamed Layer* 64) [ElementWise],(Unnamed Layer* 65) [Activation],(Unnamed Layer* 66) [Concatenation],(Unnamed Layer* 67) [Convolution],(Unnamed Layer* 68) [Scale],(Unnamed Layer* 69) [Activation],(Unnamed Layer* 70) [Pooling],(Unnamed Layer* 71) [Pooling],(Unnamed Layer* 72) [Convolution],(Unnamed Layer* 73) [Scale],(Unnamed Layer* 74) [Convolution],(Unnamed Layer* 75) [Scale],(Unnamed Layer* 76) [Activation],(Unnamed Layer* 77) [Convolution],(Unnamed Layer* 78) [Scale],(Unnamed Layer* 79) [ElementWise],(Unnamed Layer* 80) [Activation],(Unnamed Layer* 81) [Convolution],(Unnamed Layer* 82) [Scale],(Unnamed Layer* 83) [Activation],(Unnamed Layer* 84) [Convolution],(Unnamed Layer* 85) [Scale],(Unnamed Layer* 86) [ElementWise],(Unnamed Layer* 87) [Activation],(Unnamed Layer* 88) [Concatenation],(Unnamed Layer* 89) [Convolution],(Unnamed Layer* 90) [Scale],(Unnamed Layer* 91) [Activation],(Unnamed Layer* 92) [Convolution],(Unnamed Layer* 93) [Scale],(Unnamed Layer* 94) [Activation],(Unnamed Layer* 95) [Convolution],(Unnamed Layer* 96) [Scale],(Unnamed Layer* 97) [ElementWise],(Unnamed Layer* 98) [Activation],(Unnamed Layer* 99) [Convolution],(Unnamed Layer* 100) [Scale],(Unnamed Layer* 101) [Activation],(Unnamed Layer* 102) [Convolution],(Unnamed Layer* 103) [Scale],(Unnamed Layer* 104) [ElementWise],(Unnamed Layer* 105) [Activation],(Unnamed Layer* 106) [Concatenation],(Unnamed Layer* 107) [Convolution],(Unnamed Layer* 108) [Scale],(Unnamed Layer* 109) [Activation],(Unnamed Layer* 110) [Pooling],(Unnamed Layer* 111) [Convolution],(Unnamed Layer* 112) [Scale],(Unnamed Layer* 113) [Convolution],(Unnamed Layer* 114) [Scale],(Unnamed Layer* 115) [Activation],(Unnamed Layer* 116) [Convolution],(Unnamed Layer* 117) [Scale],(Unnamed Layer* 118) [ElementWise],(Unnamed Layer* 119) [Activation],(Unnamed Layer* 120) [Convolution],(Unnamed Layer* 121) [Scale],(Unnamed Layer* 122) [Activation],(Unnamed Layer* 123) [Convolution],(Unnamed Layer* 124) [Scale],(Unnamed Layer* 125) [ElementWise],(Unnamed Layer* 126) [Activation],(Unnamed Layer* 127) [Concatenation],(Unnamed Layer* 128) [Convolution],(Unnamed Layer* 129) [Scale],(Unnamed Layer* 130) [Activation],(Unnamed Layer* 131) [Convolution],(Unnamed Layer* 132) [Scale],(Unnamed Layer* 133) [Activation],(Unnamed Layer* 139) [Convolution],(Unnamed Layer* 140) [Scale],(Unnamed Layer* 141) [Activation],(Unnamed Layer* 155) [Convolution],(Unnamed Layer* 156) [Scale],(Unnamed Layer* 157) [Activation]}, {(Unnamed Layer* 135) [ElementWise],(Unnamed Layer* 136) [Convolution],(Unnamed Layer* 137) [Scale],(Unnamed Layer* 138) [Activation],(Unnamed Layer* 143) [ElementWise],(Unnamed Layer* 144) [Convolution],(Unnamed Layer* 145) [Scale],(Unnamed Layer* 146) [Activation],(Unnamed Layer* 147) [Convolution],(Unnamed Layer* 148) [Scale],(Unnamed Layer* 149) [Activation],(Unnamed Layer* 159) [ElementWise],(Unnamed Layer* 160) [Convolution],(Unnamed Layer* 161) [Scale],(Unnamed Layer* 162) [Activation],(Unnamed Layer* 163) [Convolution],(Unnamed Layer* 164) [Scale],(Unnamed Layer* 165) [Activation],(Unnamed Layer* 187) [Convolution],(Unnamed Layer* 188) [Scale],(Unnamed Layer* 189) [Activation]}, {(Unnamed Layer* 151) [ElementWise],(Unnamed Layer* 152) [Convolution],(Unnamed Layer* 153) [Scale],(Unnamed Layer* 154) [Activation],(Unnamed Layer* 167) [ElementWise],(Unnamed Layer* 168) [Convolution],(Unnamed Layer* 169) [Scale],(Unnamed Layer* 170) [Activation],(Unnamed Layer* 171) [Convolution],(Unnamed Layer* 172) [Scale],(Unnamed Layer* 173) [Activation],(Unnamed Layer* 179) [Convolution],(Unnamed Layer* 180) [Scale],(Unnamed Layer* 181) [Activation]}, {(Unnamed Layer* 175) [ElementWise],(Unnamed Layer* 176) [Convolution],(Unnamed Layer* 177) [Scale],(Unnamed Layer* 178) [Activation],(Unnamed Layer* 183) [ElementWise],(Unnamed Layer* 184) [Convolution],(Unnamed Layer* 185) [Scale],(Unnamed Layer* 186) [Activation],(Unnamed Layer* 191) [ElementWise],(Unnamed Layer* 192) [Convolution],(Unnamed Layer* 193) [Scale],(Unnamed Layer* 194) [Activation],(Unnamed Layer* 195) [Convolution],(Unnamed Layer* 196) [Scale],(Unnamed Layer* 197) [Activation],(Unnamed Layer* 198) [Convolution],(Unnamed Layer* 199) [Convolution],(Unnamed Layer* 200) [Scale],(Unnamed Layer* 201) [Activation],(Unnamed Layer* 202) [Convolution],(Unnamed Layer* 203) [Activation],(Unnamed Layer* 204) [Pooling]},
[10/21/2022-10:31:57] [I] [TRT] --------------- Layers running on GPU:
[10/21/2022-10:31:57] [I] [TRT] (Unnamed Layer* 134) [Deconvolution], (Unnamed Layer* 142) [Deconvolution], (Unnamed Layer* 158) [Deconvolution], (Unnamed Layer* 150) [Deconvolution], (Unnamed Layer* 166) [Deconvolution], (Unnamed Layer* 190) [Deconvolution], (Unnamed Layer* 174) [Deconvolution], (Unnamed Layer* 182) [Deconvolution],
[10/21/2022-10:31:59] [W] [TRT] DLA Node compilation Failed.
[10/21/2022-10:31:59] [W] [TRT] DLA Node compilation Failed.
[10/21/2022-10:31:59] [E] [TRT] Try increasing the workspace size with IBuilderConfig::setMaxWorkspaceSize() if using IBuilder::buildEngineWithConfig, or IBuilder::setMaxWorkspaceSize() if using IBuilder::buildCudaEngine.
[10/21/2022-10:31:59] [E] [TRT] …/builder/tacticOptimizer.cpp (1715) - TRTInternal Error in computeCosts: 0 (Could not find any implementation for node {(Unnamed Layer* 0) [Convolution],(Unnamed Layer* 1) [Scale],(Unnamed Layer* 2) [Activation],(Unnamed Layer* 3) [Convolution],(Unnamed Layer* 4) [Scale],(Unnamed Layer* 5) [Activation],(Unnamed Layer* 6) [Convolution],(Unnamed Layer* 7) [Scale],(Unnamed Layer* 8) [Activation],(Unnamed Layer* 9) [Pooling],(Unnamed Layer* 10) [Convolution],(Unnamed Layer* 11) [Scale],(Unnamed Layer* 12) [Convolution],(Unnamed Layer* 13) [Scale],(Unnamed Layer* 14) [Activation],(Unnamed Layer* 15) [Convolution],(Unnamed Layer* 16) [Scale],(Unnamed Layer* 17) [ElementWise],(Unnamed Layer* 18) [Activation],(Unnamed Layer* 19) [Convolution],(Unnamed Layer* 20) [Scale],(Unnamed Layer* 21) [Activation],(Unnamed Layer* 22) [Convolution],(Unnamed Layer* 23) [Scale],(Unnamed Layer* 24) [ElementWise],(Unnamed Layer* 25) [Activation],(Unnamed Layer* 26) [Concatenation],(Unnamed Layer* 27) [Convolution],(Unnamed Layer* 28) [Scale],(Unnamed Layer* 29) [Activation],(Unnamed Layer* 30) [Pooling],(Unnamed Layer* 31) [Pooling],(Unnamed Layer* 32) [Convolution],(Unnamed Layer* 33) [Scale],(Unnamed Layer* 34) [Convolution],(Unnamed Layer* 35) [Scale],(Unnamed Layer* 36) [Activation],(Unnamed Layer* 37) [Convolution],(Unnamed Layer* 38) [Scale],(Unnamed Layer* 39) [ElementWise],(Unnamed Layer* 40) [Activation],(Unnamed Layer* 41) [Convolution],(Unnamed Layer* 42) [Scale],(Unnamed Layer* 43) [Activation],(Unnamed Layer* 44) [Convolution],(Unnamed Layer* 45) [Scale],(Unnamed Layer* 46) [ElementWise],(Unnamed Layer* 47) [Activation],(Unnamed Layer* 48) [Concatenation],(Unnamed Layer* 49) [Convolution],(Unnamed Layer* 50) [Scale],(Unnamed Layer* 51) [Activation],(Unnamed Layer* 52) [Convolution],(Unnamed Layer* 53) [Scale],(Unnamed Layer* 54) [Activation],(Unnamed Layer* 55) [Convolution],(Unnamed Layer* 56) [Scale],(Unnamed Layer* 57) [ElementWise],(Unnamed Layer* 58) [Activation],(Unnamed Layer* 59) [Convolution],(Unnamed Layer* 60) [Scale],(Unnamed Layer* 61) [Activation],(Unnamed Layer* 62) [Convolution],(Unnamed Layer* 63) [Scale],(Unnamed Layer* 64) [ElementWise],(Unnamed Layer* 65) [Activation],(Unnamed Layer* 66) [Concatenation],(Unnamed Layer* 67) [Convolution],(Unnamed Layer* 68) [Scale],(Unnamed Layer* 69) [Activation],(Unnamed Layer* 70) [Pooling],(Unnamed Layer* 71) [Pooling],(Unnamed Layer* 72) [Convolution],(Unnamed Layer* 73) [Scale],(Unnamed Layer* 74) [Convolution],(Unnamed Layer* 75) [Scale],(Unnamed Layer* 76) [Activation],(Unnamed Layer* 77) [Convolution],(Unnamed Layer* 78) [Scale],(Unnamed Layer* 79) [ElementWise],(Unnamed Layer* 80) [Activation],(Unnamed Layer* 81) [Convolution],(Unnamed Layer* 82) [Scale],(Unnamed Layer* 83) [Activation],(Unnamed Layer* 84) [Convolution],(Unnamed Layer* 85) [Scale],(Unnamed Layer* 86) [ElementWise],(Unnamed Layer* 87) [Activation],(Unnamed Layer* 88) [Concatenation],(Unnamed Layer* 89) [Convolution],(Unnamed Layer* 90) [Scale],(Unnamed Layer* 91) [Activation],(Unnamed Layer* 92) [Convolution],(Unnamed Layer* 93) [Scale],(Unnamed Layer* 94) [Activation],(Unnamed Layer* 95) [Convolution],(Unnamed Layer* 96) [Scale],(Unnamed Layer* 97) [ElementWise],(Unnamed Layer* 98) [Activation],(Unnamed Layer* 99) [Convolution],(Unnamed Layer* 100) [Scale],(Unnamed Layer* 101) [Activation],(Unnamed Layer* 102) [Convolution],(Unnamed Layer* 103) [Scale],(Unnamed Layer* 104) [ElementWise],(Unnamed Layer* 105) [Activation],(Unnamed Layer* 106) [Concatenation],(Unnamed Layer* 107) [Convolution],(Unnamed Layer* 108) [Scale],(Unnamed Layer* 109) [Activation],(Unnamed Layer* 110) [Pooling],(Unnamed Layer* 111) [Convolution],(Unnamed Layer* 112) [Scale],(Unnamed Layer* 113) [Convolution],(Unnamed Layer* 114) [Scale],(Unnamed Layer* 115) [Activation],(Unnamed Layer* 116) [Convolution],(Unnamed Layer* 117) [Scale],(Unnamed Layer* 118) [ElementWise],(Unnamed Layer* 119) [Activation],(Unnamed Layer* 120) [Convolution],(Unnamed Layer* 121) [Scale],(Unnamed Layer* 122) [Activation],(Unnamed Layer* 123) [Convolution],(Unnamed Layer* 124) [Scale],(Unnamed Layer* 125) [ElementWise],(Unnamed Layer* 126) [Activation],(Unnamed Layer* 127) [Concatenation],(Unnamed Layer* 128) [Convolution],(Unnamed Layer* 129) [Scale],(Unnamed Layer* 130) [Activation],(Unnamed Layer* 131) [Convolution],(Unnamed Layer* 132) [Scale],(Unnamed Layer* 133) [Activation],(Unnamed Layer* 139) [Convolution],(Unnamed Layer* 140) [Scale],(Unnamed Layer* 141) [Activation],(Unnamed Layer* 155) [Convolution],(Unnamed Layer* 156) [Scale],(Unnamed Layer* 157) [Activation]}.)
[10/21/2022-10:31:59] [E] [TRT] …/builder/tacticOptimizer.cpp (1715) - TRTInternal Error in computeCosts: 0 ()
[10/21/2022-10:31:59] [E] Engine creation failed
[10/21/2022-10:31:59] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=model_smoke.onnx --saveEngine=model_smoke.onnx.trt --fp16 --useDLACore=0 --allowGPUFallback

model_smoke.onnx size :74MB
is this size supported on DLA?
we wish it can be successed with Jetpack4.4. And it’s need long times to upgrade all of our private drivers to support Jetpack5.0.2

Hi,

It’s still recommended to upgrade the software for better support.
Since JetPack 4.4 is released years ago, you might meet lots of issues when using it with a modern network model.

Thanks.

Ok.Thank you very mush!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.