Wrong result from DLA

I have been using Xavier AGX for a while. Recently we are deploying shufflenet on it, and I noticed that DLA is giving me completely wrong result:

The model is basically a customer trained shufflenet v2, from

We converted into onnx and load into TensorRT(C++, nvinfer) to run. I can run it correctly with GPU, but if i run it with DLA, it gives completely wrong result. (images that gives 0.98 score on GPU turns out to be 0.001 on DLA).

I am sure the other part of the system is working correctly as I get same(but wrong) results from DLA if I run the same input multiple times; I also get different result for different input. Therefore, I am pretty sure the input, output is setup correctly.

Can we fully trust the engine building part of tensorRT to only schedule what DLA can successfully run on DLA?

Hi,

Which TensorRT version do you use?
If you are not using v7.1.3, it’s recommended to upgrade to it first.

Could you share the ONNX model, input, and expected output with us to do a more in-depth investigation?
In general, TensorRT builder will fallback the non-supported layer into GPU.
But since the layer configuration can be very different cross models, we will need the model to figure out the real cause.

Thanks.

yes it is V7.1.3

Here is the model. Just run it with GPU and DLA gives vastly different result.

shufflenet_fs.onnx (8.7 MB)

@AastaLLL, thanks for looking into this:

here is a better example:

with the shufflenet_fs.onnx provided from my last comment with this input: output1.dat (588 KB)

I can run the model with trtexec:
GPU:
/usr/src/tensorrt/bin/trtexec --onnx=/tmp/shufflenet_fs.onnx --loadInputs=data:/tmp/output1.dat --dumpOutput

result:

-0.531341 -0.392079 -2.12658 -2.96252 -2.2261 0.697449 -1.04333 -2.84307 -3.87462 -2.13854 -1.52387 -3.24468 -4.39194 -5.04513 ...

DLA:

/usr/src/tensorrt/bin/trtexec --onnx=/tmp/shufflenet_fs.onnx --loadInputs=data:/tmp/output1.dat --dumpOutput --useDLACore=0 --allowGPUFallback

result:
56.7812 13.0156 -28.7969 20 36.9375 -0.486328 42.0312 3.17773 -43.75 38.4688 -68.5 40.875 -2.60352 ...

I have run it multiple times, GPU and DLA both gives stable results, but different.

Update: just double checked:
with trtexec, the GPU result is stable. the DLA result is not fully stable, but generally not too far from run to run.
The DLA result is still very different from the GPU result, even the trends are different. I also can verify that the GPU result are the expected result.

Hi,

Thanks for your model and input data.
We confirm that the same issue also occurs in our environment.

Our internal team is now working on this.
Will share more information with you for any progress.

Thanks.