Use DLA directly, without TRT?

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
#1

Hi,

I’m wondering if I can submit a nvdla loadable to DLA in Xavier, without using TRT?

More precisely, is the kmd/umd for DLA in Xavier compatible with the open-sourced version here?


If not, can I build a the kmd/umd and replace the original ones in Xavier?

#2

Hi,

TensorRT is the only one high-level API for DLA currently.
But you can still try to access the DLA with the low-level driver.

Thanks.

#3

Hi AastaLLL,

Thanks for reply.
I’m now trying with nvdla_runtime with kmd/umd from Xavier,
but I have found some inconsistency between Xavier’s umd and open-sourced version of umd.
Thus I can’t run open-sourced version of nvdla_runtime over Xavier’s umd.

Anywhere I can find header files for Xavier’s umd?

Thanks!

#4

Hi,

May I know the detail error you meet?

Suppose you can compile and execute with the instruction here:
http://nvdla.org/primer.html#compilation-tools-model-creation-and-compilation

Here is the UMD header of NVDLA:
https://github.com/nvdla/sw/tree/master/umd/include

Thanks.

#5

Hi AastaLLL,

I’m trying to run this example
https://github.com/nvdla/sw/blob/master/umd/tests/runtime/RuntimeTest.cpp
on Xavier’s umd and kmd, instead of open-sourced umd and kmd.

And I run into this error:

if (!i->runtime->initEMU())
    ORIGINATE_ERROR(NvDlaError_DeviceNotFound, "runtime->initEMU() failed");

==
But I found that the open-sourced implementation for “bool Runtime::initEMU(void)” returns true if succeed, and false otherwise, while in Xavier’s implemenation, it seems to return “number of available DLA devices”.

That’s why I suspect the Xavier’s umd is inconsistent with open-sourced one.
I think I’ll need the Xavier’s umd headers, not the open-sourced ones, and modify RuntimeTest.cpp to make it work.

#6

Hi,

Thanks for your feedback.
We will try to reproduce this issue on our side and find some suggestion form our internal team.

Stay tuned.

#7

Thanks.

Btw it’ll be good to also have a runable RuntimeTest.cpp for Xavier DLA.

#8

Hi,

After checking with our internal team, the GitHub test app is for OpenDLA and cannot be used on the Xavier.
A possible workaround is to use nvdla_runtime app from OpenDLA with libnvdla_runtime.so released with the Jetpack.

They are API compatible. Would you mind to give it a try?

Thanks.

#9

I don’t quite get it. Did you mean the libnvdla_runtime.so in
JetPack_4.2_Linux_P2888/Linux_for_Tegra/rootfs/usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so ?

Looks like it’s the same as the one already in Xavier’s file system, which is not working, as described before.

#10

Hi,

Sorry about that.

In general, the GitHub source is for OpenDLA and the version of JetPack4.2 is not open source.
But we think they are API compatible. I will check with the developer again.

Really sorry about status missing.

#11

Hi,

Our internal team is going to check this issue.
Will keep updating the progress with you.

Stay tuned.

#12

Hi AastaLLL,

I see nvdla 1.2.0 released, but issue seems not addressed.
Any update on this?

#13

Hi,

Sorry to keep you waiting.

This issue is still checking by our internal team.
Will update more information with you once we got a feedback.

Thanks.

#14

Hi,

We got an update from our internal team.

Current interface are not consistent. We are planning to align the APIs for Xavier DLA and Open DLA in coming up releases.

Thanks.

#15

Are there any updates on the aligning? I need to use the Xavier DLA without TensorRT, please let me know if there is any way to achieve this. Thanks!