Unable to load greengrass compiled model in Jetson Nano

I have compiled mobilenet model using AWS. Deployed my model to Jetson Nano Device using Greengrass. When I run inference application, it throws DLR error.
I have installed jetpack4.2.1 using SD Card Image. I can see Cuda10.0; TensorRt 5.1.6; CudNN 7 on my device.

Error StackTrace:

[2020-12-21T12:46:09.789+05:30][ERROR]-init.py:1028,2020-12-21 12:46:09,622 INFO Found libdlr.so in model artifact. Using dlr from /ml_model/libdlr.so
[2020-12-21T12:46:11.974+05:30][ERROR]-NvRmPrivGetChipIdLimited: Could not read Tegra chip id/rev
[2020-12-21T12:46:11.974+05:30][ERROR]-Expected on kernels without fuse support, using Tegra K1
[2020-12-21T12:46:11.974+05:30][ERROR]-NvRmPrivGetChipPlatform: Could not read platform information
[2020-12-21T12:46:11.974+05:30][ERROR]-Expected on kernels without fuse support, using silicon
[2020-12-21T12:46:11.975+05:30][ERROR]-libnvrm_gpu.so: NvRmGpuLibOpen failed
[2020-12-21T12:46:11.984+05:30][ERROR]-[12:46:11] /home/nvidia/neo-ai-dlr/src/dlr.cc:327: [12:46:11] /home/nvidia/neo-ai-dlr/3rdparty/tvm/src/runtime/cuda/cuda_device_api.cc:115: Check failed: e == cudaSuccess || e == cudaErrorCudartUnloading: CUDA: unknown error
[2020-12-21T12:46:11.984+05:30][ERROR]-Stack trace:
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (0) /ml_model/libdlr.so(+0x531b00) [0x7f8cd92b00]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (1) /ml_model/libdlr.so(tvm::runtime::CUDADeviceAPI::AllocDataSpace(DLContext, unsigned long, unsigned long, DLDataType)+0x178) [0x7f8cd97a28]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (2) /ml_model/libdlr.so(+0x13100b0) [0x7f8db710b0]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (3) /ml_model/libdlr.so(+0x13489a8) [0x7f8dba99a8]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (4) /ml_model/libdlr.so(+0x134b528) [0x7f8dbac528]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (5) /ml_model/libdlr.so(dlr::TVMModel::SetupTVMModule(std::vector<ModelElem, std::allocator > const&)+0x3bc) [0x7f8cd8c2a4]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (6) /ml_model/libdlr.so(dlr::TVMModel::SetupTVMModule(std::vector<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const&)+0x174) [0x7f8cd8e244]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (7) /ml_model/libdlr.so(CreateDLRModel+0x2ec) [0x7f8cd40fec]
[2020-12-21T12:46:11.984+05:30][ERROR]- [bt] (8) /usr/lib/aarch64-linux-gnu/libffi.so.6(ffi_call_SYSV+0x64) [0x7f90fb6d28]
[2020-12-21T12:46:12.018+05:30][ERROR]-init.py:1028,2020-12-21 12:46:11,984 ERROR error in DLRModel instantiation
[2020-12-21T12:46:12.018+05:30][ERROR]-Traceback (most recent call last):
[2020-12-21T12:46:12.018+05:30][ERROR]- File “/usr/local/lib/python3.7/dist-packages/dlr/api.py”, line 85, in init
[2020-12-21T12:46:12.018+05:30][ERROR]- self._impl = DLRModelImpl(model_path, dev_type, dev_id, error_log_file, use_default_dlr)
[2020-12-21T12:46:12.018+05:30][ERROR]- File “/usr/local/lib/python3.7/dist-packages/dlr/dlr_model.py”, line 82, in init
[2020-12-21T12:46:12.018+05:30][ERROR]- c_int(dev_id)))
[2020-12-21T12:46:12.018+05:30][ERROR]- File “/usr/local/lib/python3.7/dist-packages/dlr/dlr_model.py”, line 160, in _check_call
[2020-12-21T12:46:12.018+05:30][ERROR]- raise DLRError(self._lib.DLRGetLastError().decode(‘ascii’))
[2020-12-21T12:46:12.018+05:30][ERROR]-dlr.dlr_model.DLRError
[2020-12-21T12:46:12.018+05:30][FATAL]-lambda_runtime.py:142,Failed to import handler function “lambda_function.lambda_handler” due to exception:
[2020-12-21T12:46:12.018+05:30][FATAL]-lambda_runtime.py:382,Failed to initialize Lambda runtime due to exception:

I am not what causes this issue. If you need any further details, please let me know.

Any hint would be appreciable.

Hi,

In general, the inference engine doesn’t support portability since the optimization is hardware dependent.
So to deploy an engine file compiled from another platform may cause some unexpected error.

Would you mind to do the compiling on the Nano directly?
If this is not an option, could you check if the “compiled model” can support portability or not first?

Thanks.