Terminate called after throwing an instance of 'std::logic_error' what(): basic_string::_M_construct null not valid

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Loaded a pre-trained Tensorflow UNET model on AGX Xavier for inference, but got an error and core dump @“model.predict()”. Worked fine on Servers with Tensorflow 2.1 & 2.3 installed. However, a pre-trained PyTorch model worked fine.

error.txt (10.1 KB)

NVIDIA Jetson AGX Xavier [16GB]
L4T 32.4.2 [ JetPack 4.4 DP ]
Ubuntu 18.04.4 LTS
Kernel Version: 4.9.140-roscube.v0.1
CUDA 10.2.89
CUDA Architecture: 7.2
OpenCV version: 4.1.1
OpenCV Cuda: NO
CUDNN: 8.0.0.145
TensorRT: 7.1.0.16
Vision Works: 1.6.0.501
VPI: 0.2.0

3

Hi,

Could you try our latest JetPack4.4.1 first?

There are some bugs fixed in our latest release.
One of them has a similar error log as yours.

Thanks.

4

Hi AastaLLL,
Thanks for the prompt reply.
I used Nvidia’s guide line to upgrade JetPack on my AGX Xavier from 4.4 to 4.4.1, but did not go well.
Looks like I have to resolve the upgrade issues before I can try out the latest JetPack release.
Do I need to open another ticket for this upgrade issue?

Thanks,

  1. Tried “apt depends nvidia-jetpack | awk ‘{print $2}’ | xargs -I {} sudo apt install -y {}”
    get the following error messages:
    The following packages have unmet dependencies:

nvidia-l4t-jetson-multimedia-api : Depends: nvidia-l4t-camera (= 32.4.4-20201027211332) but 32.4.2-20200408182620 is to be installed

                                Depends: nvidia-l4t-multimedia (= 32.4.4-20201027211332) but 32.4.2-20200408182620 is to be installed

                                Depends: nvidia-l4t-multimedia-utils (= 32.4.4-20201027211332) but 32.4.2-20200408182620 is to be installed

E: Unable to correct problems, you have held broken packages.

  1. Tried install SDK manager got the following messages:
    ros@ROScube-X:~/Downloads$ sudo apt install ./sdkmanager_1.3.1-7110_amd64.deb

Reading package lists… Done

Building dependency tree

Reading state information… Done

Note, selecting ‘sdkmanager:amd64’ instead of ‘./sdkmanager_1.3.1-7110_amd64.deb’

Some packages could not be installed. This may mean that you have

requested an impossible situation or if you are using the unstable

distribution that some required packages have not yet been created

or been moved out of Incoming.

The following information may help to resolve the situation:

The following packages have unmet dependencies:

sdkmanager:amd64 : Depends: libgconf-2-4:amd64 but it is not installable

                Depends: libcanberra-gtk-module:amd64 but it is not installable

                Depends: locales:amd64 but it is not installable

E: Unable to correct problems, you have held broken packages.

ros@ROScube-X:~/Downloads$

5

Hi,

Could you try to use SDKmanger for the upgrade first?
Pleaes noted that SDKmanager is launched on a desktop environment and connects to XavierNX with a type-c.

Thanks.

6

The online instruction uses a Linux PC, I do not have a Linux desktop at the moment. May take some more time.

Thanks,

7

Hi AastaLLL,

After numerous tries to refresh the Jetson Xavier, we can either use SDK manger or, use the “flash.sh” file to complete to the upgrade, but when rebooted Jetson Xavier,
it always stuck on the “L4T … license agreement” screen. When it happened, we also found out that we cannot access the type C USB or any other USB ports that we used for keyboard and mouse.

Below is the snapshot of the end of the refresh steps.

[ 248.7498 ] tegradevflash_v2 --write MB1_BCT_b mb1_cold_boot_bct_MB1_sigheader.bct.encrypt

[ 248.7504 ] Bootloader version 01.00.0000

[ 248.9320 ] Writing partition MB1_BCT_b with mb1_cold_boot_bct_MB1_sigheader.bct.encrypt

[ 248.9323 ] […] 100%

[ 248.9541 ]

[ 248.9585 ] tegradevflash_v2 --write MEM_BCT mem_coldboot_sigheader.bct.encrypt

[ 248.9589 ] Bootloader version 01.00.0000

[ 249.1450 ] Writing partition MEM_BCT with mem_coldboot_sigheader.bct.encrypt

[ 249.1452 ] […] 100%

[ 249.1637 ]

[ 249.1664 ] tegradevflash_v2 --write MEM_BCT_b mem_coldboot_sigheader.bct.encrypt

[ 249.1690 ] Bootloader version 01.00.0000

[ 249.3519 ] Writing partition MEM_BCT_b with mem_coldboot_sigheader.bct.encrypt

[ 249.3522 ] […] 100%

[ 249.3699 ]

[ 249.3699 ] Flashing completed

[ 249.3700 ] Coldbooting the device

[ 249.3706 ] tegrarcm_v2 --ismb2

[ 249.7329 ]

[ 249.7358 ] tegradevflash_v2 --reboot coldboot

[ 249.7384 ] Bootloader version 01.00.0000

[ 249.9206 ]

*** The target t186ref has been flashed successfully. ***

Reset the board to boot from internal eMMC.

8

Surprisingly, we downgraded the board to Jetpack 4.4.0 yesterday, the problem still persists.
Need helps to fix this, any suggestions?

9

Hi,

Could you also try it with JetPack 4.4.1?

More, which TensorFlow package do you install?
If the issue continues, could you share the model and source to reproduce it in our environment?

Thanks.

10

No, upgrading it to JetPack 4.4.1 lost access to all the USB ports including type C. Still working on that part of issue.
It is a third party product with AGX Xavier in it, need to work with the vendor to get a fix for their firmware tree!

Could you close this issue for the time being I will for sure test it out as soon as we are able to bring up the 3rd party AGX Xavier box with Jetpack 4.4.1.

Thank you!!

11

Thanks for the feedback.
You can file a new topic if you need our help for the USB port.