we received our AGX Development Kit with DRIVE Software 8.0 and a working internet connection on Xavier A and B. We followed the installation notes and pre-flashed PCIe switch (use case 4.0). Afterwards we flashed Xavier A to version 9.0 (Xavier B is still version 8.0). The flashing procedure finished successfully. Since then we have no internet access on Xavier B which uses SW 8.0. However, the connection on Xavier A (SW 9.0) is still working. The command ifconfig shows that no ip address is assigned to eth0 of Xavier B.
We´ve tested a lot and observed that an ip address could only be assigned to eth0 (Xavier B) when Xavier A was shut down at the same time. The ethernet cable is connected via ethernet dongle (E3579-1.0.3) to port 3|4 (1GbE / 100MbE) but we also tested various setups (e.g. port 1|2, port XB, 10GbE ports on the back panel).
Our goal is to upgrade Xavier B to SW 9.0, too. However, for our knowledge the flashing procedures requires a internet connection. Do you have any suggestions how to solve this problem?
It looks you only did export debug logs but didn’t try to reproduce the issue today.
So I checked the earlier messages inside. Do you think this is the messages relevant to the issue you are facing? If yes, could you try to run the .run file manually and see if you can figure out what’s the problem?
2019-12-11 11:30:32.754 - info: Start to install 'DRIVE AV on Target'
2019-12-11 11:30:32.755 - info: estimated install timeout for NV_DRIVE_AV_TARGET_COMP: 3376s
2019-12-11 11:30:32.759 - info: [INPUT] "cd '/home/had/nvidia/nvidia_sdk/DRIVE_Software_9.0_Linux_hyperion_E3550' ; '/opt/nvidia/sdkmanager/resources/app/output/installUtils/adapter' -a='install' -c='eyJpZCI6Ik5WX0RSSVZFX0FWX1RBUkdFVF9DT01QIiwibmFtZSI6IkRSSVZFIEFWIG9uIFRhcmdldCIsImRlc2NyaXB0aW9uIjoiTlZJRElBIERSSVZFIEFWIGZvciBUYXJnZXQuIiwiY29tcFR5cGUiOiJ0YXJnZXRfaW1hZ2UiLCJpc1Zpc2libGUiOnRydWUsImxpY2Vuc2VJZCI6Ik5WX0RSSVZFX1NMQV9ERUMyMDE4IiwiaXNEZXRlY3RhYmxlSW5zdGFsbCI6ZmFsc2UsImlzSW5zdGFsbFBhdGhDdXN0b21pemFibGUiOnRydWUsInZlcnNpb24iOiIyLjAiLCJvcGVyYXRpbmdTeXN0ZW1zIjpbInVidW50dTE0MDQiLCJ1YnVudHUxNjA0Il0sImluc3RhbGxTaXplTUIiOjE1ODEwLjU2LCJkb3dubG9hZEZpbGVzIjpbeyJ1cmwiOiJkcml2ZS10MTg2cmVmLWxpbnV4LTUuMS4wLjItMTUxOTE2MDQtYWFyY2g2NC1kcml2ZWF2LXYyLjAuMjA4MV8xNTIwOTYwMl8yMDE5MDUwNl8zNmIxMjdmXzE0OTMyMjM1X2RyaXZlLWxpbnV4LTUuMS4wLjIuZGViLnJ1biIsImZpbGVOYW1lIjoiZHJpdmUtdDE4NnJlZi1saW51eC01LjEuMC4yLTE1MTkxNjA0LWFhcmNoNjQtZHJpdmVhdi12Mi4wLjIwODFfMTUyMDk2MDJfMjAxOTA1MDZfMzZiMTI3Zl8xNDkzMjIzNV9kcml2ZS1saW51eC01LjEuMC4yLmRlYi5ydW4iLCJzaXplIjo2NzUzNDc3MDI2LCJjaGVja3N1bSI6IjM0ZWExYWEwZjhiNmFhYzUyN2Q0OTU4ZTc3Yjg4NmYxIiwiY2hlY2tzdW1UeXBlIjoibWQ1IiwiaW5zdGFsbFBhcmFtZXRlcnMiOnsiaW5zdGFsbFR5cGUiOiJydW4iLCJhZGRpdGlvbmFsUGFyYW1ldGVycyI6eyJpbnN0YWxsRGlyIjoie3tUQVJHRVRfUk9PVH19L3t7QlVORExFX1RJVExFfX1fe3tCT0FSRF9JRH19L0RyaXZlU0RLIiwiaW5zdGFsbENvbW1hbmRBcmd1bWVudHMiOiItLW5veDExIn19LCJjYWNoZWRVcmwiOiJodHRwczovL2RldmVsb3Blci5udmlkaWEuY29tL2RyaXZlL3NlY3VyZS9kcml2ZV9idW5kbGVzL0RSSVZFXzUxMDJMX1NES19IeXBlcmlvbl9iNjUvZHJpdmUtdDE4NnJlZi1saW51eC01LjEuMC4yLTE1MTkxNjA0LWFhcmNoNjQtZHJpdmVhdi12Mi4wLjIwODFfMTUyMDk2MDJfMjAxOTA1MDZfMzZiMTI3Zl8xNDkzMjIzNV9kcml2ZS1saW51eC01LjEuMC4yLmRlYi5ydW4iLCJjdXJyZW50RG93bmxvYWRlZFVybCI6Imh0dHBzOi8vZGV2ZWxvcGVyLm52aWRpYS5jb20vZHJpdmUvc2VjdXJlL2RyaXZlX2J1bmRsZXMvRFJJVkVfNTEwMkxfU0RLX0h5cGVyaW9uX2I2NS9kcml2ZS10MTg2cmVmLWxpbnV4LTUuMS4wLjItMTUxOTE2MDQtYWFyY2g2NC1kcml2ZWF2LXYyLjAuMjA4MV8xNTIwOTYwMl8yMDE5MDUwNl8zNmIxMjdmXzE0OTMyMjM1X2RyaXZlLWxpbnV4LTUuMS4wLjIuZGViLnJ1biJ9XSwidGFyZ2V0SWRzIjpbIkREUFgiXSwiZGVwZW5kZW5jaWVzIjpbeyJpZCI6Ik5WX1RFTlNPUlJUX1RBUkdFVF9DT01QIiwib3BlcmF0b3IiOiI+PSIsInZlciI6IjUuMCIsIl9pZCI6Ik5WX1RFTlNPUlJUX1RBUkdFVF9DT01QQEREUFgifV0sImV4dGVybmFsX2RlcGVuZGVuY2llcyI6W10sInRhcmdldElEIjoiRERQWCIsIl9pZCI6Ik5WX0RSSVZFX0FWX1RBUkdFVF9DT01QQEREUFgiLCJwYXJlbnRJRCI6Ik5WX0RSSVZFX0FWX1RBUkdFVF9HUk9VUCIsIl9wYXJlbnRJRCI6Ik5WX0RSSVZFX0FWX1RBUkdFVF9HUk9VUEBERFBYIiwiaW5zdGFsbFN0YXR1cyI6Im5vbmUiLCJhY3Rpb24iOjEsImluc3RhbGxQYXRoIjoiL2hvbWUvaGFkL252aWRpYS9udmlkaWFfc2RrL0RSSVZFX1NvZnR3YXJlXzkuMF9MaW51eF9oeXBlcmlvbl9FMzU1MCIsImRvd25sb2FkRXJyb3JzIjpbXSwiZXhlYyI6Ii9ob21lL2hhZC9Eb3dubG9hZHMvbnZpZGlhL3Nka21fZG93bmxvYWRzL2RyaXZlLXQxODZyZWYtbGludXgtNS4xLjAuMi0xNTE5MTYwNC1hYXJjaDY0LWRyaXZlYXYtdjIuMC4yMDgxXzE1MjA5NjAyXzIwMTkwNTA2XzM2YjEyN2ZfMTQ5MzIyMzVfZHJpdmUtbGludXgtNS4xLjAuMi5kZWIucnVuIiwiYnVuZGxlVGl0bGUiOiJEUklWRV9Tb2Z0d2FyZV85LjBfTGludXhfaHlwZXJpb24iLCJpbnN0YWxsUm9vdCI6Ii9ob21lL2hhZC9udmlkaWEvbnZpZGlhX3NkayIsImJvYXJkSUQiOiJFMzU1MCIsImlzRHJ5UnVuIjpmYWxzZX0=' -i='NV_DRIVE_AV_TARGET_COMP' -v='2.0' -l='/home/had/nvidia/nvidia_sdk/DRIVE_Software_9.0_Linux_hyperion_E3550' -L='/home/had/.nvsdkm/logs/DRIVE_Software_9.0_Linux_hyperion' -p='2567' -t='DDPX' -e='/home/had/Downloads/nvidia/sdkm_downloads/drive-t186ref-linux-5.1.0.2-15191604-aarch64-driveav-v2.0.2081_15209602_20190506_36b127f_14932235_drive-linux-5.1.0.2.deb.run' -b='NV_DRIVE_LINUX_BUNDLE' -x='SDK' && echo SDKM_END_CODE_SUCCESS_2e087c8c-615b-4510-8b99-f9c280759b6d || echo SDKM_END_CODE_FAILURE_2e087c8c-615b-4510-8b99-f9c280759b6d_$?\r"
2019-12-11 11:30:32.768 - info: [ Disk Avail:46.69 GB ]
2019-12-11 11:30:32.769 - info: Using GenericInstaller to Install NV_DRIVE_AV_TARGET_COMP
2019-12-11 11:30:32.769 - info: NV_DRIVE_AV_TARGET_COMP include 1 package
2019-12-11 11:30:32.769 - info: change working directory to /home/had/nvidia/nvidia_sdk/DRIVE_Software_9.0_Linux_hyperion_E3550/DriveSDK
2019-12-11 11:30:32.769 - info: [ Package Install Started ]
2019-12-11 11:30:32.769 - info: runfile installer start to install NV_DRIVE_AV_TARGET_COMP
2019-12-11 11:30:32.769 - info: change working directory to /home/had/nvidia/nvidia_sdk/DRIVE_Software_9.0_Linux_hyperion_E3550/DriveSDK
2019-12-11 11:30:32.769 - info: exec_command: /home/had/Downloads/nvidia/sdkm_downloads/drive-t186ref-linux-5.1.0.2-15191604-aarch64-driveav-v2.0.2081_15209602_20190506_36b127f_14932235_drive-linux-5.1.0.2.deb.run --nox11
2019-12-11 11:30:32.769 - error: /bin/bash: /home/had/Downloads/nvidia/sdkm_downloads/drive-t186ref-linux-5.1.0.2-15191604-aarch64-driveav-v2.0.2081_15209602_20190506_36b127f_14932235_drive-linux-5.1.0.2.deb.run: /bin/bash: bad interpreter: Text file busy
2019-12-11 11:30:32.769 - info: exit status 126
2019-12-11 11:30:32.769 - info: [ Package Install Finished with Error ]
2019-12-11 11:30:32.769 - info: [ 4.00 KB used. Disk Avail: 46.69 GB ]
2019-12-11 11:30:32.769 - info: [ NV_DRIVE_AV_TARGET_COMP Install took 0s ]
2019-12-11 11:30:32.769 - error: execution command /home/had/Downloads/nvidia/sdkm_downloads/drive-t186ref-linux-5.1.0.2-15191604-aarch64-driveav-v2.0.2081_15209602_20190506_36b127f_14932235_drive-linux-5.1.0.2.deb.run --nox11 returns 126
2019-12-11 11:30:32.770 - info:
2019-12-11 11:30:32.770 - error: cmd finished failure SDKM_END_CODE_FAILURE_2e087c8c-615b-4510-8b99-f9c280759b6d_33
2019-12-11 11:30:32.770 - error: command error code: 33
2019-12-11 11:30:32.770 - error: command terminated with error
Unfortunately, the uploaded log files are not the latest ones. Sorry for that and please find the latest log files attached. I also added the output of ifconfig command for each board.
Could you specify ‘reproduce the issue’? We flashed Xavier A only one time (from 8.0 to 9.0) and the network issue on Xavier B occured right after Xavier A was successfully upgraded to DRIVE SW 9.0. We restarted the system several time but the issue is still there.
So do you think we should flash Xavier A once again using DRIVE SW 9.0? If yes, do you recommend a downgrade of Xavier A back to 8.0 before flashing 9.0 again? SDKM_logs_2019-12-20_11-57-55.zip (1.03 MB)
Sorry that I misunderstood you. I thought you failed to install Drive Software 9.0 onto Xavier B.
It seems you are running two Xavie running different verions. But I don’t think we have validated it. Especially the network topology is complicated on the sytem. Could you check kernel message on Xavier B (run dmesg command)? It will be more clear what happened.
raffael.naida and I are working with the same AGX device.
Yes, we are running two different SW versions on the two Xaviers. We thought it would be possible because there is the option in the SDK Manager to flash only a single Xavier. It’s important to know that the setup of running different SW versions on both Xaviers has not been validated yet. Just for our understanding: what’s the use case for this option in the SDK Manager if both Xaviers should always run with the same SW version?
Please find attached the kernel messages of the command dmesg for Xaviers A and B. There are some error messages for both Xaviers.
Our goal is to upgrade Xavier B to Drive Software 9.0, too. However, to the best of our knowledge we will need a internet connection for Xavier B to be flashed from SW 8.0 to 9.0. Is this assumption correct, or is it possible to flash Xavier B without having direct internet access for this particular Xavier (internet is only available for host PC and Xavier A)? dmesg_logs_xavier_a_and_b.zip (38 KB)
I always tried on my kit with network cable connected but I don’t think the installation needs internet access.
Also, according to the note, “We care about the safety and security of your data; please change your password before enabling the network on DRIVE AGX Developer Kits.”, in this installation page, you can even plug in a network cable after installation is done and password is changed.
Please try with flash Xavier A+B at the same time. Because it was QA’ed, I think the network issue won’t happen with that.
Thanks a lot for the information and your advice. We will try to flash Xavier A+B at the same time.
Do you recommend to downgrade Xavier A to SW 8.0 first (before flashing A+B to DRIVE SW 9.0)?
we followed your advice and flashed Xavier A and B to DRIVE SW 9.0. The situation can be described as follows: basically, Xavier A and B can establish an internet connection but NOT at the same time. So either Xavier A OR Xavier B can establish an internet connection, but not both at the same time.
Do you have any suggestions how to solve this problem?
I’m checking if anyone met this on DRIVE Software 9.0 before. But if possible, please always upgrade to the the latest version (say DRIVE Software 10.0) and see if everything works well.
Thanks for your quick response and your efforts! Which logs do you mean (logs from flashing procedure or syslogs)? We already ran dmesg command but couldn´t detect any errors.
At the moment, upgrading to DRIVE Software 10.0 is no valid option for us because of the missing downgrade path to Version 9.0.
I was talking about logs relevant to the other xavier not able to establish internet connection. If you couldn’t detect any suspicious messages, may I know how you did the test? Could you elaborate the steps in detail?
By the way, do you decide to be stuck in DRIVE Software 9.0 becasue of existing effors/achievements on it? Thanks!
Dear raffel.naida,
I do not see any issue with network when DRIVE SW 9.0 and DRIVE SW 10.0 installed on DRIVE AGX platform. Could you please upgrade to latest DRIVE SW version if possible.
Hi VickNV, hi SivaRamaKrishna,
thanks for your replies. In the meantime the network issues are no longer present. To be honest, we couldn´t figure out the exact reason that caused this problem. We restarted network interface eth0 on Xavier A and B several times using the commands
sudo ifconfig eth0 down
sudo ifconfig eht0 up
sudo dhclient eth0
Then we observed that both board can access the internet at the same time.