SDK Manager Issues

It’s not clear to me why the SDK Manager was designed the way it is.

The entire thing seems overly complicated, and error prone to the point where I’m on the second day of still trying to get it to work without success. I never had this kind of issue with the Jetpacks in my experience with the TK1, TX1, TX2, and Xavier.

The biggest issue with the SDK manager is it doesn’t seem to be designed to be failure friendly. So it always wants to recreate things, and excessively download things.

Or at least it appears to be.

If the flash fails the next time it will recreate the OS image even though the OS image has already been created. Sometimes this happens quickly (like simply verifying it), and sometimes slowly (as if its recreating it).

For the Jetson SDK components it seems to download the stuff from the internet during the component install. So if there is any internet issue or site issue at the time the whole install fails, and you have to go back to the very beginning.

Sure I expect to have to re-image if a component install fails to go back to a known good point, but at the point of the flashing one should no longer have to worry about the cloud. I shouldn’t even have to have an internet connection for the second time through. Like If I’m installing on another Nano for example.

So far I’ve run into three issues on three separate tries.

Flashing → Root cause was some component package that didn’t get installed as I’m using a fresh install of Ubuntu.

CUDA (Jetson SDK Component) → Failed to fetch issues. I got around this by not use the USB connection for the Host->Nano connection, and instead used a network cable. Not sure if that was really the issue though.

Computer Vision (Jetson SDK Component) → Fails on

13:12:58 INFO : OpenCV on Target : E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/p/python-defaults/libpython-dev_2.7.15~rc1-1_arm64.deb 503 Service Unavailable [IP: 91.189.88.150 80]
13:12:58 INFO : OpenCV on Target : E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
13:12:58 INFO : OpenCV on Target : Unhandled error when running sudo apt-get install -y python-dev python-numpy python-py python-pytest
13:12:58 INFO : OpenCV on Target : exit status 1
13:12:58 INFO : OpenCV on Target : [ Package Install Finished with Error ]
13:12:58 ERROR : OpenCV on Target : NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP command scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /tmp/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh rover@10.19.1.59:~; ssh -t -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no rover@10.19.1.59 “~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh && rm -f ~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh” finished with error

So I got that error, and then I clicked “retry failed components” and now its spending lots of time recreating the OS image for some unknown reason.

Preach! But I don’t think Nvidia will do anything. They seem in love with it.

My main issue is that you shouldn’t need a PC to provision the Nano.

I can understand if it needs to install stuff on the host helpful for development, but it should never be necessary to install packages on the Nano, and it is.

Normally I could get away with not needing it, and simply use the development image or the DLI image.

But, I had to modify some kernel stuff to get the I2CS audio board to work so that meant installing the SDK.

I do like that the OS Image is separate from the Jetson Components so that can be installed when needed, but you can’t select Jetson components individually. Like I just need it to retry the OpenCV install, and don’t need to go through the long CUDA install.

I’ll probably just end up taking my boards home where I have a more reliable internet connection without so much filtering and weird glitches that happen on the corporate network. It will probably work just fine, and it won’t complain about “Installing X is taking longer than expected”.

The main enhancement I’d like to see is for it to download things to the host that’s needed on the jetson side. That way once I’ve installed it once I know I can easily re-image, and only have to redownload when the jetpack gets updated.

Sure I can make a drive image myself. But, the SDK manager should alleviate the need for that.

I did manage to get the OpenCV install to work.

To that I did the following commands on the Jetson Nano AFTER the Image OS.

sudo apt-get update
sudo apt-get install -y python-dev python-numpy python-py python-pytest

Then I continued the Component install within the SDK.

Now I’m in the process of imaging another Jetson Nano so I’ll see I run into the same issue with it. On that one I’m simply going to try “sudo apt-get update” after the OS image install.

Thanks for your suggestions S4WRXTTCS,

We’ll discuss the recreate things issue and consider how to improve SDKManager in future releases.
For now if you flashed Nano successfully and failed when installing OpenCV, please solve the network issue first, then de-select “Jetson OS” components at STEP 02, and re-install all “Jetson SDK Components”.
Once the components are downloaded to your host machine, they will not be downloaded again for future installation. SDKManager only checks md5sum values for the files in your download directory, and if the values are good, no re-download is needed during installation.

Internet connection is always required when installing “Jetson SDK Components”. This is because we need do “apt-get update” on target, and may need install a few additional packages on target depending on which version of JetPack is being installed.

I went to try to replicate the issues I was having to better document them, but then the new update was available.

So I updated it, and all the issues I was having went away.

It now clearly downloads most everything first, and doesn’t recreate the image every time I go to flash a new unit. It simply leaves the OS image alone, and reuses it.

Actually I spoke too soon.

The first board worked just fine, and the way it downloaded everything first and did the process made it seem like everything was good.

But, now I’m getting the same issue where on the second board it fails on the OpenCV again where it fails to fetch some archives.

It will fail repeatedly so it’s not just some intermittent networking error. I also have no issues installing the packages manually that it fails on. So I have no idea what’s going on with it. If those are already installed before it starts then it will work fine.

Once it’s failed it won’t simply try again at the point of the OpenCV install despite saying “retry failed”, and instead it will go back to recreating the OS Image.

Here is the log
15:15:25 INFO : OpenCV on Target : Need to get 52.6 MB of archives.
15:15:25 INFO : OpenCV on Target : After this operation, 59.8 MB of additional disk space will be used.
15:15:25 INFO : OpenCV on Target : Get:1 Index of /ubuntu-ports bionic-updates/main arm64 libexpat1 arm64 2.2.5-3ubuntu0.1 [69.1 kB]
15:15:25 INFO : OpenCV on Target : Get:2 Index of /ubuntu-ports bionic-updates/main arm64 libasan4 arm64 7.4.0-1ubuntu1~18.04.1 [335 kB]
15:15:26 INFO : OpenCV on Target : Get:3 Index of /ubuntu-ports bionic-updates/main arm64 libubsan0 arm64 7.4.0-1ubuntu1~18.04.1 [117 kB]
15:15:27 INFO : OpenCV on Target : Get:4 Index of /ubuntu-ports bionic-updates/main arm64 g+±7 arm64 7.4.0-1ubuntu1~18.04.1 [6,149 kB]
15:15:32 INFO : OpenCV on Target : Get:5 Index of /ubuntu-ports bionic-updates/main arm64 gcc-7 arm64 7.4.0-1ubuntu1~18.04.1 [6,004 kB]
15:15:36 INFO : OpenCV on Target : Get:6 Index of /ubuntu-ports bionic-updates/main arm64 libstdc+±7-dev arm64 7.4.0-1ubuntu1~18.04.1 [1,466 kB]
15:15:39 INFO : OpenCV on Target : Get:7 Index of /ubuntu-ports bionic-updates/main arm64 libgcc-7-dev arm64 7.4.0-1ubuntu1~18.04.1 [835 kB]
15:15:41 INFO : OpenCV on Target : Get:8 Index of /ubuntu-ports bionic-updates/main arm64 cpp-7 arm64 7.4.0-1ubuntu1~18.04.1 [5,333 kB]
15:15:45 INFO : OpenCV on Target : Get:9 Index of /ubuntu-ports bionic-updates/main arm64 gcc-7-base arm64 7.4.0-1ubuntu1~18.04.1 [18.9 kB]
15:15:45 INFO : OpenCV on Target : Get:10 Index of /ubuntu-ports bionic/main arm64 libblas3 arm64 3.7.1-4ubuntu1 [83.1 kB]
15:15:46 INFO : OpenCV on Target : Get:11 Index of /ubuntu-ports bionic-updates/main arm64 libexpat1-dev arm64 2.2.5-3ubuntu0.1 [110 kB]
15:15:46 INFO : OpenCV on Target : Get:12 Index of /ubuntu-ports bionic-updates/main arm64 libgfortran4 arm64 7.4.0-1ubuntu1~18.04.1 [249 kB]
15:15:46 INFO : OpenCV on Target : Get:13 Index of /ubuntu-ports bionic/main arm64 liblapack3 arm64 3.7.1-4ubuntu1 [1,371 kB]
15:15:49 INFO : OpenCV on Target : Get:14 Index of /ubuntu-ports bionic-updates/main arm64 libpython2.7-dev arm64 2.7.15-4ubuntu4~18.04 [28.0 MB]
15:16:02 INFO : OpenCV on Target : Err:15 Index of /ubuntu-ports bionic/main arm64 libpython-dev arm64 2.7.15~rc1-1
15:16:02 INFO : OpenCV on Target : 503 Service Unavailable [IP: 91.189.88.150 80]
15:16:03 INFO : OpenCV on Target : Get:16 Index of /ubuntu-ports bionic/main arm64 python-attr all 17.4.0-2 [23.7 kB]
15:16:04 INFO : OpenCV on Target : Get:17 Index of /ubuntu-ports bionic-updates/main arm64 python2.7-dev arm64 2.7.15-4ubuntu4~18.04 [281 kB]
15:16:06 INFO : OpenCV on Target : Get:18 Index of /ubuntu-ports bionic/main arm64 python-dev arm64 2.7.15~rc1-1 [1,252 B]
15:16:06 INFO : OpenCV on Target : Get:19 Index of /ubuntu-ports bionic/main arm64 python-funcsigs all 1.0.2-4 [13.5 kB]
15:16:06 INFO : OpenCV on Target : Get:20 Index of /ubuntu-ports bionic/main arm64 python-numpy arm64 1:1.13.3-2ubuntu1 [1,729 kB]
15:16:07 INFO : OpenCV on Target : Get:21 Index of /ubuntu-ports bionic/main arm64 python-pkg-resources all 39.0.1-2 [128 kB]
15:16:07 INFO : OpenCV on Target : Get:22 Index of /ubuntu-ports bionic/universe arm64 python-pluggy all 0.6.0-1 [12.2 kB]
15:16:07 INFO : OpenCV on Target : Get:23 Index of /ubuntu-ports bionic/universe arm64 python-py all 1.5.2-1 [65.1 kB]
15:16:07 INFO : OpenCV on Target : Get:24 Index of /ubuntu-ports bionic/main arm64 python-six all 1.11.0-2 [11.3 kB]
15:16:08 INFO : OpenCV on Target : Get:25 Index of /ubuntu-ports bionic/universe arm64 python-pytest all 3.3.2-2 [142 kB]
15:16:08 INFO : OpenCV on Target : Fetched 52.6 MB in 44s (1,201 kB/s)
15:16:08 INFO : OpenCV on Target : E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/p/python-defaults/libpython-dev_2.7.15~rc1-1_arm64.deb 503 Service Unavailable [IP: 91.189.88.150 80]
15:16:08 INFO : OpenCV on Target : E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
15:16:08 INFO : OpenCV on Target : Unhandled error when running sudo apt-get install -y python-dev python-numpy python-py python-pytest
15:16:08 INFO : OpenCV on Target : exit status 1
15:16:08 INFO : OpenCV on Target : [ Package PreInstall Finished with Error ]
15:16:08 ERROR : OpenCV on Target : NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP command scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /tmp/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh rover@192.168.55.1:~; ssh -t -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no rover@192.168.55.1 “~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh && rm -f ~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh” finished with error
15:16:08 INFO : OpenCV on Target :
15:16:08 INFO : OpenCV on Target : [ 52.00 KB used. Disk Avail: 163.88 GB ]
15:16:08 INFO : OpenCV on Target : [ NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP Install took 1m17s ]
15:16:08 INFO : OpenCV on Target : command scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /tmp/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh rover@192.168.55.1:~; ssh -t -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no rover@192.168.55.1 “~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh && rm -f ~/tmp_NV_L4T_OPENCV_TARGET_POST_INSTALL_COMP.sh” finished with error
15:16:08 ERROR : OpenCV on Target : command terminated with error

Some additional information.

While doing a manual install of “sudo apt-get install -y python-dev python-numpy python-py python-pytest” I ran into the " http://ports.ubuntu.com/ubuntu-ports/pool/main/p/python-defaults/libpython-dev_2.7.15~rc1-1_arm64.deb 503 Service Unavailable [IP: 91.189.88.150 80]" issue.

That was the first time I encountered that error while doing a manual install of it, and that to me shows there is something intermittent about it.

I don’t know what’s causing the network issue. It could be the USB connection that the board makes with the host, it could be an ISP issue, or a host issue.

In any case it was easy to retry, and it worked fine during a retry. The SDK manager doesn’t seem to try to retry it.