Questions about Jetson TX1 tools and setup

We are working on a product prototype, where we need to deploy a DL model on a JetsonTX1; the model has been trained on a PC host using TensorFlow. We are a very small team so far, and do not have an experienced embedded software developer on board yet, hence we have some basic questions (we are doing only userland development, no Kernel modules):

  1. What tools are recommended for cross-compilation on the PC host (Ubuntu 14.04) ?

  2. So far, we are manually converting weights from the TensorFlow model, and recreate it in CuDNN C++ on the target. This approach, obviously, does not scale. Have anybody managed to port TensorFlow on the Jetson target ? It is running Ubuntu 16.10.

Thanks !

People have reported success in building TensorFlow on the TX1 itself:

v0.11
Just looking for the wheel file?

Build it yourself?

Discussion from the TensorFlow Github issues:
https://github.com/tensorflow/tensorflow/issues/851

v0.12 Newer, a little more experimental - CUDA compiler on the JTX1 currently has a bug which requires a workaround
[url]https://devtalk.nvidia.com/default/topic/987306/internal-error-in-nvidia-driver-code-while-building-r0-12-or-master-tensorflow-on-tegra-tx1/?offset=9#5067776[/url]

It recommended to use TensorRT for fast inferencing on tx1. We will optimize the whole network into trtmodel based on GPU architecture.
https://devblogs.nvidia.com/parallelforall/production-deep-learning-nvidia-gpu-inference-engine/

The simplest way to launch tensorRT is using caffemodel as input.

If you want to run tensorflow on tx1, please refer to this issue:
https://github.com/tensorflow/tensorflow/issues/851

Thanks a lot for your replies about porting the model onto tx1.

Nobody replied about the cross-compilation. Does it mean that most of you build on the tx1 ? I guess I could keep sources on tx1, and build there, while running the rest of IDE (editor, source control) on the host PC, by having the sources mounted on the PC.

Kernels need cross-compile due to the mixed 32-bit compatibility mode and 64-bit native mode. Everything else is far simpler to just run on the JTX1.

Do note that if you cross-compile in user space you need not just the cross-compiler tool chain, you also need the sysroot (the linker and everything required for actual linking…this includes the libraries being linked to); Linaro has a sysroot available, but you can also use a loopback mounted clone of your Jetson for part of that environment.

Imagine that if your project involves OpenGL, then your sysroot will need you to have you manually add everything supporting OpenGL (and perhaps X11)…a clone simplifies this because absolutely everything for sysroot libraries is put in place on the Jetson prior to the clone (and a loopback mounted clone can be updated with rsync for incremental changes). I still consider cross-compile as slower and more complicated even with a clone (you can’t just drop in a clone and have it work without setup). Building on the Jetson itself implies you just add packages via apt-get and it resolves everything for you…plus if you use a clone you need to do the apt-get process anyway…and this is why the clone is simpler than manual setup of sysroot, yet not as simple as native Jetson compile.

The one case where you have a Jetson available and need user space build via cross-compile has a significant advantage is if you build large amounts of software…the Jetson is actually fairly fast at compiles, but the number of cores, speed of cores, amount of RAM, hard disk capacity, so on, still makes a desktop the king for large builds with many threads.