I have a Cuda application that was built with Cuda Toolkit 9.0 and running fine on Jetson TX2 board.
I now have a Jetson Xavier board, flashed with Jetpack 4 that installs Cuda Toolkit 10.0 (only 10.0 is available).
What do I need to do if I want to run the same application on Xavier? Nvidia documentation suggests that as long as I specify the correct target hardware when running nvcc, I should be able to run on future hardwares thanks to JIT compilation. But does this hold for different versions of Cuda toolkit (9 vs 10)?
You will need to compile the application on Xavier again.
For the same JetPack installer, ex JetPack3.3 for both TX1 and TX2, it is possible to compile the application for both GPU architecture.
But it is not available for different CUDA version since the required driver is also different.
Thanks for the response, but I want to understand further.
I don’t know how having the same JetPack installer is related to compiling for both GPU architectures. Isn’t specifying multiple GPU architecture targets something you should do in your nvcc command?
And what do you mean by ‘required driver is also different’? Aren’t the drivers forward-compatible [1]? Meaning if I have a driver that works with Cuda 10, then it should work fine with Cuda 9?
Just a note: Changes in JetPack version also usually changes the supported CUDA version on the Jetson. Xavier is new enough it is only CUDA 10. None of the TX1/TX2 support CUDA 10 yet (I would expect the TX2 to support the same release as Xavier after the next JetPack release…“sometime soon”…sorry, no idea if that is metric or American “soon” units). “CUDA 9” or “CUDA 10” is an example of a driver version.
The GPU architecture is a second issue, but so long as you use the same CUDA version this should just be a recompile for the correct architecture. Desktop, TX2, and Xavier would all require different GPU architecture setting during the compile.
What I ultimately want to do is to compile a Cuda application once, and be able to run it in as many different machines as possible – existing machines, and future machines with new gpu architectures, etc.
I’m getting a bit confused because nvidia’s doc [1] tells me that this is possible if I
statically link cudart library
include PTX files for JIT compilation
let the machines have drivers that are compatible with the statically linked cudart (for example, if I compiled with Cuda 9.0, then ensure that my machines have r384.81 or newer driver version [1]) to take advantage of driver's backward compatibility.
but the moderator is saying something else.
Is my above statement true? If so, what’s the difference between TX2 and Xavier that makes me unable to satisfy one of the 3 conditions?
I’m not much use on the topic since I have not tried to make a single portable application like that. However, I can guarantee you cannot mix CUDA 9 and 10. The CPU architecture also differs between a PC and Xavier, but is the same between Xavier and TX2 (so you are probably in luck there). The GPU architecture can be set to define compatibility with both GPUs, but the other issues imply that at best you need two compiles. The mix of CUDA 9 and 10 implies a complete porting is required from 9 to 10 to operate on Xavier.
More specifically, the TX2 and Xavier have the same CPU architecture…different than a PC, but your goal of making this work on both TX2 and Xavier won’t require any CPU architecture changes.
TX2 and Xavier have different GPU architectures. This is specified with the “-gencode arch=…” setting during compile. TX2 is “-gencode arch=compute_62,code=sm_62”, and Xavier is “-gencode arch=compute_72,code=sm_72”. More than one can be added to a single program, for example:
You cannot get around the requirement that only CUDA 9 is available on a TX2, and only CUDA 10 is available on Xavier. A future JetPack release should make CUDA 10 available to both (and I don’t know when that is other than “soon”). If you can port your CUDA 9 app to CUDA 10, then not too long from now you should be able to get the single compile you want.