I won’t be able to answer anything except a small part of your question, and I’ve not used ROS. This might get you started though.
If you use the cross compiler recommended by NVIDIA for kernel build, then you are probably doing the right thing, though I don’t know if any of the other software might have some release version dependency.
When you cross compile a kernel the kernel is “bare metal”. There are no libraries to link with, and there is no special support for the environment which a regular executable program might expect. Once you build a user space application you must then consider having all of the libraries in place. The libraries themselves will not do anything for you unless you have a linker, and this would be a cross linker since it runs on a desktop PC architecture, but links a foreign (arm64/aarch64) architecture.
The tools used for linking will be considered the “runtime” package when installed to the host PC. The supporting environment of libraries would be called the “sysroot”. These would be the additions above and beyond the cross compiler. I don’t know which release you will be using, but most of this is indirectly from linaro.org, under one of these releases (stick to the “apt
” mechanism for install if you can, I am just illustrating):
https://releases.linaro.org/components/toolchain/binaries/
If for example you were to look at release 7.3, running on Linux, with 64-bit ARM architecture (“aarch64”), then you would end up here:
https://releases.linaro.org/components/toolchain/binaries/7.3-2018.05/aarch64-linux-gnu/
…notices that “aarch64” in the name is the architecture, “linux” is the supporting environment, and “gnu” is the surrounding library environment.
Within this the “runtime” is basically the cross linker. The “sysroot” is basically the libraries the linker would use. The “gcc” part is a set of cross tools, e.g., cross compile, or assembler. A kernel only needs the “gcc” part, you will need all three in user space.
I recommend against using the “sysroot” from third parties. There is nothing wrong with that approach, but traditionally that sysroot is a very minimal set of libraries. Every time you needed another library you would end up building it yourself from scratch. If you have a running Jetson, then you already have the most complete sysroot possible.
A clone of the Jetson (loopback mounted), or recursive copy of various directories from the Jetson, will guarantee not only having that content, but also that you are linking against that specific release. The trick is that you would want to put the devel package content on the Jetson itself (you would be interested in header files used for compiling, and not just actual libraries…these are the devel packages).
If you look at any of the Linux PCs (or Jetsons), and examine directories “/lib
” and “/usr/lib
”, then you’ll end up finding subdirectories named after an architecture. Some content may not have an architecture listed, and this is likely all “native architecture” content. If cross architecture content is installed, then you will also likely find subdirectories named after that non-native content, e.g., “/usr/lib/aarch64-linux-gnu
” (notice how this matches the cross tool naming from linaro.org?) or “/lib/aarch64-linux-gnu
”. This architecture specific content is what the sysroot would provide, only it is complete and already an exact match for the target system. Recursive copy or symbolic links to a loopback mounted clone is guaranteed perfect.
FYI, if you were to copy files into your host at “/lib/aarch64-linux-gnu/
”, then you’d be stuck with that as your dev version. If these were instead symbolic links pointing to where a loopback clone mount exists, then simply remounting a different clone would instantly update your development environment to that release. Remounting a different clone would revert to the other environment. “Clones are your friend”. Clones can also be used to restore if you lose your Jetson.
You can also update your development environment to use paths such as “/usr/local/lib/aarch64-linux-gnu
”, and thus not mess with the system path at “/usr/lib
”. The “/usr/local
” content is intended for things not generally managed via the standard package manager, but would more or less mirror (and supplement) what you see in “/usr
”.
You have yet another option someone may be able to help with. You could use QEMU to emulate the full environment. You’d still use something like a clone, but you’d be using not only the libraries and headers of the clone, you’d also be using the binary executables. For example, the linker would be native, and not cross/foreign. QEMU could pretend to be an aarch64 operating system based on your clone. Sorry, I can’t help with the details of that, nor with ROS or its requirements, but this will get you started and someone else can give details for ROS and/or QEMU.