Kernels need cross-compile due to the mixed 32-bit compatibility mode and 64-bit native mode. Everything else is far simpler to just run on the JTX1.
Do note that if you cross-compile in user space you need not just the cross-compiler tool chain, you also need the sysroot (the linker and everything required for actual linking…this includes the libraries being linked to); Linaro has a sysroot available, but you can also use a loopback mounted clone of your Jetson for part of that environment.
Imagine that if your project involves OpenGL, then your sysroot will need you to have you manually add everything supporting OpenGL (and perhaps X11)…a clone simplifies this because absolutely everything for sysroot libraries is put in place on the Jetson prior to the clone (and a loopback mounted clone can be updated with rsync for incremental changes). I still consider cross-compile as slower and more complicated even with a clone (you can’t just drop in a clone and have it work without setup). Building on the Jetson itself implies you just add packages via apt-get and it resolves everything for you…plus if you use a clone you need to do the apt-get process anyway…and this is why the clone is simpler than manual setup of sysroot, yet not as simple as native Jetson compile.
The one case where you have a Jetson available and need user space build via cross-compile has a significant advantage is if you build large amounts of software…the Jetson is actually fairly fast at compiles, but the number of cores, speed of cores, amount of RAM, hard disk capacity, so on, still makes a desktop the king for large builds with many threads.