A Linux host PC (x86_64) is required for flashing a Jetson. The same software which is used for flash can also install various programs and software on both the host and Jetson. This includes CUDA, development tools, so on. Only a computer with an NVIDIA GPU can be used with CUDA, so if you want those tools on the host (think of developing on the host and deploying on the Jetson), then you need that GPU. If you just flash or install to the Jetson, then you don’t need the NVIDIA GPU on the host.
Typically a Jetson is good at using pre-trained models, but you need something beefier for training. If your desktop has thousands of CUDA cores, and the Jetson has only 256, you’ll see a much better training on the larger GPU (and also typically the desktop GPU may have more video RAM…I’d go 6GB or more, such as a GeForce 1060 or faster…the Jetson uses only a subset of its installed system RAM without dedicated video/GPU RAM). The difference is that you might want a 1000 watt power supply on a fast workstation, while the Jetson is using 7.5 to 30W (depending on what is being done and which Jetson you are using).
The TK1 has its most distinguishing limitation from being 32-bit. CUDA development won’t go beyond version 6.5 on any 32-bit platform. Newer CUDA is constantly evolving on the 64-bit versions. So if you want to keep up with modern software, then you need a TX1 at minimum, or even better yet a TX2. If you need the compute power with less energy drain (e.g., a battery powered drone), then go with the TX2. It is astonishing how well this little electrical supply does so much…if you just need raw power, and if you are ok with drawing 10 to 30 times more power drain, then go with a desktop computer.