Is there any available link for downloading a minimal command-line-only sample file system for the Jetson TX1?
Are you interested in R23.2 or R24.1? See:
Both have listed “sample root filesystem”. In R24.1 it gets a bit more complicated, as you can pick the 32-bit user space version or 64-bit user space (R23.2 is better tested, but is only 32-bit in user space).
After you install either 23.2 or 24.1, you can change Ubuntu’s boot settings to load into text only mode.