I have a TK1 running llama.cpp if anyone has a dgx spark they would like to give to me i will be forever grateful.
to get llama.cpp running:
Have a TK1 with Grinch kernel (so i can use wifi)
installed debian11 32-bit armv7 to TK1 (onto sata drive)
boot debian
install llama.cpp
wget llm model
run model
Issues i had:
ubuntu 14.04 could not compile llama.cpp
ubuntu 18 is possible to install but is quite bloated (2gb ram limit + 16gb emmc) but again could not install llama.cpp due to cmake versions and a few other outdated things.
debian11 install was just a few hundred MB
debian 11 uses just 52MB RAM giving almost all 1.7GB ram free for llama
other issues:
My TK1 was brick due to HDMI hardware failure & wifi not configured so no direct ssh session possible
connected db9 serial debug with minicom
fresh install ubuntu 14.04 with cuda (grinch for pcie wifi)
connected old sata drive to TK1 sata port
installed debian11 to sata ssd
enabled boot option grinch or debian
Wish i had cuda gpu, but only functioning on grinch 14.04
Was so much fun to achieve this as a personal challenge, thank you.
