LM Studio on Jetson Thor?

Robotics & Edge Computing Jetson & Embedded Systems Jetson Thor
1

I tried to load a language model on Jetson using LM Studio. It seems that only a CPU runtime is currently available here. Although a CUDA 13 runtime is also installed and can be selected, the loading process then aborts. I tried to ask ChatGPT for a solution, but so far without success. Is there a way to use LM Studio on Jetson Thor with CUDA?

3

Hi,

We don’t have an LM Studio-related container or package currently.
Can vLLM or Ollama meet your requirements?

vLLM: GPU-optimized AI, Machine Learning, & HPC Software | NVIDIA NGC
Ollama: Package ollama · GitHub

Thanks.