TLDR:
SOS-Toolkit - a developer-focused tool for building, packaging, and distributing local digital intelligence systems in a way that is accessible to non-technical users. It solves system dependency challenges by decoupling end-user environments from an application. By implementing an Intelligent Makefile, it allows developers to focus on functionality while SOS-Toolkit handles the underlying technical layers.
Currently only Jetson AGX Orin 64GB is supported.
What is needed are people to test and make suggestions about additional tools, functions, and features.
Background:
My favorite Jetson project was building a cluster of Xavier NX modules to run distributed training of a CycleGAN for a real-time vision task. It was a fun project, but not very useful. While working on it, I started thinking about different ways to make intelligent systems. The rapid progress of LLMs has been the defining metric for digital intelligence over the last few years, and their current focus is to run as large a model as possible. While working with my cluster, I found that instead of training one large model to do everything, training multiple smaller models on specific tasks worked better. The small models operated not as independent endpoints, but as cooperative building blocks for complex systems.
SOS-Toolkit is designed to run multiple specialized models concurrently while rapidly switching this working set. Current tools struggle with this due to complex dependencies, incompatible versions, and conflicting environment configuration. It’s hard enough to get a single Ollama server to work consistently; now try switching between five different versions. SOS-Toolkit absorbs the complexities into a structured format using intelligent makefiles. This approach enables it to build dynamic runtime processes that adapt to local environments. Those processes provide the foundation to build powerful new systems, enabling users to interact with advanced functionality without being burdened by technical details or extensive knowledge of system internals.
Anyone who has attempted to port a specific machine learning application to work on Jetson knows how difficult the problems can be. From initial experiments, SOS-Toolkit can solve them, but its full potential won’t be realized without further testing and refinement. As SOS-Toolkit continues to evolve, we welcome anyone with insights into its applications or ideas for enhancement to contribute in any way they wish – currently we need testing and recommendations for additional tools, functions or features.
Check out what is built so far or look forward to out next major version when we start moving beyond testing features and begin implementing unique systems.