I’m developing a Python-based real-time F1 25 telemetry AI race engineer that uses an LLM to generate short voice tips during gameplay. I’m interested in using Chat with RTX instead of a cloud model for privacy and lower latency.
so i have a bunch of question about the app and its limitations.
-
LLM / API access
Does Chat with RTX expose a local API, socket, or REST endpoint that third-party apps (like Python scripts) can call programmatically?
If not, is there any SDK or library (C++/Python/.NET) for TensorRT-LLM inference that matches the Chat with RTX model interface?
Can Chat with RTX run continuously in the background and handle frequent short queries (e.g., one every 5–10 seconds)?
Is the RAG / document retrieval pipeline optional? (I just want to feed it structured telemetry data, not documents.)
Which model architecture is currently used (e.g., Mistral 7B, Llama 3, or another variant), and can it be swapped or fine-tuned?
- Real-time performance / GPU load
When running alongside a demanding title like F1 25, what’s the expected GPU VRAM and compute overhead for Chat with RTX?
Does it automatically throttle or yield resources when the GPU is heavily loaded by another app (like a game)?
Can I dedicate specific CUDA cores or memory limits to Chat with RTX?
- Output handling / TTS
Is there a way for Chat with RTX to stream its text output to a file or stdout for external TTS processing?
Does NVIDIA plan to add speech synthesis (TTS) directly into Chat with RTX, or would that need a separate pipeline?
- General deployment
Are there any command-line arguments or config files for custom startup (e.g., specifying ports, memory, model paths)?
Can it be installed headlessly (no UI) for embedded or automation use cases?
Is there licensing or EULA restriction against embedding Chat with RTX responses into third-party software?
I have an rtx 3060ti and ryzen 5 5600g