# \#inference-fil-spark

**URL:** https://forums.developer.nvidia.com/tag/inference-fil-spark/228.md

[Latest](https://forums.developer.nvidia.com/latest.md) · [Categories](https://forums.developer.nvidia.com/categories.md) · [Tags](https://forums.developer.nvidia.com/tags.md)

---

## [DeepSeek-V4-Flash on 2x DGX Spark throughput collapses at large prompt+concurrency. Would a 4x stack + Vision-Exp fix it?](https://forums.developer.nvidia.com/t/deepseek-v4-flash-on-2x-dgx-spark-throughput-collapses-at-large-prompt-concurrency-would-a-4x-stack-vision-exp-fix-it/384069)

<div class="topic-metadata">

**Author:** [@ammarabbaxi13](https://forums.developer.nvidia.com/u/ammarabbaxi13)\
**Replies:** 1\
**Last updated:** [November 1, 2026, 7:06pm UTC](https://forums.developer.nvidia.com/t/deepseek-v4-flash-on-2x-dgx-spark-throughput-collapses-at-large-prompt-concurrency-would-a-4x-stack-vision-exp-fix-it/384069 "2026-11-01T19:06:17Z")

</div>

Hi all, I’m running DeepSeek-V4-Flash-Vision-exp across 2x DGX Spark (TP=2) using the MIAI Labs Recipe. Our target is ≥500K context with 8 concurrent users, but I’m noticing token generation speed and TTFT dropping sha…

---

## [SparkRun Auto Model Registration with LiteLLM & Local Claude Code Setup](https://forums.developer.nvidia.com/t/sparkrun-auto-model-registration-with-litellm-local-claude-code-setup/364304)

<div class="topic-metadata">

**Author:** [@griffith.mark](https://forums.developer.nvidia.com/u/griffith.mark)\
**Replies:** 4\
**Last updated:** [May 21, 2026, 10:26pm UTC](https://forums.developer.nvidia.com/t/sparkrun-auto-model-registration-with-litellm-local-claude-code-setup/364304 "2026-05-21T22:26:32Z")

</div>

A big thank you to @Eugr, @DBSIDBSI, and @Raphael.Amorin — what you’re building with SparkRun and SparkArena is genuinely changing what’s possible on the DGX Spark. For anyone who hasn’t come across it yet: SparkRun is …

---

## [\[GB10\] vLLM + DeepSeek-R1-32B Stable Setup on Blackwell — Full Protocol After 4 Days of Failures](https://forums.developer.nvidia.com/t/gb10-vllm-deepseek-r1-32b-stable-setup-on-blackwell-full-protocol-after-4-days-of-failures/363533)

<div class="topic-metadata">

**Author:** [@turgaysavaci](https://forums.developer.nvidia.com/u/turgaysavaci)\
**Replies:** 4\
**Last updated:** [March 17, 2026, 7:11pm UTC](https://forums.developer.nvidia.com/t/gb10-vllm-deepseek-r1-32b-stable-setup-on-blackwell-full-protocol-after-4-days-of-failures/363533 "2026-03-17T19:11:59Z")

</div>

Hey everyone, I spent 4 days getting vLLM stable on a Blackwell GB10 (120GB VRAM) running DeepSeek-R1-Distill-Qwen-32B. Most of what broke is not documented anywhere — so I’m sharing the full failure log here and in the…

---

## [Web Based Inference GUI Server for Easy Model Deployment on the Spark](https://forums.developer.nvidia.com/t/web-based-inference-gui-server-for-easy-model-deployment-on-the-spark/360889)

<div class="topic-metadata">

**Author:** [@matthew.rogers](https://forums.developer.nvidia.com/u/matthew.rogers)\
**Replies:** 2\
**Last updated:** [February 17, 2026, 6:45pm UTC](https://forums.developer.nvidia.com/t/web-based-inference-gui-server-for-easy-model-deployment-on-the-spark/360889 "2026-02-17T18:45:38Z")

</div>

I made this little wrapper, run via tmux or nohup. It makes adding models and model management trivial so you can focus on developing and using the spark.

---

## [TCP RTSP stream processing](https://forums.developer.nvidia.com/t/tcp-rtsp-stream-processing/182195)

<div class="topic-metadata">

**Author:** [@harsh.vijay](https://forums.developer.nvidia.com/u/harsh.vijay)\
**Replies:** 8\
**Last updated:** [June 30, 2021, 10:12am UTC](https://forums.developer.nvidia.com/t/tcp-rtsp-stream-processing/182195 "2021-06-30T10:12:52Z")

</div>

• Hardware Platform (Jetson / GPU) GPU rtx 1050 ti • DeepStream Version 5.1 • TensorRT Version 7.2.2 • NVIDIA GPU Driver Version Driver Version: 460.80 • Issue : I am running python sample application of deepst…

---

## [Can't perform inference using Python API of TensorRT](https://forums.developer.nvidia.com/t/cant-perform-inference-using-python-api-of-tensorrt/159346)

<div class="topic-metadata">

**Author:** [@mmaaz60](https://forums.developer.nvidia.com/u/mmaaz60)\
**Replies:** 1\
**Last updated:** [November 20, 2020, 8:52am UTC](https://forums.developer.nvidia.com/t/cant-perform-inference-using-python-api-of-tensorrt/159346 "2020-11-20T08:52:13Z")

</div>

Description While following the instructions at TensorRT docs to run inference on using Python API, I am getting memory allocation error. pycuda.\_driver.MemoryError: cuMemHostAlloc failed: out of memory Environment Ten…
