Originally published at: Full-Stack Optimizations for Agentic Inference with NVIDIA Dynamo | NVIDIA Technical Blog
Coding agents are starting to write production code at scale. Stripe’s agents generate 1,300+ PRs per week. Ramp attributes 30% of merged PRs to agents. Spotify reports 650+ agent-generated PRs per month. Tools like Claude Code and Codex make hundreds of API calls per coding session, each carrying the full conversation history. Behind every one…
Interesting work.
I have one quick question:
On the dynamo side, when routing requests, will they consider the different configurations of the inference engine instances? For example, the same degree of parallelism but different TP sizes would yield different performance? Would this situation be considered?