We built traceAI, an open-source tool for tracing LLM calls in production

[param.s]

Hey everyone 👋

We have been building traceAI, an open-source observability tool
for LLM applications in production.

It traces every LLM call capturing inputs, outputs, latency, costs,
and errors with minimal setup. Useful for teams running inference
on NVIDIA hardware who want full visibility into what their models
are actually doing in prod.

Repo: GitHub - future-agi/traceAI: Open Source AI Tracing Framework built on Opentelemetry for AI Applications and Frameworks · GitHub

Would love feedback from folks running LLMs at scale. What does
your current monitoring setup look like?