Critical attack: LiteLLM compromised. Pin1.82.6 NOW

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10
1

LiteLLM has been compromised, both most recent versions .7 and .8 sourced from PyPi are malicious.

Action: Pin .6 right now and follow this issue.

I know a good number of us are using this.

2

Just updating, thanks for the heads-up.

3

1774371835438912731814097418391
17743718354389127318140974183912047×1475 369 KB

It is REALLY BAD. This is everything you need to change immediately if you are affected.

4

Using Docker, it was set to littlelm==1.82.3 , thanks

5

Evil sh1t. Luckily I’m sticking to much older versions.

LiteLLM maintainer here, this is still an evolving situation, but here’s what we know so far:

  1. Looks like this originated from the trivvy used in our ci/cd - https://github.com/search?q=repo%3ABerriAI%2Flitellm%20trivy… Redirecting...

  2. If you’re on the proxy docker, you were not impacted. We pin our versions in the requirements.txt

  3. The package is in quarantine on pypi - this blocks all downloads.

We are investigating the issue, and seeing how we can harden things. I’m sorry for this.

  • Krrish
6

Is litellm included in anything in widespread use that we should be taking a look at? I know I haven’t directly installed it myself, but not sure where this package might be included, if anywhere, that was silently installed alongside other packages or software. As long as it’s something that generally needed to be installed directly (like Ollama/vLLM or the such), then I’m less concerned for myself at least.

7

LiteLLM is not included in vLLM. There is only a README explaining on how to deploy with vLLM.

Open WebUI had it bundled in the past, but they removed it with v0.2.0.

But I’ve seen other tools using LiteLLM as a backend. So be careful. Have a look in the requirements.txt / pyproject.tomls of other Python AI tools.

8

Nice. If you follow the thread on ycombinator.com you will find some tools that use LiteLLM.

DSPy, CrewAI, nanobot…

some already reacted and pinned the versions below the affected.

9

Docker images appear to be fine if you’re using them, the version of LiteLLM inside of them is 1.82.3, so it’s unaffected.

In case you’re newer to running things like docker, I would highly recommend using immutable digest (pinned/immutable image) over a floating tag (mutable tag). If you don’t know what the difference is, I’ll explain the basic concept below:

Floating Tag Example:
vllm/vllm-openai:v0.18.0

Immutable Digest Example:
vllm/vllm-openai:v0.18.0@sha256:c32358ebfc115d56ade2acfdbcd00df5b115417dbd6006547c88f07e2b39de06

The difference between these two is very important.

For the floating tag (to put it simply), you’re not specifying an exact image, you’re just really specifying a label. This means that today the floating tag could be a safe image, but tomorrow it could be a compromised image. If a malicious actor compromises the image maintainer, CI pipeline, or registry, they could republish the same tag pointing to malicious code. Meaning the next time you pull it, you’ll be pulling their new malicious image without realizing it.

For the immutable digest, notice there is a SHA256 hash. With an immutable digest, that SHA256 hash uniquely identifies the image contents. If the code changes in any way, the hash changes. By using an immutable digest, you ensure that you always pull the exact same image, regardless of what happens to tags later.

It might be a pain in the butt to maintain immutable digest tags, but it does lower the risk of compromise to a degree. I would personally consider using immutable digests a best practice for docker.