I built a local AI Model Screener for NVIDIA NIM and OpenAI-compatible providers

AI & Data Science NVIDIA NIM Models
,
1

Hi everyone,

I made a small local Windows tool for myself and decided to share it in case it is useful for other NVIDIA NIM users.

The project is called AI Model Screener / NV Screener. It is a local desktop app for browsing available AI models from API providers, selecting models, and quickly comparing their responses with basic timing and token metrics.

GitHub:

What it does

  • Loads model catalogs from NVIDIA NIM and other OpenAI-compatible providers

  • Lets you add your own providers through the settings

  • Allows multi-provider model selection

  • Runs a test prompt across selected models

  • Shows response time, token count, tokens/sec, errors, and basic ranking by speed/metrics

  • Supports saving model sets and test runs locally

  • Stores runtime data locally in AppData, outside the repository

Why I made it

I wanted a simple way to check which models are currently available through API and quickly compare them without manually testing every model one by one.

For NVIDIA NIM, I added a more conservative scan profile so the app can check a large batch of models without immediately hitting the 40 requests/minute limit. Scanning 40+ models can still take some time, but it is much more convenient than checking them manually.

Important notes

  • This is a local utility, not a hosted service

  • Windows-focused for now

  • API keys are user-provided and stored locally

  • The ranking is based on speed and basic metrics, not answer quality

  • MIT licensed

How to run from source

pip install -r requirements.txt
python app.py

Or build the Windows executable:

pyinstaller --clean "NV Screener.spec"

I originally built this for my own workflow, but I think it may be useful for people who frequently test NVIDIA NIM models or work with multiple OpenAI-compatible API providers.

Feedback, issues, and suggestions are welcome.

2

About

Find, benchmark and install in CLI 170+ FREE coding LLM models across 15+ providers in real time

3

I have log exports after testing, saved templates/presets for the most-tested models, packaging into a convenient .exe installer, and protection against running multiple copies through a Windows mutex.

For NVIDIA NIM specifically, I also added a more careful check so it does not immediately hit the 40 requests per minute limit.

Other providers do not have this kind of limitation in my setup.

The repository you sent me is already popular, although I did not know about it before. I built my own tool without basing it on other people’s work. I simply thought that something like this could be very useful for me personally, so I made it. After finishing it, I realized that I should share it with you as well.