Jetson nano 8G Run VLM Benchmark

[milk2_yu]

Hi ,

The picture show from AI Lab Benchmarks. Where those VLM model from? Where has the test sample code?

I have try ollama llava 1.6-7B model, my token/sec have more then 3 on jetson nano 8G super.

But in pictrue the llava 1.6-7B only has 0.57 token/sec, Why?

[AastaLLL]

Hi,

The score is generated with MLC.
You can find the script in the below link:

packages/llm/mlc/benchmark.sh

Thanks.

[milk2_yu]

It like for LLM Benchmark.
But i want for VLM.
Thanks

[AastaLLL]

Hi,

The VLM benchmark is generated with the huggingface script with 4-bit quantization.
Thanks.