Jetson nano 8G Run VLM Benchmark

AastaLLL · March 3, 2025, 5:05am

Hi,

The score is generated with MLC.
You can find the script in the below link:

packages/llm/mlc/benchmark.sh

master

#!/usr/bin/env bash
#
# Llama benchmark with MLC. This script should be invoked from the host and will run 
# the MLC container with the commands to download, quantize, and benchmark the models.
# It will add its collected performance data to jetson-containers/data/benchmarks/mlc.csv 
#
# Set the HUGGINGFACE_TOKEN environment variable to your HuggingFace account token 
# that has been granted access to the Meta-Llama models.  You can run it like this:
#
#    HUGGINGFACE_TOKEN=hf_abc123 ./benchmark.sh meta-llama/Llama-2-7b-hf
#
# If a model is not specified, then the default set of models will be benchmarked.
# See the environment variables below and their defaults for model settings to change.
#
# These are the possible quantization methods that can be set like QUANTIZATION=q4f16_ft
#
#  (MLC 0.1.0) q4f16_0,q4f16_1,q4f16_2,q4f16_ft,q4f16_ft_group,q4f32_0,q4f32_1,q8f16_ft,q8f16_ft_group,q8f16_1
#  (MLC 0.1.1) q4f16_0,q4f16_1,q4f32_1,q4f16_2,q4f16_autoawq,q4f16_ft,e5m2_e5m2_f16
#
set -ex

This file has been truncated. show original

Thanks.

Topic		Replies	Views
Jetson Orin Nano Super performance test issue Jetson Orin Nano jetson	3	47	May 12, 2025
VLM Refresh Rate Jetson AGX Orin camera , generative_ai	6	181	December 19, 2024
Problem: slow LLM inference speed on Jetson AGX Orin 64GB Jetson AGX Orin jetson-inference , generative_ai	2	124	April 8, 2025
MiniCPM-Llama3-V-2_5 live on Jetson Orin Jetson AGX Orin generative_ai	11	613	August 9, 2024
Can't run NanoVLM on Jetson Orin NX 16GB Jetson Orin NX generative_ai	4	259	May 16, 2024
Jetson Orin Nano Super Dev Kit Performance Jetson Orin Nano cudnn , gemma-2-9b-it , llama-31-8b-instruct , llama	6	552	January 28, 2025
Failed to MLC-compile mlc-ai/Llama-3.1-8B-Instruct-fp8-MLC on Jetson AGX orin Jetson AGX Orin generative_ai , llama-31-8b-instruct , llama	5	101	January 13, 2025
Available with Small Language Model on tutorial Jetson Orin Nano generative_ai	3	667	May 3, 2024
Jetpack6.2+TensorRT OOM issue Jetson Orin Nano generative_ai , llama	7	143	February 21, 2025
Mlc_llm(0.19.0) does not support qwen3 model Jetson Orin Nano generative_ai	5	29	May 14, 2025

Jetson nano 8G Run VLM Benchmark

Related topics