Can someone tell me how to benchmark LLama_v2_7b model on jetson Orin AGX with different quantization methods?

AI & Data Science NVIDIA AI Workbench
,
1

Can someone tell me how to benchmark LLama_v2_7b model on jetson Orin AGX with different quantization methods and also how to get perplexity score

2

Hi, thanks for reaching out. This is the forum for NVIDIA AI Workbench. You could try to find the right forum here.