Hello folks,
is it possible to run the Evo 2-40B NIM on a virtual machine with an A100 GPU? I’ve given it a try but faced the error message {"error":"The model requires FP8 support"} when trying to run the Shell client example(./nim_client.sh). My deployment method is a Docker container.
Is there any way to bypass this limitation and run it on the A100, even with lower performance?