@mausam.jain I’ve not had luck in the past with getting vLLM to compile on Jetson, and would recommend trying other LLM inferencing libraries that we have working on Jetson and pre-built container images for, like MLC, AWQ, llama.cpp, and exllama (all of which support quantization)
It would seem unlikely at this point of another library being faster than MLC, as it is already close to peak theoretical performance on Orin, and I support it and AWQ as preferred backends in the NanoLLM library.