TensorRT for Large Language Models

Will this be available for Jetson Orin?

Yes, we’re trying that, you can refer to LLaMa 2 LLMs w/ NVIDIA Jetson and textgeneration-web-ui - Jetson & Embedded Systems / Jetson Projects - NVIDIA Developer Forums

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.