🚀 ONNX Runtime-GenAI: Now Dockerized for Effortless Deployment! 🚀
We’re excited to announce that the ONNX Runtime-GenAI plugin has been fully dockerized, simplifying its deployment and usage for developers working on NVIDIA Jetson Orin devices. Thanks to the recent pull request #767 by @dusty, this cutting-edge plugin for ONNX Runtime is now available through prebuilt Docker containers.
💡 Why This is a Game-Changer:
- Plug-and-play setup: Skip the manual compilation; the Docker container has everything pre-configured.
- Optimized for Jetson Orin: Leverages hardware acceleration for seamless integration in edge AI projects.
- Supports GenAI Workloads: Perfect for generative AI tasks requiring high efficiency and low latency.
🎉 A Special Thanks to @shahizat
Your insightful guide and contributions to the NVIDIA Developer Forums have paved the way for using this library effectively on Jetson Orin. Check out their foundational work here: Running Phi 3.5 Vision Using ONNX Runtime-GenAI.
🌐 How to Get Started:
- Pull the container and explore ONNX Runtime-GenAI effortlessly.
- Dive into GitHub PR #767 for more details.
Join the edge AI revolution with ONNX Runtime-GenAI! 🚀