Running Phi-3.5-vision using ONNX Runtime GenAI on the Nvidia Jetson Orin dev kits

Robotics & Edge Computing Jetson Systems Jetson Projects
, ,
1

Hello fellow Nvidia Jetson enthusiasts,

Below is a video demonstration of ruining the inference of the Phi-3.5-vision-instruct-onnx(microsoft/Phi-3.5-vision-instruct-onnx · Hugging Face) model using ONNX Runtime GenAI(GitHub - microsoft/onnxruntime-genai: Generative AI extensions for onnxruntime) using GPU.

2

Dockerized:
onnxruntime-genai by johnnynunez · Pull Request #767 · dusty-nv/jetson-containers · GitHub @dusty_nv