How can we bring VLM of choice?

Accelerated Computing Intelligent Video Analytics Visual AI Agent
2

If you have another openAI compatible model you can run the following:

docker run … -e OPENAI_API_KEY=<OPENAI_API_KEY>
-e VIA_VLM_OPENAI_MODEL_DEPLOYMENT_NAME=gpt-4o
-e VLM_MODEL_TO_USE=openai-compat

You can also load a custom model:

  1. Mount the directory containing the inference.py file and the optional model files in the
    container.
  2. Set the MODEL_PATH env variable to the mount path. MODEL_PATH should point to the
    directory containing the inference.py file in the container.
  3. Set VLM_MODEL_TO_USE=custom

ls <MODEL_DIR_ON_HOST>
inference.py
docker run … -v <MODEL_DIR_ON_HOST>:<MODEL_DIR_IN_CONTAINER>
-e MODEL_PATH=<MODEL_DIR_IN_CONTAINER> -e VLM_MODEL_TO_USE=custom …

You can refer to pages 21 and 23 of the VIA DP User Guide for more details.