Is there any guide to building NIM-style images? I’d like to run one of the Microsoft phi models with workbench and there is no NIM for that currently.
Hey @joe173, unfortunately there’s no option for end users to create their own NIMs at the moment. For an alternative you could try using TensorRT-LLM directly: TensorRT-LLM/examples/phi at main · NVIDIA/TensorRT-LLM · GitHub
@neal.vaidya can you tell if something is planned to support custom models? we work on bioML model deployments and pipeline building but (1) currently the model support on Nvidia side is limited, and (2) bringing our own models via NIMs to production would be of interest as well. BR Fabian