I have downloaded Mistral-7B-Instruct-v0.2 model from huggingface and want to convert that model to Triton Inference Server supported framework and Run the model using Triton Inference server. I need support doc which provides steps to deploy and also infrastructure details.
Related topics
Topic | Replies | Views | Activity | |
---|---|---|---|---|
Implementation of Triton Inference server on EC2 ubuntu instance | 2 | 668 | December 7, 2023 | |
New Tutorial : Deploying Models from TensorFlow Model Zoo Using NVIDIA DeepStream and NVIDIA Triton Inference Server | 2 | 470 | March 6, 2023 | |
Deploy LPRnet sequence Classifier on Triton Inference Server | 0 | 758 | July 21, 2021 | |
Mistral AI Models | 1 | 320 | June 25, 2024 | |
MistralAI models, Mistral-7B, Mistral-7B-Instruct, Mixtral-8x7B, Mixtral-8x7B-Instruct | 0 | 208 | June 17, 2024 | |
Support for PyTorch | 1 | 521 | April 15, 2021 | |
Triton infererence server example 'simple_grpc_infer_client.py' | 11 | 4990 | March 23, 2022 | |
Inference on video/audio streams in Triton | 1 | 1803 | September 30, 2021 | |
Using TLT models with Triton Inference Server | 8 | 1435 | October 12, 2021 | |
Support for Triton Inference Server on Jetson NX | 2 | 885 | November 2, 2022 |