I’d like to initialize the state of the model with custom tensor on triton inference server.
I don’t know the structure of the datafile for the state initialization also My model is tensorrt.
Please help me.
TensorRT Version: 8.6.1
GPU Type: A100 40G
Nvidia Driver Version:
CUDA Version: 12.2
CUDNN Version: 8.9
Operating System + Version: Debian 11
Python Version (if applicable): 3.9
PyTorch Version (if applicable): 2.2.0
Please refer to the following section of the document, which may help you.
This file has been truncated.
# Copyright 2018-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
Please reach out to
Issues · triton-inference-server/server · GitHub if you need any further help.
Thanks for your reply. I just reported issue on github