How to load Custom Plugin dynamic library into triton on windows

AI & Data Science Deep Learning (Training & Inference) Triton Inference Server - archived

wen.wl.li January 26, 2021, 5:40am 1

On Linux, if we want to make the custom layers available to Triton, the TensorRT custom layer implementations must be compiled into one or more shared libraries which must then be loaded into Triton using LD_PRELOAD. for example,
$ LD_PRELOAD=libtrtcustom.so tritonserver --model-repository=/tmp/models …
But on windows, how to do it easily?

Topic		Replies	Views
How to add a custom plugin to tensorrt library? TensorRT	1	1622	December 24, 2019
Triton inference server dynamic load TensorRT inference-server-triton	0	126	July 18, 2024
Import pytorch model in TensorRT and add custom plugin layer TensorRT	7	5307	May 13, 2019
Loading of TensorRT custom plugin shared library TensorRT	3	1601	December 20, 2022
how to create custom layer in TensorRt by python GPU-Accelerated Libraries	0	583	February 5, 2018
How to compile Triton Inference Server on ppc64le Triton Inference Server - archived	1	613	October 30, 2020
How to use purpose-built models like Peoplenet for custom-pipeline TAO Toolkit	4	780	October 12, 2021
TensorRT customized plugin TensorRT	5	2399	May 17, 2021
Building Client Libraries standalone Triton Inference Server - archived	1	558	April 22, 2019
How to integrate TensorRT graph in deepstream pipline? DeepStream SDK tensorrt , jetson-inference , gstreamer	5	522	October 12, 2021

How to load Custom Plugin dynamic library into triton on windows

Related topics