Bert new plugins

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

hi, I had trid new bert plugins and demo, i have some question:

  1. I see this descripion: “These plugins only support GPUs with compute capability >= 7.0. For more information about these new BERT-related plugins”.
    Does it means that model will crash in 1080ti and other 6.0?
    actually i had run bert code in 1080ti, it crashed.
  2. So, if i want to use 1080ti, what can i do to speed up bert model?
2

Hi @gumpglh,
Plugin won’t work on 6.0 .
You may need to write your custom plugin or use TF-TRT.
Please refer to the below link
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/samplePlugin

Thanks!

1 Like
3

@AakankshaS thans for your reply. i find that tensorrt5 work on bert without new plugins, can i work on tensorrt5 with 1080ti ？

6

Hi @gumpglh
Trt 5 is old version and might not have additional fixes & improvement as TRT 7. But if in your case it’s working you can use TRT 5 .

Thanks!