Plugin development without running on GPU

I have developed a few plugins before for TensorRT. All used GPU and developed together with CUDA code.
But for this CTCGreedyDecoder code, not necessarily to run on GPU.
Is it possible to make plugin without CUDA code?

Hi @edit_or
TensorRT gives the stream where the operations are occurring, the input pointers and the output pointers. As long as the computation results are stored in the output pointers(which are device pointers), then where the code is processed doesn’t matter.
But in this case you may face the problem with serialization.
In order to get the input data, you will have to wait on the stream to finish, this will introduce a bottleneck in the GPU.
So its like either you get into this situation or execute it on GPU for better performance.


Thanks for your reply.