I am wondering whether there is any way to modify TensorRT so that we can wait between the layers. To clearly state (just an example), I want to insert a wait time after the 20th layer in an inference of DNN having 30 layers. I’ve checked sampleInference.cpp, however this does not seem visible.
I could not get any closer to better than this. Any help would be greatly appreciated.
I don’t think this is Jetson related even though I’m limited to using TensorRT versions released for Jetson. What I want to know is about TensorRT, because the source code might need to be changed to allow waiting between layers.
Note: I’ve checked the website pages you shared. Thanks for sharing it anyway.
We recommend you try to create a plugin that does a cudaLaunchHostFunc and insert that in the network in the right place.
That should help you without modifying any source code.