Waiting after specific layer in TensorRT

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I am wondering whether there is any way to modify TensorRT so that we can wait between the layers. To clearly state (just an example), I want to insert a wait time after the 20th layer in an inference of DNN having 30 layers. I’ve checked sampleInference.cpp, however this does not seem visible.

I could not get any closer to better than this. Any help would be greatly appreciated.

Environment

Jetpack 5.0.1

2

Hi,

This looks like a Jetson issue. Please refer to the below samples in case useful.

For any further assistance, we will move this post to to Jetson related forum.

Thanks!

4

I don’t think this is Jetson related even though I’m limited to using TensorRT versions released for Jetson. What I want to know is about TensorRT, because the source code might need to be changed to allow waiting between layers.

Note: I’ve checked the website pages you shared. Thanks for sharing it anyway.

Thanks.

5

Any follow-up to this?

6

Hi,

We recommend you try to create a plugin that does a cudaLaunchHostFunc and insert that in the network in the right place.
That should help you without modifying any source code.

Thank you.