Get current layers

Hi,
Is it possible to get the layer names/properties out of a context/engine object?
There is a getNbLayers() function on ICudaEngine object but I didn’t see how I can enumerate the actual layers …
Basically I’d like to know the specific of a certain layer which is reported to take most of the time when running the network…

thanks
Eyal

Hi,

There is a function can return the layer name from index.
https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/classnvinfer1_1_1_i_cuda_engine.html#ae22d5f8824082e023d13fc067581dfa5

You can get the layer number from getNbLayers() and check the layer’s name one by one from index.


virtual const char* nvinfer1::ICudaEngine::getBindingName	(	int 	bindingIndex	)	const

More, you can also use our profiler to get the layer level performance result.

/usr/src/tensorrt/samples/common/common.h


struct SimpleProfiler : public nvinfer1::IProfiler
{
    struct Record
    {
    ...

Thanks.

1 Like

Thanks a lot. I’ve indeed used my own simple implementation of the IProfiler, however the layer names were all meaningless such as these:
[(Unnamed Layer* 0) [Shuffle]]
[(Unnamed Layer* 4) [Pooling]]
[(Unnamed Layer* 5) [Convolution] + (Unnamed Layer* 7) [Activation]

Could it be that the real layer names were changed (as these are not the real layer names that the original PB file had) during the pb->onnx->plan process?

I’ve tried the getBindingName function as you’ve suggested. I’m a bit confused.
I’m getting 77 layers from getNbLayers(), which is correct, when I enumerate it, the application crashes after 13 (and it displays the input + 12 output bindings names) and not the actual layer names… am I doing something wrong here?

Also, can I get the layers properties (beside its name) from the ICudaEngine object?

Thanks
Eyal

Hi,

TensorRT apply merge for optimization so there won’t be an one-to-one matching from TensorFlow layer.
It’s recommended to locate the bottleneck layer first and then find the corresponding TF layer.

Binding indicates that the TensorRT need to make the buffer accessible for CPU.
This may cause some memory issue if too many tensors is binded.
You don’t need to set them to output to check the tensor name.

The layer properties can be found from ITensor:
https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/c_api/classnvinfer1_1_1_i_tensor.html

Thanks.

1 Like

Thanks!
Sorry for the late followup.

In one of my scenarios I call deserializeCudaEngine on the runtime object. But then I can not reach the INetworkDefinition object (TRT 5.1) in order to enumerate the layers that are available, the network seems to be empty in that scenario.

Am I missing something?
thanks
Eyal

Hi @AastaLLL ,
I’ve re-stumbled on this issue again.
I want to run the network many times but each time to indicate a different layer as the output, such that if I have 100 layers, I want to run it 100 times and each time a different layer is marked as the output.
As far as I saw this can ONLY be done on the INetworkDefinition object, however if I deserialize the full network (the full 100 layers) I don’t have the INetworkDefinition object to do so… and therefore I need to build the network 100 times? which takes huge amounts of time…

Is there any easier way to do this?

thanks
Eyal

Hi eyalhir74,

Please help to open a new topic for this issue. Thanks