Profiling custom layer with trtexec

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

Is it possible to profile a custom layer using trtexec just as you would a regular tensorrt layer? How can I go about doing this?

2

Hello,

Can you clarify what you meant by profile? Is it via the IProfiler interface? If so, you can use it for custom or regular layers.

3

Sorry, specifically I am looking to do speed test on a caffe model using custom layers.

I’ve been using the trtexec tool in the past for average over n runs only using regular layers.

I have implemented my custom layers in accordance with the Plugin samples and AastaNV/Face-Recognition examples. Is it possible to use trtexec for speed tests?

4

In particular, I get the following error:
could not parse layer type customLayer
Engine could not be created
Engine could not be created

5

Hello,

Yes, it should be possible to profile with trtexec if you are using the IPluginRegistry for the plugins. trtexec does not work with IPluginFactory.

In your case the error looks related to the parsing step because the caffeParser is unable to find the plugin implementation for the custom layer. If you register the plugin with IPluginRegistry, this error should not occur.

Regards,
NVIDIA Enterprise Support

6

It seems this is not possible with TensorRT4 on the TX2. Is there any alternative for my setup for using trtexec or similar tool for custom layers?

7

Hello,

What errors are you seeing when using trtexec with TRT4 on TX2? Have you tried it on a linux host? does that work for you? Just trying to see if it’s tx2 specific.