Calibration Cache Unnamed Layer

Description

I am trying to create a calibration cache file for ResNet-18 network with TensorRT. However, there are some Unnamed layer in the calibration cache file. How can I get the names of these layers?

TRT-7000-EntropyCalibration2
data: 3caa54fc
(Unnamed Layer* 0) [Convolution]_output: 3d6969df
(Unnamed Layer* 1) [Scale]_output: 3ac9c5e4
(Unnamed Layer* 2) [Scale]_output: 3bb23f5c
conv1: 3bb2132a
pool1: 3bb2132a
(Unnamed Layer* 5) [Convolution]_output: 3b0f3403
(Unnamed Layer* 6) [Scale]_output: 3c19f4a8
res2a_branch1: 3b39bf9e
(Unnamed Layer* 8) [Convolution]_output: 3b64b387
(Unnamed Layer* 9) [Scale]_output: 3c34ac95
(Unnamed Layer* 10) [Scale]_output: 3b309597
res2a_branch2a: 3b21cb45
(Unnamed Layer* 12) [Convolution]_output: 3acc6b2e
(Unnamed Layer* 13) [Scale]_output: 3cb1a784
res2a_branch2b: 3b5eba4f
(Unnamed Layer* 15) [ElementWise]_output: 3ba75a96
res2a: 3b73bec7
(Unnamed Layer* 17) [Convolution]_output: 3b8658ae
(Unnamed Layer* 18) [Scale]_output: 3cce329f
(Unnamed Layer* 19) [Scale]_output: 3b9b34cf
res2b_branch2a: 3b6795f7
(Unnamed Layer* 21) [Convolution]_output: 3b1fb3bc
(Unnamed Layer* 22) [Scale]_output: 3d1fdc85
res2b_branch2b: 3c0d6bd4
(Unnamed Layer* 24) [ElementWise]_output: 3ba9e0a1
res2b: 3ba9e0a1
(Unnamed Layer* 26) [Convolution]_output: 3b16170a
(Unnamed Layer* 27) [Scale]_output: 3cf6451c
res3a_branch1: 3b263b66
(Unnamed Layer* 29) [Convolution]_output: 3ba9a8f5
(Unnamed Layer* 30) [Scale]_output: 3cc67d8a
(Unnamed Layer* 31) [Scale]_output: 3ba7f0eb
res3a_branch2a: 3b86b13f
(Unnamed Layer* 33) [Convolution]_output: 3ae5e952

Environment

TensorRT Version: 7
GPU Type: GP102 [GeForce GTX 1080 Ti]
Nvidia Driver Version:
CUDA Version: 10.0
CUDNN Version: 7
Operating System + Version: Ubuntu 18
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Hi @afzaliku,
Please try setting the layer name while creating network.
Please check the below links for reference.


Thanks!

# Building engine
with trt.Builder(TRT_LOGGER) as builder, \
     builder.create_network(network_flags) as network, \
     builder.create_builder_config() as config, \
     trt.CaffeParser() as parser:
        
    config.max_workspace_size = 2**30 # 1GiB
    
    model_tensors = parser.parse(deploy="/home/hassan/tensorrt-utils/classification/imagenet/resnet18/deploy.prototxt", model="/home/hassan/tensorrt-utils/classification/imagenet/resnet18/resnet-18.caffemodel", network=network, dtype=trt.float32)
    network.mark_output(model_tensors.find("prob"))
    for i in range(92):
        print(network.get_layer(i).name)

The network which is created from the Caffe model has all the layers name. The TensorRT is removing the names of the layer.

2020-07-13 08:06:57 - ImagenetCalibrator - INFO - Calibration images pre-processed: 160/224
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use execute without batch size instead.
[TensorRT] INFO: Calibrated batch 4 in 0.0228161 seconds.
2020-07-13 08:06:57 - ImagenetCalibrator - INFO - Calibration images pre-processed: 192/224
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use execute without batch size instead.
[TensorRT] INFO: Calibrated batch 5 in 0.022112 seconds.
2020-07-13 08:06:57 - ImagenetCalibrator - INFO - Calibration images pre-processed: 224/224
[TensorRT] WARNING: Explicit batch network detected and batch size specified, use execute without batch size instead.
[TensorRT] INFO: Calibrated batch 6 in 0.0237534 seconds.
[TensorRT] INFO: Post Processing Calibration data in 7.35095 seconds.
[TensorRT] INFO: Calibration completed in 16.1074 seconds.
[TensorRT] INFO: Writing Calibration Cache for calibrator: TRT-7000-EntropyCalibration2
2020-07-13 08:07:05 - ImagenetCalibrator - INFO - Caching calibration data for future use: calibration.cache
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 1 output network tensors.
2020-07-13 08:07:21 - main - INFO - Serializing engine to file: resnet18.int8.engine
conv1
bn_conv1
scale_conv1
conv1_relu
pool1
res2a_branch1
bn2a_branch1
scale2a_branch1
res2a_branch2a
bn2a_branch2a
scale2a_branch2a
res2a_branch2a_relu
res2a_branch2b
bn2a_branch2b
scale2a_branch2b
res2a
res2a_relu
res2b_branch2a
bn2b_branch2a
scale2b_branch2a
res2b_branch2a_relu

Hi @afzaliku
Apologies for delayed response.
The scale layers TRT inserts during calibration are the Unnamed Layers
and since these layers are added during the calibration process, it seems they cant be named.
Thanks!

@AakankshaS Hi, Thank you for your response. Even if the TRT inserts new layers (Unnamed Layers), there is no scale value for some of the layers in the neural network (the layers stated in the deploy file). If the TRT wants to insert new layers that nobody knows what are these layers (what is their name), how are these layers going to be helpful? This seems a big bug for TensorRT. Do you agree?

Hi,
Can you please share your caffe prototxt file and the script, so that i can further look into this.

Thanks!

Hi,

You can download the deploy prototxt file and the caffe model from the below link:

Also do you have the script by your self for creating the calibration file?

Thank you.

@AakankshaS Hi, any updates?

Thanks.