Getting size of output layers for Dashcam model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello,
How do you get the shape of the outputs layers for the dashcam model? Is there an API to print out layer sizes for TensorRT models, or for layers in TLT models? nvdsinfer_custombboxparser.cpp has the computations but it’s not very obvious how to retrieve the constants of the output layer.

3

Actually for tlt model, you can run tlt-train or tlt-evaluate to get the information of each layer and its output size.
For trt engine, you can run “/usr/src/tensorrt/bin/trtexec --loadEngine=xxx.engine --dumpProfile” to get the layer information.

1 Like
4

trtexec only gave the time taken to run the model.

[11/05/2020-16:06:40] [I] === Profile (1672 iterations ) ===
[11/05/2020-16:06:40] [I]                                                         Layer   Time (ms)   Avg. Time (ms)   Time %
....
[11/05/2020-16:06:40] [I]     conv1/convolution + activation_1/Relu input reformatter 0       66.17             0.04      4.4
[11/05/2020-16:06:40] [I]                                        output_cov/convolution       28.55             0.02      1.9
[11/05/2020-16:06:40] [I]                                            output_cov/Sigmoid        6.52             0.00      0.4

It seems like these functions gave the numbers I required:

std::string model_data = LoadFile(model_file);
nvinfer1::IRuntime *runtime = nvinfer1::createInferRuntime(logger);
nvinfer1::ICudaEngine *engine = runtime_->deserializeCudaEngine(model_data.data(), model_data.length(), nullptr);
nvinfer1::IExecutionContext *context = engine_->createExecutionContext();
int output1 = engine_->getBindingIndex(“output_cov/Sigmoid”);
int output2 = engine_->getBindingIndex(“output_bbox/BiasAdd”);

nvinfer1::Dims covdim = context_->getBindingDimensions(output1);
nvinfer1::Dims boxdim = context_->getBindingDimensions(output2);
// 4 34 60
//16 34 60
for (int i=0;i<covdim.nbDims;i++){
std::cout << covdim.d[i] << " ";
}
std::cout <<std::endl;
for (int i=0;i<boxdim.nbDims;i++){
std::cout << boxdim.d[i] << " ";
}
std::cout <<std::endl;