Squeezenet, AxPy, TensorRT

samsonf89m5 · September 21, 2018, 2:01pm

Is there an existing conversion of the AxPy layer used in Squeezenet to TensorRT?

hujie-frank/SENet/blob/master/src/caffe/layers/axpy_layer.cu

/*
 * Axpy Layer
 *
 * Created on: May 1, 2017
 * Author: hujie
 */

#include "caffe/layers/axpy_layer.hpp"

namespace caffe {

template <typename Dtype>
__global__ void AxpyForward(const int count, const int spatial_dim, 
    const Dtype* scale_data, const Dtype* x_data, const Dtype* y_data,
    Dtype* out_data) {
  CUDA_KERNEL_LOOP(index, count) {
    out_data[index] = scale_data[index / spatial_dim] * x_data[index]
        + y_data[index];
  }
}

This file has been truncated. show original

I see some notes from a couple of years ago, but nothing about AxPy.

Thanks!

– Samson

NVES · September 21, 2018, 4:03pm

hello,

AxPy is an unsupported layer. Recommend TensorRT plugin API; it is for implementing custom layers (i.e. in a CUDA kernel)

See AastaNV’s TensorRT plugin sample here: GitHub - AastaNV/Face-Recognition: Demonstrate Plugin API for TensorRT2.1

samsonf89m5 · September 21, 2018, 6:22pm

I understand it is un-supported, and I may have to write a custom layer.

That said: I have to think someone has written this layer – squeezenet is a pretty common network. Has no one done this layer?

Also, could I potentially try to divide up the layer? Could I divide the layer in two, into an multiplication layer (Ax) and an addition layer (Py)? I’d prefer to do that then write a custom layer. Which layers should I use if I want to do that?

If I divided the layers, I would thin tensor rt would then be smart enough to merge them together during optimization.

Thanks!

Khurram5 · October 4, 2018, 2:49pm

Were you able to find solution to splitting the output of a layer into parts, as is done using view (PyTorch) or using slicing in numpy?