Uffparser fails to parse integer attribute

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I’m running TensorRT 5.0.3 on a Xavier board. I want to implement a plugin for TF’s split operation. Here is a simple model I created for testing:

import uff
import numpy as np
import tensorflow as tf

x = np.array([[31, 23, 4, 24, 27, 34],
              [18,  3, 25,  0,  6, 35],
              [28, 14, 33, 22, 20,  8]])

print(x.shape)

input = tf.placeholder(tf.float32, shape=x.shape, name='input')
split = tf.split(input, x.shape[0], name='output')

with tf.Session() as sess:
    out = sess.run(split, feed_dict = {input: x})
    print(out)
    g = tf.get_default_graph()
    print(g.get_operations())
    train_writer = tf.summary.FileWriter("session.log", sess.graph)

uff_model = uff.from_tensorflow(
    tf.get_default_graph(),
    output_nodes=['output'],
    output_filename='../data/model-split.uff',
    text=False,
)

When running this code, the graph generation and UFF conversion seems fine:

(3, 6)
2018-12-03 13:30:08.907213: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:924] ARM64 does not support NUMA - returning NUMA node zero
2018-12-03 13:30:08.907603: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: Xavier major: 7 minor: 2 memoryClockRate(GHz): 1.5
pciBusID: 0000:00:00.0
totalMemory: 15.46GiB freeMemory: 2.95GiB
2018-12-03 13:30:08.907758: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2018-12-03 13:30:10.125579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-12-03 13:30:10.125765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988]      0
2018-12-03 13:30:10.125864: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0:   N
2018-12-03 13:30:10.126235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2473 MB memory) -> physical GPU (device: 0, name: Xavier, pci bus id: 0000:00:00.0, compute capability: 7.2)
[array([[31., 23.,  4., 24., 27., 34.]], dtype=float32), array([[18.,  3., 25.,  0.,  6., 35.]], dtype=float32), array([[28., 14., 33., 22., 20.,  8.]], dtype=float32)]
[<tf.Operation 'input' type=Placeholder>, <tf.Operation 'Const' type=Const>, <tf.Operation 'output/split_dim' type=Const>, <tf.Operation 'output' type=Split>]
=== Automatically deduced input nodes ===
[name: "input"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: 3
      }
      dim {
        size: 6
      }
    }
  }
}
]
=========================================

Using output node output
Converting to UFF graph
Warning: No conversion function registered for layer: Split yet.
Converting as custom op Split output
name: "output"
op: "Split"
input: "output/split_dim"
input: "input"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "num_split"
  value {
    i: 3
  }
}

No. nodes: 4
UFF Output written to ../data/model-split.uff

However when I parse this UFF file in a C++ TensorRT program, I cannot recover the num_split attribute of the expected output. It seems to be an integer according to the UFF generation:

name: "output"
op: "Split"
input: "output/split_dim"
input: "input"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "num_split"
  value {
    i: 3
  }
}

But when I try to parse the FieldCollection parameter in the plugin it returns an unknown format.

SplitLayer::SplitLayer(const FieldCollection fc, const Weights* weights, int nbWeights)
:mNumSplit(0) {
  std::cout << "SplitLayer::SplitLayer - nbWeights=" << nbWeights << std::endl;

  std::cout << "Fields:" << std::endl;
  for (int i = 0; i < fc.nbFields; i++) {
    std::cout << i << ": " << fc.fields[i].name << " (" << static_cast<int>(fc.fields[i].type)
        << ") " << fc.fields[i].length << std::endl;
  }
}

UFFParser: parsing output/split_dim
UFFParser: parsing input
UFFParser: parsing output
UFFParser: Parse Plugin node output
UFFParser: PluginNode Input Descriptor input
UFFParser: Operation of node Input
SplitLayer::SplitLayer - nbWeights=0
Fields:
0: num_split (6) 1
1: T (5) 1
SplitLayer::getNbOutputs
SplitLayer::getNbOutputs

The num_split attribute is returned as a field with type FieldType::kUNKNOWN and length 1. Attribute T shows up properly.

I tried force-casting the data pointer into an int or a char to check if I could retrieve the expected value (3) but it does not work.

Can you double-check this specific use-case? It seems something wrong occurs during parsing. Thanks for your support.

2

Hello,

we are triaging. will keep you updated.

regards,
NVES.

3

Hello,

Per engineering:

Which Plugin API does this use? It looks like this code is using the IPluginFactory API
The issue here is that the parser is not able to figure out the type from the plugin field information. It parses the field name to figure out the type by looking at the last few characters of the name. To make this work, you will have to append “_u_int” to the field name (num_split_u_int).

If possible, switch to IPluginV2 and IPluginCreator for your plugin. Then if you use graphsurgeon API to add plugin node by calling create_plugin_node, that will be automatically taken care of internally. Refer to sampleUffSSD/config.py (C++) or sample_uff_ssd/utils/model.py (python)

4

I migrated to IPluginV2/IPluginCreator and I managed to access the plugin parameters. Thanks for your support.