Can I add a new node in the middle of a Tensorflow model graph using the graphsurgeon tool?

Hello,
I’m using TensorRT version 4.0.1.6.

I successfully replaced an unsupported Tensorflow operation with a TensorRT PlugIn layer using the graphsurgeon APIs.

Now, I want to add a totally new node in the middle of the model.

I saw that the graphsurgeon support these APIs:
create_node,
create_plugin_node,
extend,
append

But I couldn’t understand if I can use them for my purpose.

I know how to find the specific model location where I want to add my new node but I cannot find the right way to use the graphsurgeon APIs to do that.

Please advise,

You can convert graph_def to GraphSurgeons DynamicGraph, modify it and finally write it as uff:

import graphsurgeon as gs
import uff

...
# Take graph_def from current TF session and convert it to DynamicGraph
graph = session.graph_def
dynamic_graph = gs.DynamicGraph(graph)

# Edit dynamic graph by changing operations, adding nodes, etc...
...
# Input node strings need to have following format:
# '<name_of_input>,<name_of_input>,<dtype>,<dim0>,<dim1>,<dim2>,<dim3>,...'
input_nodes = ['input,input,float32,-1,3,512,512']
output_nodes = ['names','of','output','nodes']
uff.from_tensorflow(dynamic_graph, output_nodes=output_nodes, output_filename='output.uff',
                            input_node=input_nodes)

Thanks juko.lam

My purpose is to add a debug node multiple times in the middle of the grpah in order to implement a simple functionallity:
Save to disk the tensor input data.

By this way I will have the ability to investigate the graph nodes tensor outputs data correctness.

So, as I wrote previously, I have the following abilities:

  1. uff file generation using uff.from_tensorflow_frozen_model API
  2. Replace an existing nodes in the graph using graphsurgeon API
  3. Remove an existing nodes in the graph using graphsurgeon API

What I don’t have and till now I couldn’t succeeded to do is to add totally new node somewhere in the graph

Can you provide me please an example of how to add new node in the middle of the graph using graphsurgeon APIs?

Based on my constraints (I have only the frozen graph pb file),
Do you have any other idea (not related to the graphsurgeon) to to achieve my purpose?

You can load the frozen pb file which gives you the graph, then convert the graph to DynamicGraph and modify it. So something like this (probably not runnable directly):

import tensorflow as tf
from tensorflow.core.framework import types_pb2
import graphsurgeon as gs

# Load the graph
graph = tf.Graph()
with tf.gfile.GFile(frozen_graph_filename, "rb") as f:
    graph_def = tf.GraphDef()
    graph_def.ParseFromString(f.read())
with tf.Graph().as_default() as graph:
    tf.import_graph_def(graph_def, name='')
graph = gs.DynamicGraph(graph)

# Add new node
node = tf.NodeDef()
node.op = 'my-custom-operation'
node.name = 'my-custom-node'
graph.append(node)

# Now the new node is disconnected from rest of the graph
# Modify existing graph to point to the new node
# Modification is done by changing input lists which contain names of input nodes
next = graph.find_nodes_by_name('next_node')[0] # assume only one node is found
node.input = next.input
next.input = [node.name]

I suggest that you inspect graphs created by the TRT examples just before they are written to UFF file.

juko.lam

I tried this to replace the ResizeBilinear with Conv2DTranspose but I got error :

“convert_lanenet_uff.py”, line 32, in add_plugin
node.input = next_.input
AttributeError: Assignment not allowed to repeated field “input” in protocol message object.

Code snippet :
import tensorflow as tf
from tensorflow.core.framework import types_pb2
import graphsurgeon as gs

graph = gs.DynamicGraph(‘frozen_graph.pb’)

Add new node

node = tf.NodeDef()
node.op = ‘Conv2DTranspose’
node.name = ‘Conv2DTranspose’
graph.append(node)

want to replace ResizeBilinear with Conv2DTranspose layer

next = graph.find_nodes_by_op(“ResizeBilinear”)[0] # assume only one node is found
node.input = next.input
next.input = [node.name]