Misunderstandings with standalone plan for TF-TRT

AI & Data Science Deep Learning (Training & Inference) Frameworks
,
1

Hello. I have a problem with creating a .plan file with code:

  if n.op == "TRTEngineOp":
    print("Node: %s, %s" % (n.op, n.name.replace("/", "_")))
    with tf.gfile.GFile("%s.plan" % (n.name.replace("/", "_")), 'wb') as f:
      f.write(n.attr["serialized_segment"].s)
  else:
    print("Exclude Node: %s, %s" % (n.op, n.name.replace("/", "_"))

just right after trt.TrtGraphConverter(…) I have several .plan files, and according to this link I can convert one of it to TRT engine, but the problem is in that I have a couple of .plan FILES and as I understood it’s optimized engines which were saved to specific binaries, and they represent just simplified blocks of different operations and I need to understand how to build it together in the one engine. I will appreciate any help so I will wait for your advice and also will be glad if you will share with me your own approaches and code.

Kind regards.

1 Like