How to run faster_rcnn_inception_v2 trained object detection tensorflow model on Jetson tx2?

Virat_Baahubali · October 25, 2020, 4:30pm

I have already configured Jetson Tx2 with tf_trtt_models and tensorflow successfully. I am able to import both tensorflow and tf_trt_models in python3 module on Jetson Tx2. I have faster_rcnn_inception_v2 trained object detection tensorflow model now I have couple of questions

tf_trtt_models repository converts models whose config files resembles to ssd_mobilenet_v1_coco and ssd_inception_v2_coco model’s config files. So is there any other way to convert faster_rcnn_inception_v2 model into tr_tf_model? or am I missing something?
what steps I need to follow in order to convert it into tf_trt_model? Can anyone please guide me through as it would really help me learn new concepts.

Please feel to reach out should you require more details thanks

Virat_Baahubali · October 25, 2020, 9:46pm

Running https://github.com/NVIDIA-AI-IOT/tf_trt_models/blob/master/examples/detection/detection.ipynb with faster_rcnn_inception_v2 it gives me following error

InvalidArgumentError: node BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Slice (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) has inputs from different frames. The input node BatchMultiClassNonMaxSuppression/map/while/Reshape_1 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) is in frame 'BatchMultiClassNonMaxSuppression/map/while/while_context'. The input node BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Slice/begin (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) is in frame ''.

AastaLLL · October 26, 2020, 3:48am

Hi,

The error seems to be a TensorFlow issue:

github.com/tensorflow/tensorflow

control_dependencies throw Frame error when used in while_loop

opened 03:52PM - 05 Dec 16 UTC

closed 05:09AM - 17 Feb 17 UTC

gongbudaizhe

Hello, ### Problem description I need to put InceptionV3 in a while loop to …save both GPU and CPU memory usage since I am handling videos, each contains hundreds of images. The problem is, InceptionV3 uses `control_dependencies` for BatchNorm and TensorFlow throws Frame Error if `control_dependencies` function is in the `while_loop`. It can run without errors if `control_dependencies` is removed though. Below is a minimal snippet that reproduces the error: ```python sess = tf.Session() with tf.variable_scope('state'): x = tf.get_variable('x', shape=(), initializer=tf.constant_initializer(1), dtype=tf.float32) update_x = tf.assign(x, x+1) def iter_fun(i, y): # comment the line below, the program will run without any error # but I need control_dependencies, or at least some way to replace it... with tf.control_dependencies([update_x]): y = y + x return (i+1, y) with tf.variable_scope('iteration'): num_iterations = 5 initial_i = tf.constant(0, dtype=tf.int32) initial_y = tf.constant(0, dtype=tf.float32) _, result = tf.while_loop( cond=lambda i, *_: i < num_iterations, body=iter_fun, loop_vars=(initial_i, initial_y)) init_op = tf.global_variables_initializer() sess.run(init_op) sess.run(result) ``` The stack trace of the error: ``` Traceback (most recent call last): File "demo.py", line 28, in <module> sess.run(result) File "/workspace/bily/anoaconda2/envs/tensorflow0.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run run_metadata_ptr) File "/workspace/bily/anoaconda2/envs/tensorflow0.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run feed_dict_string, options, run_metadata) File "/workspace/bily/anoaconda2/envs/tensorflow0.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run target_list, options, run_metadata) File "/workspace/bily/anoaconda2/envs/tensorflow0.12/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: The node 'iteration/while/add' has inputs from different frames. The input 'iteration/while/add/Enter' is in frame 'iteration/while/iteration/while/'. The input 'state/Assign' is in frame ''. ``` ### Environments - CentOS Linux release 7.2.1511 - TensorFlow 0.12 built from source - Python 2.7.12 - CUDA 7.5 and CUDNN v5.1 ### Related issues 1. [This issue in tflearn](https://github.com/tflearn/tflearn/issues/427) seems to be related to my problem but removing `control_denpendencies` isn't a solution for me. 2. #4478 and #3114 are issues about frame errors but these errors are caused by variables instead of `control_dependencies`. Any help will be appreciated : )

May I know which TensorFlow version do you use?
Would you mind to try the conversion on the latest package we shared this month?

tensorflow-1.15.4+nv20.10-cp36-cp36m-linux_aarch64

Thanks.

Virat_Baahubali · October 26, 2020, 9:28am

Hi,

I am having jetpack 4.2 L4T 32.1.0 with tensorrt 5.0.6.3 and below is the tensorflow version installed
$ sudo pip3 install --extra-index-url https://developer.download.nvidia.com/compute/redist/jp/v42 tensorflow-gpu==1.13.1+nv19.3

AastaLLL · October 27, 2020, 3:16am

Hi,

Do you have any dependence on JetPack4.2?
If not, it’s recommended to upgrade your device to the latest first.

Thanks.

Virat_Baahubali · October 27, 2020, 3:34am

Hi,

Can you please explain what does dependence on jetpack 4.2 means? Thank you

AastaLLL · October 28, 2020, 3:58am

Hi,

If you don’t need to use JetPack4.2, it’s recommended to reflash your device with JetPack4.4.x first.

Thanks.

Virat_Baahubali · November 3, 2020, 3:42am

Hi,

Installed latest jetpack version and tensorflow 1.15.4 container 20.10 successfully on jetson TX2. I am still getting InvalidArgumentError: node BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Slice (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) has inputs from different frames. The input node BatchMultiClassNonMaxSuppression/map/while/Reshape_1 (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) is in frame 'BatchMultiClassNonMaxSuppression/map/while/while_context'. The input node BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Slice/begin (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) is in frame ''.
when I run faster_rcnn_inception_v2_coco in https://github.com/NVIDIA-AI-IOT/tf_trt_models/blob/master/examples/detection/detection.ipynb

Can anyone guide me through this it would really help continue my learning? This how I have reached this much far. is it even possible to convert faster_rcnn_inception_v2_coco model in tf_trt model
Thanks

AastaLLL · November 17, 2020, 8:02am

Hi,

Since the GitHub doesn’t be updated for a while, it is quite possible that it doesn’t support the newer operations.
May I know your target first? Do you want to enable the model with TF-TRT or standalone TenosrRT?

It’s more recommended to use standalone TensorRT due to performance.
And you can find an example for Faster-RCNN here:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleUffFasterRCNN

Thanks.

Virat_Baahubali · November 17, 2020, 2:51pm

Hi,

I am trying to deploy my custom object detection faster_rcnn model trained with tensorflow on jetson tx2 platform. Since faster_rcnn model is not supported by tf-trt model but as you mentioned it has the workaround of using a convert-to-uff utility with trained custom object detection faster_rcnn model on tensorflow. I believe I should give it a try for this approach.

Can you please explain how tf-trt differs from convert-to-uff?
In order to use trained custom object detection faster_rcnn model would I just need frozen_graph.pb(Inference graph of trained model) ? as it mentions about .pb file in the attached guide on prerequisites section 3 point 2 so I am just assuming that would be frozen_graph of trained model.

Thanks

AastaLLL · December 1, 2020, 6:37am

Hi,

1.
TF-TRT use TensorFlow interface and moves some layers into TensorRT implementation if compatible.
This allows you to use the TensorFlow interface (pre-processing and post-processing) but requires much more resources.

For standalone TensorRT, there are two possible flow for TensorFlow based model:
[1] .pb → .uff → .plan
[2] .pb/.h5 → .onnx → .plan
In general, this requires a model conversion that represents the model in other format supported by the TensorRT(uff or onnx).

2.
Yes, please get the frozen file from your model and following steps to do the conversion.

Thanks.