Problem started when I was trying to convert Keras model with TensorRT.
Traceback (most recent call last):
File “TensorRT.py”, line 113, in
main()
File “TensorRT.py”, line 106, in main
convert(args.p)
File “TensorRT.py”, line 88, in convert
convertFP16(modelPath, fp16Path)
File “TensorRT.py”, line 59, in convertFP16
converter = trt.TrtGraphConverterV2(input_saved_model_dir=modelPath, conversion_params=conversion_params)
File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compiler/tensorrt/trt_convert.py”, line 899, in init
assert context.executing_eagerly()
AssertionError
Ran into issue after fresh install of Jetpack 4.4 on Jetson Nano which contains tensorflow==2.1.0+nv20.4. After looking into tensorflow trt_convert.py source, found the exception was due to tensorflow executing with tf.executing_eagerly() == False. That was a bit confusing since I was not using any TF 1.x compatibility components in conversion program, model, or training.
– To work around, I added tf.compat.v1.enable_eager_execution() as demonstrated below –
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
…
…
tf.compat.v1.enable_eager_execution()
print(‘Using Tensorflow version: {0}’.format(tf.version.VERSION))
print(tf.executing_eagerly())
This change allowed me to execute trt.TrtGraphConverterV2 on my keras models
I was under the impression that all TF 2.x versions ran with executing_eagerly == TRUE as default. So as a test I ran the commands below in python3 on Jetson Nano with JP 4.4/tensorflow 2.1.0+nv20.4
> > > import tensorflow as tf
> > > 2020-05-08 22:22:59.965357: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.2
> > > 2020-05-08 22:23:07.427738: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libnvinfer.so.7
> > > 2020-05-08 22:23:07.474677: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libnvinfer_plugin.so.7
> > > print(tf.executing_eagerly())
> > > False
When I do the same thing on JP 4.3 with tensorflow 2.1.0+nv20.3.tf2
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
> > > import tensorflow as tf
> > > 2020-05-08 22:47:32.666516: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
> > > 2020-05-08 22:47:37.064950: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
> > > 2020-05-08 22:47:37.067243: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
> > > print(tf.executing_eagerly())
> > > True
I’ve been using TF 1.15 on Nano for a long time without much problems and converted to TF 2.0 a few months back. Have had nothing but problems. In JP 4.3 I converted models to run native tf 2.0 and ran into issues with “RuntimeError: Can’t copy Tensor with type string to device” and the only solution was to run in v1 compatibility mode. Now with JP 4.4 it seems there is an issue with executing_eager behaving differently on Nano than on my desktop. The reason I went down this rabbit hole initially was because I needed to improve model inference performance with TensorRT because my TF 2.x native model was running approx 3 times slower than the model I used in TF 1.15
Would like to see TF 2.x stabilized on the platform. Spending way to much time debugging platform problems.