Teachable Machine h5 file not working on Jetson Nano Tensorflow version

Hey folks,

I have been trying to get a model loaded into a script generated by google’s teachable machine. It exports a keras h5 file.

I cannot find a build of Tensorflow beyond v1.14 for the nano. Currently when trying to load the keras h5 file it gives me an error that too many arguments are given, caleld “ragged”.

I looked this up and apparently the teachable machine generates a keras h5 file that is based on Tensorflow 1.15 onward that takes a ‘ragged’ argument and is not backward compatible.

Below is the code teachable machine generates to use the model:

import tensorflow.keras
from PIL import Image
import numpy as np

# Disable scientific notation for clarity
np.set_printoptions(suppress=True)

# Load the model
model = tensorflow.keras.models.load_model('keras_model.h5')

# Create the array of the right shape to feed into the keras model
# The 'length' or number of images you can put into the array is
# determined by the first position in the shape tuple, in this case 1.
data = np.ndarray(shape=(1, 224, 224, 3), dtype=np.float32)

# Replace this with the path to your image
image = Image.open('Path to your image')

# Make sure to resize all images to 224, 224 otherwise they won't fit in the array
image = image.resize((224, 224))
image_array = np.asarray(image)

# Normalize the image
normalized_image_array = (image_array.astype(np.float32) / 127.0) - 1

# Load the image into the array
data[0] = normalized_image_array

# run the inference
prediction = model.predict(data)
print(prediction)

I tried to install Tensorflow 2 but cannot find a build for Jetson Nano. Only place I found it was here https://jkjung-avt.github.io/build-tensorflow-2.0.0/ however it is going to take 40 hours to build.

Please advise! If I can work out how to simply plug the keras h5 file into a script like the one provided it would save me a whole heap of work.

Note: I’m new to the Jetson so yeah I might be overlooking something simples, I just don’t know!

Thank You!

Hi,

Here is a TensorFlow 2.0 prebuilt package for Jetson from the community:
https://github.com/peterlee0127/tensorflow-nvJetson/releases

Thanks.

Thannk you very much!!

It installed fine. However now any tensorflow script I run hangs on the bottom line, just sits there forever using max cpu on and off. Doesn’t freeze or crash just hangs forever.

2019-12-04 12:00:42.245894: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-04 12:00:42.246061: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties:
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-04 12:00:42.246155: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-04 12:00:42.246229: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-04 12:00:42.246288: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-04 12:00:42.246341: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-04 12:00:42.246394: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-04 12:00:42.246446: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-04 12:00:42.246495: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-04 12:00:42.246678: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-04 12:00:42.246946: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-04 12:00:42.247051: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-04 12:00:42.247175: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0.0    <---Hangs here

This happens when I am loading my keras file. I also tried to convert it to a tensorRT but that script hangs in exact same place too.

Screenshots:

1: https://ibb.co/jTkbj2n
2: https://ibb.co/xSSmLWw

Should I start a new thread with this?

Hi,

Please noticed that the package is built with JetPack4.2.
Are you using JetPack4.2?

Thanks.

Yeah I am using that. Eventually it just worked haha, it took ages but got passed that point. It does the prediction and returns it. but then there is a memory error at the end. I think from what I have read its because of the GPU/Swap memory share on the nano so I should learn to use TensorRT. Is that correct?

It takes 1.5 minutes to load up, the prediction itself is under a second.

Also it does some mad number rounding when the prediction result is too long as you can see below, the memory problem is at the end.

output:

/usr/bin/python3 /home/tom/Python_Progs/Tensor/h5Use.py
2019-12-05 11:36:09.897103: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 11:36:20.984050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-12-05 11:36:21.085774: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.086227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 11:36:21.086502: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 11:36:21.086819: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 11:36:21.167661: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 11:36:21.275727: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 11:36:21.432666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 11:36:21.517205: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 11:36:21.517415: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 11:36:21.517890: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.518500: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.518678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 11:36:21.546720: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.546910: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 11:36:21.546993: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 11:36:21.547063: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 11:36:21.547114: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 11:36:21.547161: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 11:36:21.547257: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 11:36:21.547313: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 11:36:21.547355: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 11:36:21.547517: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.547705: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:21.547784: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 11:36:21.547879: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 11:36:37.782954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-05 11:36:37.800403: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2019-12-05 11:36:37.800441: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2019-12-05 11:36:37.963727: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:38.190216: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:38.190522: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 11:36:38.190846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 198 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
sequential_1 (Sequential)    (None, 1280)              410208    
_________________________________________________________________
sequential_3 (Sequential)    (None, 2)                 128300    
=================================================================
Total params: 538,508
Trainable params: 524,428
Non-trainable params: 14,080
_________________________________________________________________
2019-12-05 11:37:27.446351: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 11:37:35.720135: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 11:38:04.388947: W tensorflow/core/common_runtime/bfc_allocator.cc:239] Allocator (GPU_0_bfc) ran out of memory trying to allocate 331.79MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
/home/tom/Python_Progs/Tensor/test02car.jpg
[[9.9974626e-01 2.5374891e-04]]

The output above is from the below script, although it works same as my original script both share the same memory failure, I have modified it since first post (the original script just needed compile=False in the load model call), works perfect on PC CPU version but not Jetson GPU version:

import tensorflow as tf
import tensorrt as trt
import numpy as np
from keras_preprocessing import image
import pathlib as plib
#import matplotlib                  # To display the image
#import matplotlib.pyplot as plt

keras_File = plib.Path('/home/tom/Python_Progs/Tensor/keras_model.h5')        #Jetson

#keras_File = r'T:\Highlight\ImageDetect_Py\ImageDetect\keras_model.h5'             #PC

model = tf.keras.models.load_model(keras_File, compile = False)
model.summary()

#Jetson
chosen_img = plib.Path('/home/tom/Python_Progs/Tensor/test02car.jpg')

#PC:

#chosen_img = r'T:\Highlight\ImageDetect_Py\ImageDetect\CarDetect\Occupied\test01car.jpg'
#chosen_img = r'T:\Highlight\ImageDetect_Py\ImageDetect\CarDetect\Occupied\test02car.jpg'
#chosen_img = r'T:\Highlight\ImageDetect_Py\ImageDetect\CarDetect\Occupied\test01vacant.jpg'

img = image.load_img(chosen_img, target_size = (224,224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis = 0)

images = np.vstack([x])

classes = model.predict(images, batch_size = 10)
print(chosen_img)
print(classes)

This is the output from my original script:

2019-12-05 11:48:28.323261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 178 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2019-12-05 11:49:08.690666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 11:49:10.673712: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 11:49:45.063301: W tensorflow/core/common_runtime/bfc_allocator.cc:239] Allocator (GPU_0_bfc) ran out of memory trying to allocate 331.79MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
[[0.9999987  0.00000129]]

Really appreciate your help :) Thanks!

So I have tried to convert a Tensorflow Saved Model (compiled by teachable machine with google) for TensorRT optimization. Hoping this will speed it up and work properly fix the memory issue above… however I am getting an error:

Converter Code:

from tensorflow.python.compiler.tensorrt import trt_convert as trt

import pathlib as plib

input_saved_model_dir = plib.Path('/home/tom/Python_Progs/Tensor/savedmodel/')
output_saved_model_dir = plib.Path('/home/tom/Python_Progs/Tensor/outmodel/')
input_saved_model_dir = str(input_saved_model_dir)
output_saved_model_dir = str(output_saved_model_dir)

converter = trt.TrtGraphConverterV2(input_saved_model_dir=input_saved_model_dir)
converter.convert()
converter.save(output_saved_model_dir)

Output:

/usr/bin/python3 /home/tom/Python_Progs/Tensor/convSavedModel.py
2019-12-05 15:44:35.792007: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:44:42.490310: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.5
WARNING:tensorflow:TensorRT mismatch. Compiled against version 5.0.6, but loaded 5.1.6. Things may not work
2019-12-05 15:45:02.995995: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-12-05 15:45:03.039523: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.039697: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 15:45:03.039774: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:45:03.039883: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 15:45:03.055304: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 15:45:03.093331: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 15:45:03.109549: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 15:45:03.124828: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 15:45:03.125019: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 15:45:03.125255: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.125521: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.125648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 15:45:03.128144: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.128302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 15:45:03.128386: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:45:03.128449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 15:45:03.128506: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 15:45:03.128563: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 15:45:03.128616: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 15:45:03.128664: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 15:45:03.128711: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 15:45:03.128891: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.129111: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:03.129198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 15:45:03.129300: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:45:15.797571: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-05 15:45:15.802419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2019-12-05 15:45:15.802447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2019-12-05 15:45:15.808713: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:15.808989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:15.809209: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:15.817460: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 143 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1781: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
WARNING:tensorflow:Issue encountered when serializing variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
to_proto not supported in EAGER mode.
WARNING:tensorflow:Issue encountered when serializing trainable_variables.
Type is unsupported, or the types of the items don't match field type in CollectionDef. Note this is a warning and probably safe to ignore.
to_proto not supported in EAGER mode.
2019-12-05 15:45:34.273897: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.274067: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2019-12-05 15:45:34.274297: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2019-12-05 15:45:34.275038: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.275173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 15:45:34.275361: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:45:34.275456: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 15:45:34.275523: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 15:45:34.275572: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 15:45:34.275620: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 15:45:34.275666: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 15:45:34.275709: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 15:45:34.275861: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.276050: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.276126: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 15:45:34.288217: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-05 15:45:34.288293: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2019-12-05 15:45:34.288319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2019-12-05 15:45:34.288569: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.288822: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:34.288945: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 143 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2019-12-05 15:45:34.872098: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize
2019-12-05 15:45:34.872190: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.005ms.
2019-12-05 15:45:34.872223: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   function_optimizer: function_optimizer did nothing. time = 0.001ms.
2019-12-05 15:45:42.493717: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.493950: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2019-12-05 15:45:42.494158: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2019-12-05 15:45:42.494981: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.495138: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: NVIDIA Tegra X1 major: 5 minor: 3 memoryClockRate(GHz): 0.9216
pciBusID: 0000:00:00.0
2019-12-05 15:45:42.495362: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-12-05 15:45:42.495472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-12-05 15:45:42.495569: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-12-05 15:45:42.495653: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-12-05 15:45:42.495733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-12-05 15:45:42.495810: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-12-05 15:45:42.495889: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-12-05 15:45:42.496183: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.496449: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.496529: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0
2019-12-05 15:45:42.496606: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-12-05 15:45:42.496638: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      0 
2019-12-05 15:45:42.496668: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0:   N 
2019-12-05 15:45:42.496860: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.497257: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:973] ARM64 does not support NUMA - returning NUMA node zero
2019-12-05 15:45:42.497424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 143 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2019-12-05 15:45:43.844906: I tensorflow/compiler/tf2tensorrt/segment/segment.cc:460] There are 4 ops of 3 different types in the graph that are not converted to TensorRT: Softmax, NoOp, Placeholder, (For more information see https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#supported-ops).
2019-12-05 15:45:43.974022: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:633] Number of TensorRT candidate segments: 1
2019-12-05 15:45:44.037919: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:734] TensorRT node TRTEngineOp_0 added for segment 0 consisting of 430 nodes succeeded.
2019-12-05 15:45:44.468514: W tensorflow/compiler/tf2tensorrt/convert/trt_optimization_pass.cc:183] TensorRTOptimizer is probably called on funcdef! This optimizer must *NOT* be called on function objects.
2019-12-05 15:45:44.589282: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: tf_graph
2019-12-05 15:45:44.589380: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 430 nodes (-263), 439 edges (-263), time = 170.223ms.
2019-12-05 15:45:44.589409: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   layout: Graph size after: 437 nodes (7), 446 edges (7), time = 206.055ms.
2019-12-05 15:45:44.589440: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 432 nodes (-5), 441 edges (-5), time = 136.568ms.
2019-12-05 15:45:44.589469: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   TensorRTOptimizer: Graph size after: 3 nodes (-429), 2 edges (-439), time = 296.231ms.
2019-12-05 15:45:44.589493: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 3 nodes (0), 2 edges (0), time = 53.271ms.
2019-12-05 15:45:44.589520: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: TRTEngineOp_0_native_segment
2019-12-05 15:45:44.589549: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 432 nodes (0), 441 edges (0), time = 48.034ms.
2019-12-05 15:45:44.589578: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   layout: Graph size after: 432 nodes (0), 441 edges (0), time = 60.248ms.
2019-12-05 15:45:44.589605: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 432 nodes (0), 441 edges (0), time = 53.446ms.
2019-12-05 15:45:44.589626: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   TensorRTOptimizer: Graph size after: 432 nodes (0), 441 edges (0), time = 6.602ms.
2019-12-05 15:45:44.589647: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718]   constant folding: Graph size after: 432 nodes (0), 441 edges (0), time = 57.387ms.
2019-12-05 15:46:17.035132: W tensorflow/core/framework/op_kernel.cc:1622] OP_REQUIRES failed at trt_engine_resource_ops.cc:183 : Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0)
Traceback (most recent call last):
  File "/home/tom/Python_Progs/Tensor/convSavedModel.py", line 12, in <module>
    converter.save(output_saved_model_dir)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/compiler/tensorrt/trt_convert.py", line 1069, in save
    save.save(self._saved_model, output_saved_model_dir, signatures)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/saved_model/save.py", line 872, in save
    signatures = signature_serialization.canonicalize_signatures(signatures)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/saved_model/signature_serialization.py", line 119, in canonicalize_signatures
    **tensor_spec_signature)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py", line 776, in get_concrete_function
    self._initialize(args, kwargs, add_initializers_to=initializer_map)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py", line 408, in _initialize
    *args, **kwds))
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py", line 1848, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py", line 2150, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/function.py", line 2041, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py", line 915, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/eager/def_function.py", line 358, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/func_graph.py", line 905, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in converted code:
    relative to /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/saved_model:

    signature_serialization.py:107 signature_wrapper  *
        return _normalize_outputs(
    signature_serialization.py:156 _normalize_outputs
        .format(value, key, compat.as_str_any(function_name)))

    ValueError: Got a dictionary containing non-Tensor value None for key __saved_model_init_op in the output of the function __inference_pruned_9207 used to generate a SavedModel signature. Dictionaries outputs for functions used as signatures should have one Tensor output per string key.

Hi,

It looks like you are using TF-TRT rather than pure TensorRT.

Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0)

Our suggestion is to use pure TensorRT to save the large required memory from TensorFlow.
To do this, you can start from this tutorial: https://github.com/AastaNV/TRT_object_detection
It demonstrates several models from TF Object Detection API.

Thanks.

Thank you I will have a look through that now!

I too am trying to run a model trained off teachable machine. Is there a simple way to run the keras model from the ./imagenet-camera.py script?