Description
I am trying to convert my model which is in the SavedModel format to a TensorRT optimized model on the Jetson Nano. Error i’m getting:
Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0_0)
Environment
TensorRT Version: 8.0.1.6
GPU Type: NVIDIA Maxwell with 128 NVIDIA CUDA cores
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Python Version (if applicable): 3.6.9
TensorFlow Version (if applicable): 2.5.0
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
Relevant Files
from tensorflow.python.compiler.tensorrt import trt_convert as trt
conversion_params = trt.DEFAULT_TRT_CONVERSION_PARAMS
conversion_params = conversion_params._replace(precision_mode=“FP16”)
conversion_params = conversion_params._replace(max_workspace_size_bytes=1500000000)
encoder_model = trt.TrtGraphConverterV2(
input_saved_model_dir=‘/home/rohan/Desktop/original_models/encoder’,
conversion_params=conversion_params)
encoder_model.convert()
encoder_model.save(‘/home/rohan/Desktop/’)
OUTPUT:
rohan@rohan-desktop:~/Desktop/IR+RGB_1/Project files$ python3 conversion_trt.py
2021-12-10 00:03:07.522102: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.10.2
2021-12-10 00:03:13.071695: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libnvinfer.so.8
2021-12-10 00:03:14.018062: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2021-12-10 00:03:14.066070: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.066258: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1734] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-12-10 00:03:14.066357: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.10.2
2021-12-10 00:03:14.070564: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.10
2021-12-10 00:03:14.070823: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.10
2021-12-10 00:03:14.135587: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2021-12-10 00:03:14.191878: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2021-12-10 00:03:14.304716: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.10
2021-12-10 00:03:14.374083: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.10
2021-12-10 00:03:14.374872: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-12-10 00:03:14.375129: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.375374: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.375490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1872] Adding visible gpu devices: 0
2021-12-10 00:03:14.378149: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.378330: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1734] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-12-10 00:03:14.378514: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.378706: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:14.378802: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1872] Adding visible gpu devices: 0
2021-12-10 00:03:14.378903: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.10.2
2021-12-10 00:03:19.262801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-12-10 00:03:19.268118: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-12-10 00:03:19.268171: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-12-10 00:03:19.268567: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:19.269279: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:19.269514: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:19.269721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 184 MB memory) → physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2021-12-10 00:03:24.826897: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.827155: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2021-12-10 00:03:24.827583: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session
2021-12-10 00:03:24.828605: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.848923: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1734] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-12-10 00:03:24.862656: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.862936: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.863096: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1872] Adding visible gpu devices: 0
2021-12-10 00:03:24.863224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-12-10 00:03:24.863274: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-12-10 00:03:24.863311: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-12-10 00:03:24.863505: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.863713: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:24.863851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 184 MB memory) → physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2021-12-10 00:03:24.973046: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 19200000 Hz
2021-12-10 00:03:25.915354: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1171] Optimization results for grappler item: graph_to_optimize
function_optimizer: Graph size after: 48 nodes (37), 69 edges (58), time = 217.84ms.
function_optimizer: function_optimizer did nothing. time = 20.519ms.
2021-12-10 00:03:26.611717: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.611940: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2021-12-10 00:03:26.612406: I tensorflow/core/grappler/clusters/single_machine.cc:357] Starting new session
2021-12-10 00:03:26.613288: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.613448: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1734] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-12-10 00:03:26.613631: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.613821: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.613914: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1872] Adding visible gpu devices: 0
2021-12-10 00:03:26.614035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-12-10 00:03:26.614084: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-12-10 00:03:26.614124: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-12-10 00:03:26.614355: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.614632: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1001] ARM64 does not support NUMA - returning NUMA node zero
2021-12-10 00:03:26.614785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 184 MB memory) → physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2021-12-10 00:03:27.040761: I tensorflow/compiler/tf2tensorrt/segment/segment.cc:790] There are 5 ops of 3 different types in the graph that are not converted to TensorRT: Identity, NoOp, Placeholder, (For more information see Accelerating Inference In TF-TRT User Guide :: NVIDIA Deep Learning Frameworks Documentation).
2021-12-10 00:03:27.041774: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:759] Number of TensorRT candidate segments: 1
2021-12-10 00:03:27.045715: I tensorflow/compiler/tf2tensorrt/convert/convert_graph.cc:853] Replaced segment 0 consisting of 33 nodes by TRTEngineOp_0_0.
2021-12-10 00:03:27.074754: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1171] Optimization results for grappler item: tf_graph
constant_folding: Graph size after: 32 nodes (-16), 53 edges (-16), time = 122.766ms.
layout: Graph size after: 39 nodes (7), 60 edges (7), time = 113.376ms.
constant_folding: Graph size after: 36 nodes (-3), 57 edges (-3), time = 30.25ms.
TensorRTOptimizer: Graph size after: 4 nodes (-32), 3 edges (-54), time = 13.813ms.
constant_folding: Graph size after: 4 nodes (0), 3 edges (0), time = 1.62ms.
Optimization results for grappler item: TRTEngineOp_0_0_native_segment
constant_folding: Graph size after: 35 nodes (0), 48 edges (0), time = 3.846ms.
layout: Graph size after: 35 nodes (0), 48 edges (0), time = 4.584ms.
constant_folding: Graph size after: 35 nodes (0), 48 edges (0), time = 4.036ms.
TensorRTOptimizer: Graph size after: 35 nodes (0), 48 edges (0), time = 0.418ms.
constant_folding: Graph size after: 35 nodes (0), 48 edges (0), time = 3.749ms.
2021-12-10 00:03:28.423337: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at trt_engine_resource_ops.cc:193 : Not found: Container TF-TRT does not exist. (Could not find resource: TF-TRT/TRTEngineOp_0_0)
Steps To Reproduce
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered