Memory problem with keras-ocr?

Hi,

I’m currently working on the jetson nano to perform OCR. I tried to use it on an image but got an error related to memory I believe.

Here is my code :

import keras_ocr

pipeline = keras_ocr.pipeline.Pipeline()

images = [
    keras_ocr.tools.read(img) for img in ['images/cropped.jpg']
]

prediction_groups = pipeline.recognize(images)

for i in range(len(prediction_groups[0])):
    print(prediction_groups[0][i][0])

And the error I got :

Sum Total of in-use chunks: 138.96MiB
2021-07-27 16:35:29.532472: I tensorflow/core/common_runtime/bfc_allocator.cc:1040] total_region_allocated_bytes_: 167763968 memory_limit_: 167763968 available bytes: 0 curr_region_allocation_bytes_: 335527936
2021-07-27 16:35:29.532513: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Stats: 
Limit:                       167763968
InUse:                       145711360
MaxInUse:                    154744576
NumAllocs:                        1084
MaxAllocSize:                 25690112
Reserved:                            0
PeakReserved:                        0
LargestFreeBlock:                    0

2021-07-27 16:35:29.532616: W tensorflow/core/common_runtime/bfc_allocator.cc:439] ********************************************xxx*******************______******__************xx
Traceback (most recent call last):
  File "keras_test.py", line 21, in <module>
    prediction_groups = pipeline.recognize(images)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/keras_ocr/pipeline.py", line 55, in recognize
    box_groups = self.detector.detect(images=images, **detection_kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/keras_ocr/detection.py", line 682, in detect
    boxes = getBoxes(self.model.predict(np.array(images), **kwargs),
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 130, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 1599, in predict
    tmp_batch_outputs = predict_function(iterator)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 780, in _call_
    result = self._call(*args, **kwds)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 846, in _call
    return self._concrete_stateful_fn._filtered_call(canon_args, canon_kwds)  # pylint: disable=protected-access
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
    cancellation_manager=cancellation_manager)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
    ctx, args, cancellation_manager=cancellation_manager))
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 550, in call
    ctx=ctx)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
    inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InternalError:  Dst tensor is not initialized.
	 [[{{node IteratorGetNext/_2}}]] [Op:__inference_predict_function_6505]

Function call stack:
predict_function

As I said it seems to come from the gpu memory being overloaded but I don’t know how i could fix this ? I have the 4GB version by the way.

Thanks for your help

Hi,

Dst tensor is not initialized.

This error looks like an API usage problem.

Have you tried the script on a desktop environment with dGPU before?
If not, could you give it a try?

Thanks.

Thanks for your answer,

If I understood correctly what you meant, yes the script works on my ubuntu 20.04 computer with a RTX 2060. So I don’t think the code is the source of the problem.

I tried running it again on the nano and got another error, I put you everything this time :

2021-07-28 12:05:40.697152: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2
Looking for /home/quadient/.keras-ocr/craft_mlt_25k.h5
2021-07-28 12:05:50.206055: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2021-07-28 12:05:50.212491: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.212652: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1742] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-07-28 12:05:50.212758: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2
2021-07-28 12:05:50.217804: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-07-28 12:05:50.221326: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-07-28 12:05:50.222332: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-07-28 12:05:50.227968: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-07-28 12:05:50.232081: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-07-28 12:05:50.232790: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-07-28 12:05:50.233274: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.233668: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.233782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1884] Adding visible gpu devices: 0
2021-07-28 12:05:50.263544: W tensorflow/core/platform/profile_utils/cpu_utils.cc:108] Failed to find bogomips or clock in /proc/cpuinfo; cannot determine CPU frequency
2021-07-28 12:05:50.264127: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x151f2610 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-07-28 12:05:50.264186: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-07-28 12:05:50.337820: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.338132: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x15234fa0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-07-28 12:05:50.338187: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): NVIDIA Tegra X1, Compute Capability 5.3
2021-07-28 12:05:50.338725: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.338838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1742] Found device 0 with properties:
pciBusID: 0000:00:00.0 name: NVIDIA Tegra X1 computeCapability: 5.3
coreClock: 0.9216GHz coreCount: 1 deviceMemorySize: 3.86GiB deviceMemoryBandwidth: 194.55MiB/s
2021-07-28 12:05:50.338945: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2
2021-07-28 12:05:50.339072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.10
2021-07-28 12:05:50.339161: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2021-07-28 12:05:50.339237: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2021-07-28 12:05:50.339309: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2021-07-28 12:05:50.339380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.10
2021-07-28 12:05:50.339450: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2021-07-28 12:05:50.339707: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.339973: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:50.340041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1884] Adding visible gpu devices: 0
2021-07-28 12:05:50.340124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.2
2021-07-28 12:05:52.510201: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1283] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-07-28 12:05:52.510361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1289]      0
2021-07-28 12:05:52.510398: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1302] 0:   N
2021-07-28 12:05:52.510956: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:52.511342: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1046] ARM64 does not support NUMA - returning NUMA node zero
2021-07-28 12:05:52.511501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1428] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 88 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
2021-07-28 12:06:04.705958: W tensorflow/core/common_runtime/bfc_allocator.cc:431] Allocator (GPU_0_bfc) ran out of memory trying to allocate 18.00MiB (rounded to 18874368)requested by op RandomUniform
Current allocation summary follows.
2021-07-28 12:06:04.706038: I tensorflow/core/common_runtime/bfc_allocator.cc:970] BFCAllocator dump for GPU_0_bfc
2021-07-28 12:06:04.706080: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (256): Total Chunks: 29, Chunks in use: 29. 7.2KiB allocated for chunks. 7.2KiB in use in bin. 2.6KiB client-requested in use in bin.
2021-07-28 12:06:04.706118: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (512): Total Chunks: 10, Chunks in use: 10. 5.2KiB allocated for chunks. 5.2KiB in use in bin. 5.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706156: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (1024): Total Chunks: 16, Chunks in use: 16. 16.8KiB allocated for chunks. 16.8KiB in use in bin. 16.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706191: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (2048): Total Chunks: 30, Chunks in use: 30. 60.0KiB allocated for chunks. 60.0KiB in use in bin. 60.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706223: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (4096): Total Chunks: 1, Chunks in use: 1. 6.8KiB allocated for chunks. 6.8KiB in use in bin. 6.8KiB client-requested in use in bin.
2021-07-28 12:06:04.706253: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (8192): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706282: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (16384): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706311: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (32768): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706339: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (65536): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706372: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (131072): Total Chunks: 2, Chunks in use: 1. 358.2KiB allocated for chunks. 144.0KiB in use in bin. 144.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706404: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (262144): Total Chunks: 2, Chunks in use: 1. 576.0KiB allocated for chunks. 288.0KiB in use in bin. 288.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706438: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (524288): Total Chunks: 1, Chunks in use: 1. 576.0KiB allocated for chunks. 576.0KiB in use in bin. 576.0KiB client-requested in use in bin.
2021-07-28 12:06:04.706469: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (1048576): Total Chunks: 3, Chunks in use: 1. 3.38MiB allocated for chunks. 1.12MiB in use in bin. 1.12MiB client-requested in use in bin.
2021-07-28 12:06:04.706499: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (2097152): Total Chunks: 2, Chunks in use: 2. 4.50MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2021-07-28 12:06:04.706530: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (4194304): Total Chunks: 3, Chunks in use: 1. 13.50MiB allocated for chunks. 4.50MiB in use in bin. 4.50MiB client-requested in use in bin.
2021-07-28 12:06:04.706563: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (8388608): Total Chunks: 7, Chunks in use: 5. 65.70MiB allocated for chunks. 45.00MiB in use in bin. 45.00MiB client-requested in use in bin.
2021-07-28 12:06:04.706592: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (16777216): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706621: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (33554432): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706649: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (67108864): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706677: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (134217728): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706705: I tensorflow/core/common_runtime/bfc_allocator.cc:977] Bin (268435456): Total Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.
2021-07-28 12:06:04.706736: I tensorflow/core/common_runtime/bfc_allocator.cc:993] Bin for 18.00MiB was 16.00MiB, Chunk State:
2021-07-28 12:06:04.706760: I tensorflow/core/common_runtime/bfc_allocator.cc:1006] Next region of size 92950528
2021-07-28 12:06:04.706794: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30000 of size 1280 next 1
2021-07-28 12:06:04.706821: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30500 of size 256 next 5
2021-07-28 12:06:04.706847: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30600 of size 256 next 8
2021-07-28 12:06:04.706872: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30700 of size 256 next 9
2021-07-28 12:06:04.706898: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30800 of size 256 next 10
2021-07-28 12:06:04.706925: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30900 of size 256 next 11
2021-07-28 12:06:04.706952: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30a00 of size 256 next 12
2021-07-28 12:06:04.706982: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30b00 of size 256 next 13
2021-07-28 12:06:04.707008: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30c00 of size 256 next 15
2021-07-28 12:06:04.707033: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30d00 of size 256 next 16
2021-07-28 12:06:04.707058: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30e00 of size 256 next 14
2021-07-28 12:06:04.707083: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b30f00 of size 256 next 19
2021-07-28 12:06:04.707107: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31000 of size 256 next 20
2021-07-28 12:06:04.707132: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31100 of size 256 next 21
2021-07-28 12:06:04.707156: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31200 of size 256 next 22
2021-07-28 12:06:04.707181: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31300 of size 256 next 23
2021-07-28 12:06:04.707206: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31400 of size 256 next 24
2021-07-28 12:06:04.707232: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31500 of size 512 next 27
2021-07-28 12:06:04.707256: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31700 of size 512 next 28
2021-07-28 12:06:04.707281: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31900 of size 512 next 29
2021-07-28 12:06:04.707306: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31b00 of size 512 next 30
2021-07-28 12:06:04.707333: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b31d00 of size 768 next 2
2021-07-28 12:06:04.707358: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32000 of size 256 next 3
2021-07-28 12:06:04.707383: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32100 of size 256 next 4
2021-07-28 12:06:04.707407: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32200 of size 256 next 32
2021-07-28 12:06:04.707431: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32300 of size 256 next 33
2021-07-28 12:06:04.707455: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32400 of size 512 next 31
2021-07-28 12:06:04.707479: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32600 of size 512 next 36
2021-07-28 12:06:04.707505: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32800 of size 512 next 37
2021-07-28 12:06:04.707530: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32a00 of size 512 next 38
2021-07-28 12:06:04.707555: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32c00 of size 512 next 39
2021-07-28 12:06:04.707579: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32e00 of size 256 next 40
2021-07-28 12:06:04.707603: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b32f00 of size 256 next 41
2021-07-28 12:06:04.707628: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b33000 of size 1024 next 44
2021-07-28 12:06:04.707653: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b33400 of size 1024 next 45
2021-07-28 12:06:04.707678: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b33800 of size 1536 next 6
2021-07-28 12:06:04.707705: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b33e00 of size 6912 next 7
2021-07-28 12:06:04.707731: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b35900 of size 1024 next 46
2021-07-28 12:06:04.707756: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b35d00 of size 1024 next 47
2021-07-28 12:06:04.707781: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36100 of size 256 next 49
2021-07-28 12:06:04.707805: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36200 of size 256 next 50
2021-07-28 12:06:04.707830: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36300 of size 1024 next 48
2021-07-28 12:06:04.707855: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36700 of size 1024 next 53
2021-07-28 12:06:04.707880: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36b00 of size 1024 next 54
2021-07-28 12:06:04.707904: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b36f00 of size 1024 next 55
2021-07-28 12:06:04.707929: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b37300 of size 1024 next 56
2021-07-28 12:06:04.707953: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b37700 of size 1024 next 57
2021-07-28 12:06:04.707977: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b37b00 of size 1024 next 59
2021-07-28 12:06:04.708003: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b37f00 of size 1024 next 60
2021-07-28 12:06:04.708028: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b38300 of size 1024 next 61
2021-07-28 12:06:04.708053: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b38700 of size 1024 next 62
2021-07-28 12:06:04.708078: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b38b00 of size 256 next 63
2021-07-28 12:06:04.708103: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b38c00 of size 256 next 64
2021-07-28 12:06:04.708129: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b38d00 of size 2048 next 67
2021-07-28 12:06:04.708154: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b39500 of size 2048 next 68
2021-07-28 12:06:04.708178: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b39d00 of size 2048 next 69
2021-07-28 12:06:04.708203: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3a500 of size 2048 next 70
2021-07-28 12:06:04.708227: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3ad00 of size 2048 next 71
2021-07-28 12:06:04.708251: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3b500 of size 256 next 73
2021-07-28 12:06:04.708276: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3b600 of size 256 next 74
2021-07-28 12:06:04.708302: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3b700 of size 2048 next 72
2021-07-28 12:06:04.708326: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3bf00 of size 2048 next 77
2021-07-28 12:06:04.708350: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3c700 of size 2048 next 78
2021-07-28 12:06:04.708374: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3cf00 of size 2048 next 79
2021-07-28 12:06:04.708398: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3d700 of size 2048 next 80
2021-07-28 12:06:04.708423: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3df00 of size 2048 next 81
2021-07-28 12:06:04.708448: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3e700 of size 2048 next 83
2021-07-28 12:06:04.708473: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3ef00 of size 2048 next 84
2021-07-28 12:06:04.708499: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3f700 of size 2048 next 85
2021-07-28 12:06:04.708524: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b3ff00 of size 2048 next 86
2021-07-28 12:06:04.708550: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b40700 of size 2048 next 87
2021-07-28 12:06:04.708574: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b40f00 of size 2048 next 89
2021-07-28 12:06:04.708598: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b41700 of size 2048 next 90
2021-07-28 12:06:04.708622: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b41f00 of size 2048 next 91
2021-07-28 12:06:04.708647: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b42700 of size 2048 next 92
2021-07-28 12:06:04.708672: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b42f00 of size 2048 next 93
2021-07-28 12:06:04.708697: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b43700 of size 2048 next 95
2021-07-28 12:06:04.708721: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b43f00 of size 2048 next 96
2021-07-28 12:06:04.708746: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b44700 of size 2048 next 97
2021-07-28 12:06:04.708770: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b44f00 of size 2048 next 98
2021-07-28 12:06:04.708795: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b45700 of size 2048 next 99
2021-07-28 12:06:04.708820: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b45f00 of size 2048 next 101
2021-07-28 12:06:04.708845: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b46700 of size 2048 next 102
2021-07-28 12:06:04.708870: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b46f00 of size 2048 next 103
2021-07-28 12:06:04.708894: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b47700 of size 2048 next 104
2021-07-28 12:06:04.708919: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b47f00 of size 256 next 105
2021-07-28 12:06:04.708945: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f00b48000 of size 219392 next 18
2021-07-28 12:06:04.709026: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00b7d900 of size 147456 next 17
2021-07-28 12:06:04.709056: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f00ba1900 of size 294912 next 26
2021-07-28 12:06:04.709081: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00be9900 of size 294912 next 25
2021-07-28 12:06:04.709107: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f00c31900 of size 1179648 next 35
2021-07-28 12:06:04.709132: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00d51900 of size 589824 next 34
2021-07-28 12:06:04.709157: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f00de1900 of size 1179648 next 43
2021-07-28 12:06:04.709182: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f00f01900 of size 1179648 next 42
2021-07-28 12:06:04.709208: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f01021900 of size 4718592 next 52
2021-07-28 12:06:04.709233: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f014a1900 of size 2359296 next 51
2021-07-28 12:06:04.709259: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f016e1900 of size 2359296 next 58
2021-07-28 12:06:04.709285: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f01921900 of size 4718592 next 66
2021-07-28 12:06:04.709310: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f01da1900 of size 4718592 next 65
2021-07-28 12:06:04.709336: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f02221900 of size 9437184 next 100
2021-07-28 12:06:04.709362: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f02b21900 of size 9437184 next 76
2021-07-28 12:06:04.709387: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f03421900 of size 9437184 next 75
2021-07-28 12:06:04.709411: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f03d21900 of size 9437184 next 82
2021-07-28 12:06:04.709436: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f04621900 of size 9437184 next 88
2021-07-28 12:06:04.709460: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] InUse at f04f21900 of size 9437184 next 94
2021-07-28 12:06:04.709485: I tensorflow/core/common_runtime/bfc_allocator.cc:1026] Free  at f05821900 of size 12269312 next 18446744073709551615
2021-07-28 12:06:04.709509: I tensorflow/core/common_runtime/bfc_allocator.cc:1031]      Summary of in-use Chunks by size:
2021-07-28 12:06:04.709540: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 29 Chunks of size 256 totalling 7.2KiB
2021-07-28 12:06:04.709568: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 9 Chunks of size 512 totalling 4.5KiB
2021-07-28 12:06:04.709595: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 768 totalling 768B
2021-07-28 12:06:04.709622: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 14 Chunks of size 1024 totalling 14.0KiB
2021-07-28 12:06:04.709650: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 1280 totalling 1.2KiB
2021-07-28 12:06:04.709676: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 1536 totalling 1.5KiB
2021-07-28 12:06:04.709705: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 30 Chunks of size 2048 totalling 60.0KiB
2021-07-28 12:06:04.709733: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 6912 totalling 6.8KiB
2021-07-28 12:06:04.709762: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 147456 totalling 144.0KiB
2021-07-28 12:06:04.709790: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 294912 totalling 288.0KiB
2021-07-28 12:06:04.709817: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 589824 totalling 576.0KiB
2021-07-28 12:06:04.709844: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 1179648 totalling 1.12MiB
2021-07-28 12:06:04.709871: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 2 Chunks of size 2359296 totalling 4.50MiB
2021-07-28 12:06:04.709897: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 1 Chunks of size 4718592 totalling 4.50MiB
2021-07-28 12:06:04.709925: I tensorflow/core/common_runtime/bfc_allocator.cc:1034] 5 Chunks of size 9437184 totalling 45.00MiB
2021-07-28 12:06:04.709952: I tensorflow/core/common_runtime/bfc_allocator.cc:1038] Sum Total of in-use chunks: 56.20MiB
2021-07-28 12:06:04.709977: I tensorflow/core/common_runtime/bfc_allocator.cc:1040] total_region_allocated_bytes_: 92950528 memory_limit_: 92950528 available bytes: 0 curr_region_allocation_bytes_: 185901056
2021-07-28 12:06:04.710011: I tensorflow/core/common_runtime/bfc_allocator.cc:1046] Stats:
Limit:                        92950528
InUse:                        58933248
MaxInUse:                     80629248
NumAllocs:                         305
MaxAllocSize:                 12269312
Reserved:                            0
PeakReserved:                        0
LargestFreeBlock:                    0

2021-07-28 12:06:04.710082: W tensorflow/core/common_runtime/bfc_allocator.cc:439] ******____******____******__________***************************************************_____________
2021-07-28 12:06:04.710166: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at random_op.cc:77 : Resource exhausted: OOM when allocating tensor with shape[3,3,512,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
Traceback (most recent call last):
  File "keras_test.py", line 8, in <module>
    pipeline = keras_ocr.pipeline.Pipeline()
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/keras_ocr/pipeline.py", line 19, in __init__
    detector = detection.Detector()
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/keras_ocr/detection.py", line 610, in __init__
    self.model = build_keras_model(weights_path=weights_path, backbone_name=backbone_name)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/keras_ocr/detection.py", line 318, in build_keras_model
    name='basenet.slice5.1')(s5)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 926, in __call__
    input_list)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1098, in _functional_construction_call
    self._maybe_build(inputs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2643, in _maybe_build
    self.build(input_shapes)  # pylint:disable=not-callable
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py", line 204, in build
    dtype=self.dtype)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 614, in add_weight
    caching_device=caching_device)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py", line 750, in _add_variable_with_custom_getter
    **kwargs_for_getter)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py", line 145, in make_variable
    shape=variable_shape if variable_shape else None)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 260, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 221, in _variable_v1_call
    shape=shape)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 199, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 2597, in default_variable_creator
    shape=shape)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 264, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1518, in __init__
    distribute_strategy=distribute_strategy)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py", line 1651, in _init_from_args
    initial_value() if init_from_fn else initial_value,
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/keras/initializers/initializers_v2.py", line 397, in __call__
    return super(VarianceScaling, self).__call__(shape, dtype=_get_dtype(dtype))
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/init_ops_v2.py", line 561, in __call__
    return self._random_generator.random_uniform(shape, -limit, limit, dtype)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/init_ops_v2.py", line 1044, in random_uniform
    shape=shape, minval=minval, maxval=maxval, dtype=dtype, seed=self.seed)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/random_ops.py", line 302, in random_uniform
    shape, dtype, seed=seed1, seed2=seed2)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/ops/gen_random_ops.py", line 726, in random_uniform
    _ops.raise_from_not_ok_status(e, name)
  File "/home/quadient/Documents/autolabelocr/venvocr/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 6843, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[3,3,512,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:RandomUniform]

Hi,

Based on the latest log, the app is running out of memory.

total_region_allocated_bytes_: 92950528 memory_limit_: 92950528 available bytes: 0 curr_region_allocation_bytes_: 185901056

Since Nano only has 4GiB memory, the support range is quite limited.

An alternative is to use CPU mode for inference instead.
With CPU mode, you can add some swap memory from the disk to enlarge the size of the memory.

However, CPU mode is expected to run much slower than the GPU version.
But the swap workaround is only available for CPU implementation.

Thanks.