2021-11-25 13:18:23.325521: I tensorflow/core/common_runtime/bfc_allocator.cc:1066] Stats:
Limit: 91222016
InUse: 61331456
MaxInUse: 74037248
NumAllocs: 365
MaxAllocSize: 8519680
Reserved: 0
PeakReserved: 0
LargestFreeBlock: 0
2021-11-25 13:18:23.325594: W tensorflow/core/common_runtime/bfc_allocator.cc:467] *********************_____****************************x_________________
2021-11-25 13:18:23.325789: W tensorflow/core/framework/op_kernel.cc:1767] OP_REQUIRES failed at random_op.cc:74 : Resource exhausted: OOM when allocating tensor with shape[3,3,512,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
Traceback (most recent call last):
File “/home/cdac/Desktop/Single-Multiple-Custom-Object-Detection-and-Tracking-master/object_tracker.py”, line 24, in
yolo = YoloV3(classes=len(class_names))
File “/home/cdac/Desktop/Single-Multiple-Custom-Object-Detection-and-Tracking-master/yolov3_tf2/models.py”, line 211, in YoloV3
x_36, x_61, x = Darknet(name=‘yolo_darknet’)(x)
File “/home/cdac/Desktop/Single-Multiple-Custom-Object-Detection-and-Tracking-master/yolov3_tf2/models.py”, line 78, in Darknet
x = DarknetBlock(x, 1024, 4)
File “/home/cdac/Desktop/Single-Multiple-Custom-Object-Detection-and-Tracking-master/yolov3_tf2/models.py”, line 65, in DarknetBlock
x = DarknetConv(x, filters, 3, strides=2)
File “/home/cdac/Desktop/Single-Multiple-Custom-Object-Detection-and-Tracking-master/yolov3_tf2/models.py”, line 49, in DarknetConv
use_bias=not batch_norm, kernel_regularizer=l2(0.0005))(x)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 970, in call
input_list)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 1108, in _functional_construction_call
inputs, input_masks, args, kwargs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 840, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 878, in _infer_output_signature
self._maybe_build(inputs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 2625, in _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/convolutional.py”, line 204, in build
dtype=self.dtype)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py”, line 655, in add_weight
caching_device=caching_device)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/tracking/base.py”, line 815, in _add_variable_with_custom_getter
**kwargs_for_getter)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer_utils.py”, line 139, in make_variable
shape=variable_shape if variable_shape else None)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py”, line 260, in call
return cls._variable_v1_call(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py”, line 221, in _variable_v1_call
shape=shape)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py”, line 199, in
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py”, line 2626, in default_variable_creator
shape=shape)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py”, line 264, in call
return super(VariableMetaclass, cls).call(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py”, line 1595, in init
distribute_strategy=distribute_strategy)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py”, line 1722, in _init_from_args
initial_value = initial_value()
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/initializers/initializers_v2.py”, line 523, in call
return self._random_generator.random_uniform(shape, -limit, limit, dtype)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/initializers/initializers_v2.py”, line 979, in random_uniform
shape=shape, minval=minval, maxval=maxval, dtype=dtype, seed=self.seed)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/dispatch.py”, line 206, in wrapper
return target(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/random_ops.py”, line 310, in random_uniform
shape, dtype, seed=seed1, seed2=seed2)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_random_ops.py”, line 721, in random_uniform
_ops.raise_from_not_ok_status(e, name)
File “/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py”, line 6897, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File “”, line 3, in raise_from
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[3,3,512,1024] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:RandomUniform]