• Hardware AWS EC2 instance
• Network Type: Classification, efficientNetB1
• TAO Version 3.0-21.08
I’ve used the following command as always:
!tao classification evaluate -e $SPECS_DIR/classification_spec.cfg -k $KEY | tee “eval_output.txt”
And then these error appears (I put the full log):
Found 75995 images belonging to 133 classes.
2021-12-10 00:13:47,281 [INFO] main: Processing dataset (evaluation): /workspace/tao-experiments/data/split/TEST
Traceback (most recent call last):
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/evaluate.py”, line 228, in
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/utils.py”, line 494, in return_func
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/utils.py”, line 482, in return_func
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/evaluate.py”, line 224, in main
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/evaluate.py”, line 190, in run_evaluate
File “/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py”, line 91, in wrapper
return func(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/keras/engine/training.py”, line 1472, in evaluate_generator
verbose=verbose)
File “/usr/local/lib/python3.6/dist-packages/keras/engine/training_generator.py”, line 330, in evaluate_generator
generator_output = next(output_generator)
File “/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py”, line 709, in get
six.reraise(*sys.exc_info())
File “/usr/local/lib/python3.6/dist-packages/six.py”, line 696, in reraise
raise value
File “/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py”, line 685, in get
inputs = self.queue.get(block=True).get()
File “/usr/lib/python3.6/multiprocessing/pool.py”, line 644, in get
raise self._value
File “/usr/lib/python3.6/multiprocessing/pool.py”, line 119, in worker
result = (True, func(*args, **kwds))
File “/usr/local/lib/python3.6/dist-packages/keras/utils/data_utils.py”, line 626, in next_sample
return six.next(_SHARED_SEQUENCES[uid])
File “/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/iterator.py”, line 104, in next
return self.next(*args, **kwargs)
File “/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/iterator.py”, line 116, in next
return self._get_batches_of_transformed_samples(index_array)
File “/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/iterator.py”, line 230, in _get_batches_of_transformed_samples
interpolation=self.interpolation)
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/utils/preprocess_crop.py”, line 68, in load_and_crop_img
File “/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/utils.py”, line 114, in load_img
img = pil_image.open(io.BytesIO(f.read()))
File “/usr/local/lib/python3.6/dist-packages/PIL/Image.py”, line 2818, in open
raise IOError(“cannot identify image file %r” % (filename if filename else fp))
OSError: cannot identify image file <_io.BytesIO object at 0x7f52337cc888>
I’ve never had this problem before and anything has changed before last time I used it and it worked fine.
Any idea of what could be the cause of this error?