Please provide complete information as applicable to your setup.
• Hardware (T4)
• DeepStream Version 6.3
• TensorRT Version 8.5.3.1
• Issue Type( questions, bugs)
• Cant open the pretrain efficient net model for training efficientdet object detection. I have only one class named person for detection and i need to train them in efficientdet_tf1.
• For multi-GPU, change --gpus based on your machine.
2023-12-28 04:16:33,866 [INFO] root: Registry: [‘nvcr.io’]
2023-12-28 04:16:33,897 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:4.0.0-tf1.15.5
2023-12-28 04:16:33,956 [WARNING] tlt.components.docker_handler.docker_handler:
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the “user”:“UID:GID” in the
DockerOptions portion of the “/home/vanorhq/.tao_mounts.json” file. You can obtain your
users UID and GID by using the “id -u” and “id -g” commands on the
terminal.
Using TensorFlow backend.
2023-12-28 12:16:34.585451: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
/usr/local/lib/python3.6/dist-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.26.5) or chardet (3.0.4) doesn’t match a supported version!
RequestsDependencyWarning)
2023-12-28 12:16:36.692618: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2023-12-28 12:16:36.871066: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2023-12-28 12:16:37.219327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libnvinfer.so.8
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
/usr/local/lib/python3.6/dist-packages/requests/init.py:91: RequestsDependencyWarning: urllib3 (1.26.5) or chardet (3.0.4) doesn’t match a supported version!
RequestsDependencyWarning)
Using TensorFlow backend.
Loading experiment spec at %s. /workspace/tao-experiments/efficientdet/specs/efficientdet_d0_train.txt
2023-12-28 12:16:39,815 [INFO] iva.efficientdet.utils.spec_loader: Merging specification from /workspace/tao-experiments/efficientdet/specs/efficientdet_d0_train.txt
2023-12-28 12:16:39,817 [INFO] iva.common.logging.logging: Log file already exists at /workspace/tao-experiments/efficientdet/experiment_dir_unpruned/status.json
2023-12-28 12:16:39,817 [INFO] root: Starting EfficientDet training.
2023-12-28 12:16:39,817 [INFO] root: [train] AMP is activated - Experiment Feature
2023-12-28 12:16:39,818 [INFO] root: Create EncryptCheckpointSaverHook.
2023-12-28 12:16:39,818 [INFO] root: Loading pretrained model…
2023-12-28 12:16:39,818 [INFO] root: Unable to open file (unable to open file: name = ‘/home/vanorhq/tao/cv_samples_v1.4.1/efficientdet/pretrained_efficientdet/efficientnet_b5.hdf5’, errno = 2, error message = ‘No such file or directory’, flags = 0, o_flags = 0)
Traceback (most recent call last):
File “</usr/local/lib/python3.6/dist-packages/iva/efficientdet/scripts/train.py>”, line 3, in
File “”, line 141, in
File “”, line 84, in main
File “”, line 71, in main
File “”, line 91, in run_executer
File “”, line 334, in train_and_eval
File “”, line 223, in get_training_hooks
File “”, line 247, in load_pretrained_model
File “”, line 34, in load_keras_model
File “/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py”, line 417, in load_model
f = h5dict(filepath, ‘r’)
File “/usr/local/lib/python3.6/dist-packages/keras/utils/io_utils.py”, line 186, in init
self.data = h5py.File(path, mode=mode)
File “/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py”, line 312, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File “/usr/local/lib/python3.6/dist-packages/h5py/_hl/files.py”, line 142, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File “h5py/_objects.pyx”, line 54, in h5py._objects.with_phil.wrapper
File “h5py/_objects.pyx”, line 55, in h5py._objects.with_phil.wrapper
File “h5py/h5f.pyx”, line 78, in h5py.h5f.open
OSError: Unable to open file (unable to open file: name = ‘/home/vanorhq/tao/cv_samples_v1.4.1/efficientdet/pretrained_efficientdet/efficientnet_b5.hdf5’, errno = 2, error message = ‘No such file or directory’, flags = 0, o_flags = 0)
Execution status: FAIL
2023-12-28 04:16:41,827 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.
efficientdet_d0_train.txt (1.1 KB)
