Please provide the following information when requesting support.
• Hardware (Desktop/rtx 2070)
• Network Type (fpenet)
• TLT Version (nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5)
• Training spec file(—
class_name: FpeNetTrainer
checkpoint_dir: /workspace/416x416_pictures/outputs/checkpoints
checkpoint_n_epoch: 1
dataloader:
class_name: FpeNetDataloader
augmentation_info:
augmentation_resize_probability: 0.5
augmentation_resize_scale: 1.6
enable_occlusion_augmentation: false
enable_online_augmentation: true
enable_resize_augmentation: true
gamma_augmentation:
gamma_max: 1.6
gamma_min: 0.6
gamma_probability: 0.1
gamma_type: uniform
modulus_spatial_augmentation:
hflip_probability: 0.0
rotate_rad_max: 0.35
translate_max_x: 10
translate_max_y: 10
zoom_max: 1.2
zoom_min: 0.8
patch_probability: 0.5
size_to_image_ratio: 0.5
mask_augmentation_patch: true
batch_size: 64
dataset_info:
image_extension: png
no_occlusion_masking_sets: none
root_path: /workspace/416x416_pictures/
test_file_name: data.tfrecords
tfrecord_folder_name: FpeTfRecords
tfrecords_directory_path: /workspace/416x416_pictures/tfrecords
tfrecords_set_id_train: train
tfrecords_set_id_val: validate
tfrecord_file_name: data.tfrecords
use_extra_dataset: false
image_info:
image:
channel: 1
height: 80
width: 80
kpiset_info:
tfrecords_set_id_kpi: s578-usercalibration-incar-1
num_keypoints: 1
enable_visualization: true
hooks: null
infrequent_summary_every_n_steps: 0
log_every_n_secs: 10
loss:
class_name: FpeLoss
kpts_coeff: 0.01
loss_type: square_euclidean
mask_occ: true
weights_dict: null
elt_loss_info:
elt_alpha: 0.5
enable_elt_loss: true
modulus_spatial_augmentation:
hflip_probability: 0.0
rotate_rad_max: 0.35
translate_max_x: 10
translate_max_y: 10
zoom_max: 1.2
zoom_min: 0.8
model:
class_name: FpeNetBaseModel
model_parameters:
beta: 0.01
dropout_rate: 0.5
freeze_Convlayer: null
regularizer_type: l2
regularizer_weight: 1.0e-05
train_fpe_model: true
type: FpeNet_public
use_less_face_layers: false
use_upsampling_layer: false
visualization_parameters: null
num_epoch: 50
num_keypoints: 1
optimizer:
class_name: AdamOptimizer
beta1: 0.9
beta2: 0.999
epsilon: 1.0e-08
learning_rate_schedule:
class_name: SoftstartAnnealingLearningRateSchedule
annealing: 0.5
base_learning_rate: 0.0005
last_step: 1000000
min_learning_rate: 1.0e-07
soft_start: 0.3
random_seed: 35
visualize_num_images: 3)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
LOG:root@3c1de3eceb20:/workspace/416x416_pictures# fpenet train -e /workspace/416x416_pictures/fpenet_config.yaml
2024-12-07 21:52:29.288695: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
WARNING:tensorflow:TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:30,352 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:30,385 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
WARNING:tensorflow:TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:30,388 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:31,417 [TAO Toolkit] [WARNING] tensorflow 40: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
Using TensorFlow backend.
2024-12-07 21:52:32,455 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:32,489 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:32,492 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
2024-12-07 21:52:32,911 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py:110: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.
2024-12-07 21:52:32,911 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py:110: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py:120: YAMLLoadWarning: calling yaml.load() without Loader=… is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
spec = yaml.load(yaml_file.read())
/workspace/416x416_pictures/outputs/checkpoints
2024-12-07 21:52:33,249 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.common.logging.logging 197: Log file already exists at /workspace/416x416_pictures/outputs/checkpoints/status.json
2024-12-07 21:52:33,252 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/dataloader/fpenet_dataloader.py:262: The name tf.FixedLenFeature is deprecated. Please use tf.io.FixedLenFeature instead.
2024-12-07 21:52:33,256 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:153: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.
/workspace/416x416_pictures/tfrecords/train/FpeTfRecords/data.tfrecords: 2521
Total Samples: 2521
2024-12-07 21:52:33,333 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/tensorflow_core/python/autograph/converters/directives.py:119: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.
2024-12-07 21:52:37,567 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/dataloader/fpenet_dataloader.py:575: The name tf.matrix_inverse is deprecated. Please use tf.linalg.inv instead.
2024-12-07 21:52:37,597 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,601 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,603 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/core/augment/random_gamma.py:87: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.
2024-12-07 21:52:37,610 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,711 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,715 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,723 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,825 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,829 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,837 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,938 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,942 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:37,950 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,051 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,056 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,063 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,166 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,170 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,178 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,280 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,284 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,294 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,461 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,465 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,473 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,576 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,580 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,588 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,691 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,695 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,703 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,805 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,810 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,818 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,922 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,926 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:38,934 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,037 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,042 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,049 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,152 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,156 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,164 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,267 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,271 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,279 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,382 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,386 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,394 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,496 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,500 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,508 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,610 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,614 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,623 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,725 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,729 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,736 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,839 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,844 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,851 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,955 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,959 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:39,967 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,069 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,073 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,081 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,183 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,187 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,195 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,296 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,300 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,309 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,412 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,416 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,424 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,609 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,613 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,621 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,725 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,729 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,737 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,840 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,845 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,853 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,957 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,961 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:40,969 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,073 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,077 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,085 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,189 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,193 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,201 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,305 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,309 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,317 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,419 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,424 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,431 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,535 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,539 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,547 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,650 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,654 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,662 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,766 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,770 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,778 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,882 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,886 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,894 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:41,996 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,000 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,008 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,111 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,115 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,123 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,226 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,230 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,238 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,353 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,358 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,367 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,494 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,499 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,509 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,634 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,638 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,646 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,751 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,755 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,763 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,867 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,872 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,879 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,991 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:42,995 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,003 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,110 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,115 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,122 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,326 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,330 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,338 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,443 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,447 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,455 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,562 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,566 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,574 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,680 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,684 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,692 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,795 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,799 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,806 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,910 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,914 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:43,922 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,024 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,028 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,036 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,138 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,142 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,150 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,253 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,257 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,265 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,369 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,373 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,380 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,483 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,487 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,495 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,598 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,602 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,609 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,712 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,716 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,723 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,828 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,832 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,840 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,944 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,948 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:44,955 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,059 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,063 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,071 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,173 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,177 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,185 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,241 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/visualization/fpenet_visualization.py:74: The name tf.summary.image is deprecated. Please use tf.compat.v1.summary.image instead.
2024-12-07 21:52:45,243 [TAO Toolkit] [INFO] root 2082: model type is: FpeNet_public
2024-12-07 21:52:45,257 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/third_party/keras/tensorflow_backend.py:195: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.
2024-12-07 21:52:45,496 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.fpenet.models.fpenet_basemodel 166: This model will be trained from scratch.
2024-12-07 21:52:45,608 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,612 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,622 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,626 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,637 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,641 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,652 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,656 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,666 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,670 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,680 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,684 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,694 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,698 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,709 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,713 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,723 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,727 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,738 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,742 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,752 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,756 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,767 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,771 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,782 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,786 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,796 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,800 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,810 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,815 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,825 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,829 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,839 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,843 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,854 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,858 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,868 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,872 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,882 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,886 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,897 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,901 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,911 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,915 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,926 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,930 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,940 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,944 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,955 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,959 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,969 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,973 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,984 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,988 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:45,998 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,002 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,013 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,017 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,027 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,031 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,042 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,046 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,056 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,060 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,071 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,075 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,085 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,089 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,100 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,104 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,114 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,119 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,129 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,133 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,143 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,147 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,157 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,162 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,172 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,176 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,186 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,190 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,201 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,205 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,215 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,219 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,229 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,234 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,244 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,248 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,258 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,263 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,273 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,277 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,288 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,292 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,302 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,306 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,317 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,321 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,331 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,335 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,346 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,350 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,360 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,364 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,375 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,379 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,389 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,393 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,404 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,408 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,418 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,422 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,433 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,437 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,558 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,562 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,572 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,576 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,586 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,591 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,601 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,605 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,615 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,619 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,629 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:46,634 [TAO Toolkit] [WARNING] tensorflow 1776: The operation tf.image.convert_image_dtype will be skipped since the input and output dtypes are identical.
2024-12-07 21:52:47,870 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.fpenet.trainers.fpenet_trainer 217: Evaluation model file path wasn’t provided. Getting the latest checkpoint in /workspace/416x416_pictures/outputs/checkpoints
Checkpoint results dir /workspace/416x416_pictures/outputs/checkpoints
2024-12-07 21:52:47,871 [TAO Toolkit] [INFO] nvidia_tao_tf1.cv.fpenet.trainers.fpenet_trainer 227: Evaluating using the model at None
/workspace/416x416_pictures/tfrecords/validate/FpeTfRecords/data.tfrecords: 348
Total Samples: 348
2024-12-07 21:52:51,305 [TAO Toolkit] [INFO] main 157: Build trainer finished. Starting training…
2024-12-07 21:52:51,305 [TAO Toolkit] [INFO] root 2102: Starting fpenet training.
2024-12-07 21:52:51,314 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/trainers/fpenet_trainer.py:290: The name tf.train.get_global_step is deprecated. Please use tf.compat.v1.train.get_global_step instead.
2024-12-07 21:52:51,316 [TAO Toolkit] [INFO] tensorflow 541: Create CheckpointSaverHook.
Checkpoint results dir /workspace/416x416_pictures/outputs/checkpoints
2024-12-07 21:52:58,877 [TAO Toolkit] [INFO] tensorflow 240: Graph was finalized.
2024-12-07 21:53:01,160 [TAO Toolkit] [INFO] tensorflow 500: Running local_init_op.
2024-12-07 21:53:05,009 [TAO Toolkit] [INFO] tensorflow 502: Done running local_init_op.
2024-12-07 21:53:05,732 [TAO Toolkit] [WARNING] tensorflow 137: From /usr/local/lib/python3.8/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.
2024-12-07 21:53:22,658 [TAO Toolkit] [INFO] tensorflow 81: Saving checkpoints for step-0.
2024-12-07 21:53:52,353 [TAO Toolkit] [INFO] root 2102: 2 root error(s) found.
(0) Invalid argument: Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
(1) Invalid argument: Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
[[RegexFullMatch_31/_1160]]
0 successful operations.
0 derived errors ignored.
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1365, in _do_call
return fn(*args)
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1349, in _run_fn
return self._call_tf_sessionrun(options, feed_dict, fetch_list,
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1441, in _call_tf_sessionrun
return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: {{function_node __inference_Dataset_map__map_func_set_random_wrapper_45018}} Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
(1) Invalid argument: {{function_node __inference_Dataset_map__map_func_set_random_wrapper_45018}} Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
[[RegexFullMatch_31/_1160]]
0 successful operations.
0 derived errors ignored.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py”, line 195, in
raise e
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py”, line 179, in
main()
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/scripts/train.py”, line 169, in main
trainer.train()
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/trainers/fpenet_trainer.py”, line 342, in train
self.run_training_loop(
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/fpenet/trainers/fpenet_trainer.py”, line 390, in run_training_loop
sess.run(train_op)
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/monitored_session.py”, line 750, in run
return self._sess.run(
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/monitored_session.py”, line 1360, in run
raise six.reraise(*original_exc_info)
File “/usr/local/lib/python3.8/dist-packages/six.py”, line 719, in reraise
raise value
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/monitored_session.py”, line 1345, in run
return self._sess.run(*args, **kwargs)
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/monitored_session.py”, line 1413, in run
outputs = _WrappedSession.run(
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/monitored_session.py”, line 1176, in run
return self._sess.run(*args, **kwargs)
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 955, in run
result = self._run(None, fetches, feed_dict, options_ptr,
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1179, in _run
results = self._do_run(handle, final_targets, final_fetches,
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1358, in _do_run
return self._do_call(_run_fn, feeds, fetches, targets, options,
File “/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/client/session.py”, line 1384, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
(1) Invalid argument: Feature: train/facebbx_h (data type: int64) is required but could not be found.
[[{{node ParseExampleProto/ParseSingleExample/ParseSingleExample}}]]
[[IteratorGetNext_1]]
[[RegexFullMatch_31/_1160]]
0 successful operations.
0 derived errors ignored.
Execution status: FAIL
on 416x416 images I havea annotated “lim” bboxes and “rfx” keypoints. The rfx are points in lim … (it is analog to face=>lim ; facelandmarks=> rfx… but there is just one point as landmark) Then I cropped the boboxes and resized them to 80x80. then I created a coco json ONLY for Keypoints. Then I converted the cocoson to TFRecords… Fpenet Training on Tao docker asks for " train/facebbx" Can we not train the landmarks on cropped images (analog to cropped face images)… But on deepstream, face works as pgie, landmarks as sgie, and they are seperate models, arnt they? Can I not train one model just for one keypoint on fpenet?
