TLT - Retrain trafficcamnet with customized data failed

Description

First of all, my own dataset only contains cars, no people, bike etc. Does this dataset work?

The specs I copied from from the othe topic and made adjustment accordingly.

The errors are like below,

2021-06-14 09:17:29,871 [INFO] tensorflow: Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/tmpndu68y5w/model.ckpt-193080
2021-06-14 09:17:30,369 [INFO] tensorflow: Restoring parameters from /tmp/tmpndu68y5w/model.ckpt-193080
Traceback (most recent call last):
File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py”, line 1365, in _do_call
return fn(*args)
File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py”, line 1350, in _run_fn
target_list, run_metadata)
File “/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/client/session.py”, line 1443, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: 2 root error(s) found.
(0) Not found: Key block_1b_bn_shortcut/beta/Adam not found in checkpoint
[[{{node save/RestoreV2}}]]
(1) Not found: Key block_1b_bn_shortcut/beta/Adam not found in checkpoint
[[{{node save/RestoreV2}}]]
[[save/RestoreV2/_167]]
0 successful operations.
0 derived errors ignored.

Can someone pinpoint where should I look into?

Thanks,
Kai

Environment

TensorRT Version:
GPU Type:
Nvidia Driver Version:
CUDA Version:
CUDNN Version:
Operating System + Version:
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Hi @1733208392,

This forum talks more about updates and issues related to TensorRT. We recommend to post your query on TLT forum to get better help.

Thank you.