AssertionError: Cannot find val tfrecords for fold 0for tfrecord: /home/tfrecords. Please check the validation fold number and retry

Got the following error when train with detectnet_v2

root@ca8af7905bbc:/home# tlt-train detectnet_v2 -k ‘MnA0c3ZhYjQ5MTR1Y25mcmJvc2R0YzYwdHU6ZTE3MmNiNGItMzhmOC00MjczLWE1NWQtZmQyNmI4NmFlYzYz’ -r ‘/home/result/’ -e ‘/home/specs/train_spec.txt’
Using TensorFlow backend.

[[4173,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
Host: ca8af7905bbc

Another transport will be used instead, although this may result in
lower performance.

NOTE: You can disable this warning by setting the MCA parameter
btl_base_warn_component_unused to 0.

2020-05-29 11:59:13,276 [INFO] iva.detectnet_v2.scripts.train: Loading experiment spec at /home/specs/train_spec.txt.
2020-05-29 11:59:13,278 [INFO] iva.detectnet_v2.spec_handler.spec_loader: Merging specification from /home//specs/train_spec.txt
Traceback (most recent call last):
File “/usr/local/bin/tlt-train-g1”, line 8, in
sys.exit(main())
File “./common/magnet_train.py”, line 47, in main
File “”, line 2, in main
File “./detectnet_v2/utilities/timer.py”, line 46, in wrapped_fn
File “./detectnet_v2/scripts/train.py”, line 667, in main
File “./detectnet_v2/scripts/train.py”, line 591, in run_experiment
File “./detectnet_v2/scripts/train.py”, line 457, in train_gridbox
File “./detectnet_v2/dataloader/build_dataloader.py”, line 109, in build_dataloader
File “./detectnet_v2/dataloader/default_dataloader.py”, line 84, in init
File “./detectnet_v2/dataloader/base_dataloader.py”, line 125, in get_data_sources
AssertionError: Cannot find val tfrecords for fold 0for tfrecord: /home/n/tfrecords. Please check the validation fold number and retry

There should be something missing for val dataset.
Please see Training Object Detection with pretrained resnet 18 , following Detectnet_V2 example

Hi,

While training i used -k ‘abc’. So where should i add this key ?

I got error

IOError: Invalid decryption. Unable to open file (File signature not found). The key used to load the model is incorrect.

Can you elaborate you new issue? Command and log are appreciated.

Issue solved by passing -k tlt_encode … Please add this in the PeopleNet documentation

I read that TLT will not work with the dataset which contains different resolutioned images. My dataset consist of small and very high resolution images . Now training on the image without resizing , Got 16% accuracy too. So how its working now with my dataset?

Suggest to resize your images to the same resolution.

But half of the images are in 4k resolution , and the ROI’s are very small . If i resized into small sizes then the roi features will be lost

For your case, you can consider the same sized cropping. To crop the 4k images with the same size.
This is also a way of data augmentation.