Train_ssd.py - Could not find image warning

dragos-g11 · July 13, 2024, 1:04pm

Hello guys,

I have this problem regarding the train_ssd.py, which is the

warning - could not find image yoto10383 - ignoring from dataset

error. I have been following this tutorial, for doing transfer learning on the SSD-Mobilenet model.

As I downloaded my data from the internet, along with .xml files and images, I did not use the camera capture, and so I jumped directly to training the model.

I have the labels.txt file, the Annotations folder, the ImageSets folder, which contains Main folder with train.txt file, all inside my images folder. They weren’t inside the images folder prior to this, but the errors led me to adding them there, and now this is the last error I am getting:

warning - could not find image yoto10383 - ignoring from dataset
warning - could not find image yoto10412 - ignoring from dataset
warning - could not find image yoto10441 - ignoring from dataset
warning - could not find image yoto10499 - ignoring from dataset
warning - could not find image yoto10557 - ignoring from dataset
warning - could not find image yoto10586 - ignoring from dataset
warning - could not find image yoto10702 - ignoring from dataset
warning - could not find image yoto10731 - ignoring from dataset
warning - could not find image yoto10760 - ignoring from dataset
warning - could not find image yoto10789 - ignoring from dataset
warning - could not find image yoto10818 - ignoring from dataset
warning - could not find image yoto10847 - ignoring from dataset
warning - could not find image yoto10876 - ignoring from dataset
warning - could not find image yoto10905 - ignoring from dataset
warning - could not find image yoto10934 - ignoring from dataset
warning - could not find image yoto10963 - ignoring from dataset
warning - could not find image yoto10992 - ignoring from dataset
warning - could not find image yoto11021 - ignoring from dataset
warning - could not find image yoto11050 - ignoring from dataset
warning - could not find image yoto11079 - ignoring from dataset
warning - could not find image yoto11108 - ignoring from dataset
warning - could not find image yoto11485 - ignoring from dataset
warning - could not find image yoto11920 - ignoring from dataset
2024-07-12 16:34:48 - VOC Labels read from file: (‘BACKGROUND’, ‘’, ‘drone’)
2024-07-12 16:34:48 - Stored labels into file data/images/labels.txt.
2024-07-12 16:34:48 - Train dataset size: 0
Traceback (most recent call last):
File “/home/jetson-orin/jetson-inference/python/training/detection/ssd/pytorch-ssd/train_ssd.py”, line 284, in
train_loader = DataLoader(train_dataset, args.batch_size,
File “/usr/local/lib/python3.10/dist-packages/torch/utils/data/dataloader.py”, line 349, in init
sampler = RandomSampler(dataset, generator=generator) # type: ignore[arg-type]
File “/usr/local/lib/python3.10/dist-packages/torch/utils/data/sampler.py”, line 140, in init
raise ValueError(f"num_samples should be a positive integer value, but got num_samples={self.num_samples}")
ValueError: num_samples should be a positive integer value, but got num_samples=0

I got all the paths right, and so I don’t understand why the train_ssd.py code does not see my images.
This is the command:

$ python3 train_ssd.py --dataset-type=voc --data=/home/jetson-orin/jetson-inference/python/training/detection/ssd/pytorch-ssd/images --model-dir=/home/jetson-orin/jetson-inference/data-models

Could it be that I am missing a folder or .txt file? If so, what should be inside them, and where should I place them? All the tutorials regarding re-training this model involve using the camera capture, which I am not, so maybe this has something to do with it.

My .xml files (in VOC format) point correctly to at least 1 of the images.

AastaLLL · July 15, 2024, 4:16am

Hi,

Which dataset do you use? Does it save in Pascal VOC format?
Please find below comment to check if your dataset met the requirement:

Thanks.

dragos-g11 · July 15, 2024, 7:15am

This is the dataset that I am using.
It doesn’t come in Pascal VOC format, but I managed to transform it into Pascal VOC using Python.
If it helps, I can share the type of format my XML file has.

I created a JPEGImages folder inside my dataset folder and added all the images inside of it, and now it works. It threw me another error related to a blank test.txt file, but I copy pasted the data from trainval.txt to the test.txt file and now it is performing the training.

Except for trainval.txt and test.txt which have the same values, my train.txt and val.txt are empty. Not sure what I should write in each.

Also, how does the algorithm provide validation images, if there hasn’t been any mention or path to them?

dusty_nv · July 15, 2024, 11:38pm

@dragos-g11 if you haven’t already, I would recommend downloading the Pascal VOC dataset to inspect it, and make sure that your conversion adheres to the same structure. Your images are all expected to be under JPEGImages/, and the XML files under Annotations/

The image should be referred to in the XML like <filename>yoto10383.jpg</filename> (presuming that is indeed the naming scheme of your images)

And then under ImageSets/Main in train.txt and trainval.txt, it should refer to the XML filenames (without .xml extension). I would also recommend testing the training on a limited subset of the data initially to iron out any issues with the training, rather than waiting for it to train over several thousand images just to discover an issue (such as there only being one drone class biasing the model)

dragos-g11 · July 16, 2024, 4:49am

@dusty_nv Thank you for the clarifications and advice. I managed to get it working properly now.
The solution was to add all my images inside of JPEGImages folder. Seems to be mandatory to have that folder.

dragos-g11 · July 16, 2024, 5:22am

One more question. How does the validation image set come into play here? I am used to validation being part of the training process, but I don’t see how it fits into training here.

Or is it just for testing the model afterwards?

dusty_nv · July 16, 2024, 6:01pm

@dragos-g11 ok glad you got it working - if you prefer, you are welcome to edit the VOCDataset code here with changes to the file structure for your own datasets:

github.com

dusty-nv/pytorch-ssd/blob/master/vision/datasets/voc_dataset.py

#!/usr/bin/env python3
import os
import logging

import torch
import numpy as np
import xml.etree.ElementTree as ET

from PIL import Image


class VOCDataset(torch.utils.data.Dataset):
    """
    Object detection dataset for Pascal VOC (http://host.robots.ox.ac.uk/pascal/VOC/)
    """
    def __init__(self, root, transform=None, target_transform=None, is_test=False, keep_difficult=False, label_file=None):
        """
        Dataset for VOC data.
        
        Parameters:

This file has been truncated. show original

When train_ssd.py is using VOCDataset, it uses the combined train+val sets (trainval.txt) for training, and then the test set (test.txt) at the end of each epoch for accuracy validation. I can’t recall if that misalignment is an artifact of how the original VOC dataset was organized, or if it’s just a peculiarity inherited from the upstream repo. Alas you are welcome to change the code around so that it is canonically correct and doing proper train/val/test splits. I suspect that since these fine-tuning datasets used by these scripts are smaller than normal, that was to leverage as much training data as possible while still providing some modicum of accuracy eval.

system · August 13, 2024, 7:42am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Train_ssd.py indices error Jetson Nano jetson-inference	12	1719	December 15, 2021
Train_ssd.py dosen't work with pascal voc dataset Jetson Nano ai-training	5	1123	February 9, 2022
Jetson nano start the Docker an error occurred while training your detection model ：Segmentation fault (core dumped) Jetson Nano jetson-inference	7	1233	April 21, 2022
Train_ssd.py error - Training Object Detection Models Jetson Nano ai-training	10	1413	October 6, 2022
Problems with train_ssd.py Jetson Nano	2	1018	October 14, 2021
Having trouble setting up pytorch code for training ssd-mobilenet Jetson Nano jetson-inference , python , training	7	1877	October 15, 2021
Transfer learning (train_ssd.py) with combined dataset: "Train dataset size: 0"? Jetson Xavier NX ai-training	3	712	October 18, 2021
Object Detection could not parse xml file Jetson Nano jetson-inference	3	653	November 7, 2022
Error in python train_ssd.py Jetson Nano ai-training	7	820	January 18, 2022
Train_ssd.py -- Not found under jetson-inference/python/training/detection/ssd Jetson Nano ai-training	2	1428	October 18, 2021

Train_ssd.py - Could not find image warning

Related topics