Grayscale image and DetectNet

Hello, I’m trying to detect object with grayscale image.

I have camera, at day in color, (RGB), but at night, in grayscale.
So at day, DetectNet work good, ( with few false detections ) but, at night it give nothing.

@borelli.g92 try to apply sepia color filter. It look like to work. But I think it is better to train a model accordingly.
The goal is to convert, day and night images in grayscale. And train a custom model with grayscale images.

So I follow this tutorial.
I downloaded 5000 images with label “Person” with open_images_downloader.py

And I made few changes in train_ssd.py :

def train(loader, net, criterion, optimizer, device, debug_steps=100, epoch=-1):
    net.train(True)
    running_loss = 0.0
    running_regression_loss = 0.0
    running_classification_loss = 0.0
    images = []
    for i, data in enumerate(loader):
        images, boxes, labels = data
        # convert images to grayscale with torchvision tools 
        images = [torchvision.transforms.ToPILImage()(x) for x in images]
        images = [torchvision.transforms.Grayscale()(x) for x in images]
        # going back to RGB shape without color informations
        images = [(x.convert('RGB')) for x in images]
        # convert PIL Image to tensor
        images = [torchvision.transforms.ToTensor()(x) for x in images]
        # stack tensor because images is list of tensor 
        images = torch.stack(images)
        images = images.to(device)
        boxes = boxes.to(device)
        labels = labels.to(device)

I do the same operation to test method :

def test(loader, net, criterion, device):
    net.eval()
    running_loss = 0.0
    running_regression_loss = 0.0
    running_classification_loss = 0.0
    num = 0
    images = []
    for _, data in enumerate(loader):
        images, boxes, labels = data
        images = [torchvision.transforms.ToPILImage()(x) for x in images]
        images = [torchvision.transforms.Grayscale()(x) for x in images]
        images = [(x.convert('RGB')) for x in images]
        images = [torchvision.transforms.ToTensor()(x) for x in images]
        images = torch.stack(images)
        images = images.to(device)
        boxes = boxes.to(device)
        labels = labels.to(device)
        num += 1

So I launch the training. It seems to be learning, because at first Epoch loss is around 4.
After 71 Epochs loss is approx 3.67.

I have somes questions :

  • did it really learn ?
  • are my changes valid?
  • how many epochs will be needed ?
  • the model is based on SSD-MobileNet-V1, but with a good training will it be precise ? accurate ?
  • have you another solution to propose

I followed your previous advice, DeepStream work really fine, but not with grayscale images. It seems to be no way to add sepia filter or convert to grayscale inputs images ( to retrain PeopleNet model for DeepStream directly )

Thank you !!

Hi,

Color is important in distinguishing objects.
We recommends to use the original model for day, and re-retrain a grayscale model for night.
Please expect that the accuracy of grayscale model to be lower since it is more difficult to classify an object only with its shape.

In general, we verify a training job by a separated testing dataset.
Please preserve 10%-25% database that not being used for training.
And apply the testing periodically (ex. one per 10 epochs) to validate the training tasks.

Thanks.