Problems With Jetson Orin Nano Inference classication when using more that one imported camera pictures

Currently I’m working on the Nvidia Jetson inference classification using the software from Dustin’s S3E3 on YouTube. All original pictures were taken with a GoPro Hero 11. The project currently has over 11,000 pictures for 15 classes. I have another 924 pictures not used in the training phased were input for imaginenet. The result was only 2 pictures out of 924 were misidentified. Yes the sounds great until I gave it pictures from my Iphone. Those picture are misidentified better than 80% of the time. I have tried resnets 18, 34, 50 and 101. Resnet34 seems to be a little better but not a lot. Other info I’m using Jetson Orin Nano JetPack 5.1.1 and no docker. The GoPro pictures were reduced to to 18% of the orginal size using imagemagik ( 5568 by 4872 to 1002 by 877 ) . The Iphone is 2016 by 1518.

Hi @wpriddy, what was the train.py command that you used to train your model, and what accuracy did the training achieve?

If the issue is between switching cameras, then you may want to add pictures from your iPhone or other cameras to your training dataset, or crop/rescale them so they are of a similar aspect ratio.

Hello Dustin;

The train.py is still the orignal code but uses a bash script to call it. Also just added -a , --batch-size and --workers to speed things up. I redirect output to file to process in to a graph. Also the Gopro picture were reduced to 18% by ImageMagick during the loading process.

command line is /.train.sh project_name epochs network

real example ./train.sh furry 35 resent18

EPOCHS=$2
NETWORK=$3
MODELS=$EXT/ext/models/$1
XDATA=$EXT/ext/data/$1
program=~/jetson-inference/python/training/classification/train.py
python3 $program --model-dir=$MODELS $XDATA --batch-size=10 --workers=4 --epochs=$EPOCHS -a=$NETWORK >$1_train.log

below is the accuracy. This is using just GoPro pictures. The accuracy remain around 98% regardless of network ie resnets 18, 34, 50 and 101.
=> Epoch 34

  • Train Loss 4.9754e-02
  • Train Accuracy 98.1377
  • Val Loss 4.9861e-04
  • Val Accuracy 100.0000

I do go a farther step using imaginenet. I input other pictures that were not used for the train/test/val. The picture class reported by imainenet must match with the class embedded in the file name. Below is a final error check with GoPro only pictures.

greyhare@Orino-Nano:~/scripts$ ./errors.sh furry resnet18
Network = resnet18
******** input class= Arrow_the_Thylacine output class= Luna
******** input file= Arrow_the_Thylacine-G0127455.jpg output file = 61.jpg
******** conf= 47.46094% number of errors = 1

errors = 1
total images = 924

Added Iphone pictures for the Greyhare class.
Now I will rerun imaginenet and errors adding the Iphone pictures as a final test.
Network = resnet18
******** input class= Arrow_the_Thylacine output class= Luna
******** input file= Arrow_the_Thylacine-G0127455.jpg output file = 61.jpg
******** conf= 47.46094% number of errors = 1

******** input class= Greyhare output class= Arrow_the_Thylacine
******** input file= Greyhare-IMG_5077.jpg output file = 393.jpg
******** conf= 55.61523% number of errors = 2

******** input class= Greyhare output class= Arrow_the_Thylacine
******** input file= Greyhare-IMG_5080.jpg output file = 396.jpg
******** conf= 85.69336% number of errors = 3

******** input class= Greyhare output class= Furzy_The_White_Tiger
******** input file= Greyhare-IMG_5081.jpg output file = 397.jpg
******** conf= 98.29102% number of errors = 4

******** input class= Greyhare output class= Fox_alexis
******** input file= Greyhare-IMG_5082.jpg output file = 398.jpg
******** conf= 56.10352% number of errors = 5

******** input class= Greyhare output class= Bubble_Gum
******** input file= Greyhare-IMG_5083.jpg output file = 399.jpg
******** conf= 45.01953% number of errors = 6

******** input class= Greyhare output class= Bubble_Gum
******** input file= Greyhare-IMG_5084.jpg output file = 400.jpg
******** conf= 46.48438% number of errors = 7

******** input class= Greyhare output class= Arrow_the_Thylacine
******** input file= Greyhare-IMG_5085.jpg output file = 401.jpg
******** conf= 62.25586% number of errors = 8

errors = 8
total images = 933

added 9 iphone picture and got 7 errors from adding them.

Now I will rerun imaginenet and errors adding the Iphone pictures as a final test.

I will try using ImageMagicK to adjust them to be similar pixel size. Currently the project has 11,000 pictures.

Adding pictures from Iphone or other camera are not possible due to the limit time I have access to the subjects to take the 600 pictures.

@wpriddy does the inferencing stage have higher accuracy if you run it on pictures from your GoPro? i.e. does the decreased accuracy only occur when using pictures from your iPhone? If so, you can try pre-processing them but to make your model more robust you may want/need to add photos to your dataset from other cameras in the future (if the inferencing is done with a different camera)

Thanks for the reply Dusty_nv. I tried resizing image size for final test with 35 different sizes. My finding are the image size thru imagenet input rarely changed the accuracy. I went back to my test project plush because the objects are at the house. Unlike my other project where I rarely have access to the object. I took another 500 pictures with a Canon 60D at the lowest resolution added them to train, test, val and final test today. The result were quite good. See below.

x y1 y2

0 41.4148 83.2769
1 83.2797 97.0210
2 89.5820 99.8646
3 92.0533 99.3907
4 93.4865 99.8646
5 93.3670 100.000
6 94.5521 99.7969
7 94.6532 99.9323
8 94.5430 99.8646
9 95.2044 99.8646
10 95.4341 99.8646
11 95.5811 100.000
12 95.6362 99.7292
13 95.4065 100.000
14 95.7373 99.5938
15 95.6821 99.9323
16 96.0312 98.7813
17 96.1599 100.000
18 95.3973 100.000
19 96.2425 99.9323
20 95.5811 100.000
21 96.5916 99.9323
22 95.7832 99.6615
23 96.1047 100.000
24 95.6913 100.000
25 95.9210 97.9012
26 96.4722 99.5938
27 95.5719 99.7969
28 96.3895 99.8646
29 96.3803 99.9323
30 97.7676 100.000
31 97.9789 100.000
32 97.9972 100.000
33 98.1350 100.000
34 97.9329 100.000

The final test was using imagenet out of 1848 images 2 failed. Below is the details.

Network = resnet18
******** input class= bunny_chocolate output class= bunny_kung_fu
******** input file= bunny_chocolate-G0334270.jpg output file = 175.jpg
******** conf= 52.85747% number of errors = 1

******** input class= cow_bw_singing output class= panda_singing
******** input file= cow_bw_singing-IMG_5183.jpg output file = 1047.jpg
******** conf= 76.52087% number of errors = 2

errors = 2
total images = 1848

I will try some more get Iphone picture. Problem is the Iphone is much slower for gathering lots of images. Other info about the plush project is 19 classes and over 28,000 images that I took… The other project will grow to hundreds of classes and over 500,000. images.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.