Inference bug using --stats while running segnet.py when no class is detected

[nishantshrivastav23]

Hello @dusty_nv ,

There seems to be an issue in segnet.py, when none of the classes are detected on inference i.e. nothing other than background. We used this image (shown below) as input to fcn-resnet18-mhp-640*360 pre-trained segmentation model.

An the resultant mask was this along with the --stats output

grid size:   20x12
num classes: 21
-----------------------------------------
 ID  class name        count     %
-----------------------------------------
  0  background           0   0.000000
  1  hat/helmet/headwear   0   0.000000
  2  face                 0   0.000000
  3  hair                 0   0.000000
  4  arm                  0   0.000000
  5  hand                 0   0.000000
  6  shirt                0   0.000000
  7  jacket/coat          0   0.000000
  8  dress/robe           0   0.000000
  9  bikini/bra           0   0.000000
 10  torso_skin         240   1.000000
 11  pants                0   0.000000
 12  shorts               0   0.000000
 13  socks/stockings      0   0.000000
 14  shoe/boot            0   0.000000
 15  leg                  0   0.000000
 16  foot                 0   0.000000
 17  backpack/purse/bag   0   0.000000
 18  sunglasses/eyewear   0   0.000000
 19  other_accessory      0   0.000000
 20  other_item           0   0.000000

As you can see, background pixel count is 0%, and torso_skin shows 100% pixel count. Can you please let us know why this is happening? Ideally we expect the background pixel count to be 100% when no other classes are detected.

Alternatively, it would be great if you can share a way to infer that no classes are detected using segnet.py in the given input.

[dusty_nv]

Hi @nishantshrivastav23, can you send me the original input image that you used when running segnet.py so I can take a look? Or is the image you processed simply an all-black image? Thanks.

[nishantshrivastav23]

Hi @dusty_nv ,
We used a simple all-black image for the fcn-resnet18-mhp-640*360 pre-trained model. Actually, we did this to replicate the situation we faced while running inference on our custom trained model with classes - Person and Background

The input to our custom trained model when Person was detected:

The output mask:

The --stats output:

grid size: 40x40
num classes: 2
-----------------------------------------
ID class name count %
-----------------------------------------
0 background 1382 0.863750
1 person 218 0.136250

The input to our custom trained model when Person was not detected:

The output mask:

The --stats output:

grid size: 40x40
num classes: 2
-----------------------------------------
ID class name count %
-----------------------------------------
0 background 0 0.000000
1 person 1600 1.000000

As you can see, in the case where Person was not detected, we expected the background count to be 100% and Person count to be 0%. However, it is exactly the opposite, that is happening.

[dusty_nv]

@nishantshrivastav23

As you can see, in the case where Person was not detected, we expected the background count to be 100% and Person count to be 0%. However, it is exactly the opposite, that is happening.

Hmm, if that were actually the case, I would expect to see the mask image covered in green (since green is your color for person). Instead, I suspect some issue or bug in the class ID mask statistics. Can you log/dump the class ID mask of the image with No Person Detected to confirm which class ID’s are printed?

[dusty_nv]

OK, so I tried an all-black image with the fcn-resnet18-mhp model, and reproduced the same results:

grid size:   16x10
num classes: 21
-----------------------------------------
 ID  class name        count     %
-----------------------------------------
  0  background           0   0.000000
  1  hat/helmet/headwear   0   0.000000
  2  face                 0   0.000000
  3  hair                 0   0.000000
  4  arm                  0   0.000000
  5  hand                 0   0.000000
  6  shirt                0   0.000000
  7  jacket/coat          0   0.000000
  8  dress/robe           0   0.000000
  9  bikini/bra           0   0.000000
 10  torso_skin         160   1.000000
 11  pants                0   0.000000
 12  shorts               0   0.000000
 13  socks/stockings      0   0.000000
 14  shoe/boot            0   0.000000
 15  leg                  0   0.000000
 16  foot                 0   0.000000
 17  backpack/purse/bag   0   0.000000
 18  sunglasses/eyewear   0   0.000000
 19  other_accessory      0   0.000000
 20  other_item           0   0.000000

However, when I dumped the class ID mask, it was all zeros (as expected). So I don’t know where the stats is getting class 10 from - it must be some bug in the usage of np.histogram() (not sure exactly what yet). Suffice it to say, the core segNet functions that make the masks and overlays are functioning correctly though.

[dusty_nv]

OK, found the issue and just fixed it in commit f2dae7. The problem was that I needed to manually specify the bin range to np.histogram(), because otherwise numpy will automatically compute it based on the min/max values of the array (which are skewed when only background is detected)

If you pull from master, the segnet.py --stats should be fixed now. Thanks for reporting this.

[nishantshrivastav23]

@dusty_nv Tested and it worked.

Thanks