Advice of semantic segmentation on agx

[fcj]

I am looking for a semantic segmentation model which can run real time on AGX platform.
The input image resolution is 1920x1080.
Is any benchmark of semantic segmentation model running on AGX platform available?

Thanks,
FC
.

[AastaLLL]

Hi,

Please check the following tutorial for information:

dusty-nv/jetson-inference/blob/master/docs/segnet-console-2.md

The fcn-resnet18-cityscapes-2048x1024 model can achieve 47fps for 2048x1024 input on Xavier.

Thanks.

[fcj]

I have tried fcn-resnet18-cityscapes-2048x1024 model. Unfortunately, the output segmentation map is 1/16 of input image resolution which is not suitable to my application.
My application requires that segmentation output has the same resolution as input image.

Thanks,
FC

[dusty_nv]

@fcj

Unfortunately, the output segmentation map is 1/16 of input image resolution which is not suitable to my application.

Hi @fcj, the output will be rescaled using bilinear or nearest-neighbor interpolation to whatever size output image you feed into segNet. The raw grid on FCN segmentation models is typically always a fraction of the size of the input, and it just gets upsampled to the original size. The only difference is that I perform the upsampling manually in CUDA because it is faster than the way that PyTorch does the upsampling in the original model.

[fcj]

Hi, @dusty_nv

object boundary is quite blurry by upscaling 16 times of segmentation which is not acceptable to my application. Most semantic segmentation network has a decoder in the network which can do upsampling without blurring object boundary.

Thanks,

[dusty_nv]

The PyTorch FCN-ResNet segmentation models use an upsample instead of decoder, but still that part is often the slowest part and may slow it down to being sub-realtime, particularly on HD resolution. If you were to use a model from PyTorch that still has the upsample/decoder built-in, you could use that instead.