YOLOv8s Benchmark: Shuold mAP value should increase with image resolution?

[varun.pratapsingh]

We have been running evaluation commands for a Nvdia orin nx 8GB using YOLOv8s model at different image resolutions with the aim of achieving a mean Average Precision (mAP) between 50-95 on the COCO validation 2017 dataset.

$ yolo val model=yolov8s.pt data=coco.yaml batch=1 imgsz=640
$ yolo val model=yolov8s.pt data=coco.yaml batch=1 imgsz=1080
$ yolo val model=yolov8s.pt data=coco.yaml batch=1 imgsz=3840

However, we have noticed that the mAP value droop across all three resolutions (640x480, 1920x1080, and 2160x3840).

We initially expected that increasing the resolution would result in a higher mAP value. What are your thoughts on this observation?

[AastaLLL]

Hi,

Could you run the same command on other platforms (ex. x64 desktop) as well?
This will help to check whether the accuracy drop is caused by hardware differences or the different image resolution.

Thanks.

[varun.pratapsingh]

Hi

I tested what you suggested.

we are getting mAP Val (50-95) - 44.7 for 640p, 40.2 for 1080p, 20.3 for 3840p. What are your thoughts ?

Note: All imges dataset are of 640p.

[AastaLLL]

Hi,

Please check with the YOLOv8 team.
Maybe there are some configs that need to be changed for different resolutions.

Thanks.