mAP Accuracy - how to improve it?

I’ve been training a 3,500 images model on DIGITS with standard bvlc_googlenet.caffemodel model and am getting an mAP of about 20 % after 100 epochs on the AWS.

I tried adding another 500 new, original, images and this made no difference to the mAP.

I added another 1200 ‘dontcare’ images …… no difference!

Has anybody got ideas on what should be my next step to get more accuracy? …… Thanks!