Advices for my custom object detection task

I am trying to train a custom object detection model, my dataset contains 2 objects that are similar to each other, and also some of them are small objects which are about 1/20 of image size.

I tried to train an SSD mobilenet however it can not detect small objects, also sometimes it can not recognize most of the objects.

Also, I tried to train R-Cnn however it was a very slow model that can not run real-time. Also found that R-cnn can not be converted to TensorRT with improvement in inference time.

I see that segmentation models are good and fast on Jetson, but I can not find any Framework for train a Resnet-18 segmentation.

Could you please give me a piece of advice about what type of model should I use? Thank you.