Disable T4 from trying to display in Ubuntu

I am using a T4 for Inference. Whenever someone plugs in a different monitor the T4 becomes the main display device and the other monitor is the extended device. How I can disable the T4 from being a display device or at force the monitors to default as cloned displays.

Environment

GPU Type : T4
Operating System + Version : Ubuntu 18.04
Baremetal or Container (if container which image + tag) : Baremetal