I’m using a HDMI to VGA converter to connect the vga monitor. But screen blanks frequently and then comes back. Tried different resolution but not much difference.
Is there anything I can play with…settings, kernel patch etc.?

VGA is not supported. It lacks the wire used for automatic configuration (the “DDC” wire, which provides the “EDID” data via an i2c protocol), and the driver itself only accepts automatic configuration. What you are seeing is video failing, followed by falling back to some default mode.