Onboard crystal oscillators typically drift, and so I would think 2 or 3 seconds is normal and expected. More accurate crystals can be used, but often they include a temperature control by means of a heating resistor (they use more power). Even this isn’t particularly accurate. I wouldn’t know what the best means is to provide an external clock source, but for precision you will need one of network source, GPS radio sources (which is basically the same as a network time setting, but over radio), or an external precision clock. What are your accuracy requirements?