All hardware IRQ access starts in CPU0 core. The network card no doubt uses time on that core. It is possible that flooding CPU0 with network interrupts could change things for other drivers, but not necessarily (it’s plausible but you’d have to have evidence that the IRQs are overwhelming CPU0).
usleep() would likely improve things as it gives the ksoftirq scheduler a chance to smooth out its load and context switch to handle the higher priority soft IRQs (I’m assuming the usleep() is not within a hardware driver which is locking CPU0…a locked CPU0 core sleeping would be extraordinarily bad) and context switch among the non-CPU0 cores. This would give even a heavily loaded system the feeling of running smoothly.
Other calls like gettimeofday() would depend upon how the drivers are structured; perhaps gettimeofday() has some of its work offloaded to ksoftirq, which would be good, but if gettimeofday() were indirectly related to NTP and locked CPU0 while waiting for a remote answer this would be bad (I do not believe this is an actual issue, this is just for illustration).