InfoRom is corrupted upon waking from sleep on Ubuntu machine

I get the following warning

WARNING: infoROM is corrupted at gpu 0000:0C:00.0
WARNING: infoROM is corrupted at gpu 0000:41:00.0

when I resume the machine from sleep mode. I tried doing the same on Windows, where I do not get such warnings. I need help in having this resolved.
nvidia-bug-report1.log.gz (1.6 MB)
nvidia-bug-report.log.gz (1.54 MB)

Please run nvidia-bug-report.sh as root and attach the resulting .gz file to your post. Hovering the mouse over an existing post of yours will reveal a paperclip icon.
[url]https://devtalk.nvidia.com/default/topic/1043347/announcements/attaching-files-to-forum-topics-posts/[/url]

Did this also happen with an earlier driver, e.g. v410? Does running the nvidia-persistenced help with it? Besides the warning, are there any other issues noticeable after resume?

Relating to the driver, I believe v418 is the first driver I installed since the computer is less than 30 days old.

There has not been noticeable side effects after resuming; Just to ensure that the GPU is alright, I ran a few deep learning applications like TensorFlow on both the GPUs, and they seem to work fine. I am worried whether the warning indicates something serious or something that is about to happen in the near future. You know when you keep restarting the computer improperly, it starts to throw warning in subsequent boots and eventually result in hard drive not being accessible. Although I have not done so, I am concerned whether it could be a similar case with my GPUs.

what is nvidia-persistenced ?

nvidia-persistenced is a daemon installed alongside the driver and is necessary on headless or multi-gpu systems for the driver to work correctly.
AFAIK, the inforom is mainly needed on ECC enabled gpus.
Since the warning message is showing up for two different gpus, I’d rather suspect it’s a driver bug. Does it go away after a clean reboot?

Yes, the message disappears after reboot. Since you have mentioned the purpose of nvidia-persistenced, I tried installing it, but no luck though. It still throws the same warning when resumed from sleep.

Even I believe this is a driver bug since this does not show up on Windows. Any hint on how to solve it ? In the first place, do you think this would at any cost cause damage to the GPUs I have on the machine because TITAN was so expensive that I cannot afford to buy again.

IDK if this could cause any persistent damage, you’d better mail the logs and problem description to linux-bugs[at]nvidia.com to maybe get answer.
Looking for a bios update for your mainboard might also be a good idea and to feel safer, you could use nvflash to make a backup of your gpu bios.