CUDA program causes NVDIA driver to crash, should I disable the TDR?

Hello ,
The equipment produced by our company runs on the Windows 10 operating system, uses the Geforce RTX series graphics card to do some image optimization work, and the display and CUDA calculation work run on the same graphics card.

We found that some devices will appear blue screen phenomenon under some accidental circumstances. We checked the system log and found that the graphics card driver experienced three times of unresponsive conditions and then appeared blue screen. We have studied the TDR (timeout detect and recovery) mechanism of Windows. It is this mechanism that determines the non-response state of the graphics card driver and finally triggers the blue screen.

We can see from Google that many people have encountered this problem and there are recommended solutions:

  1. Use multiple graphics cards. The display and cuda use different graphics cards, but we do not have such conditions for our equipment.

  2. Modify the TDR setting in the Windows registry, set the timeout time to longer, or close the TDR directly.
    But when we look at Windows knowledge, TDR is used for debugging and development in hardware driver development.

So should we change the TDR setting?
What situations do we need to set TDR?
What changes will be made to the system after setting the TDR, and what adverse effects will it have?
We can’t determine whether to set it.