Hi,
I’ve noticed that if we turn on torch.autograd.set_detect_anomaly(True), CUDA Graphs will not be able to capture the operations. I looked materials in PyTorch and CUDA Graphs, but there’s no mentioning of it. My question is, are we able to safely turn off this feature and use CUDA Graphs?
Thanks!
Best wishes,
Yifan
From here:
This mode should be enabled only for debugging as the different tests will slow down your program execution.
Beyond that, there are pretty smart people on the pytorch forum. Its where I usually recommend that people post pytorch questions.