Should I synchronize after forward/backward APIs before destroying descriptors?

[Shuai_Guo]

For example:

cudnnCreateTensorDescriptor(&xdesc);

cudnnXXXForward(..., xdesc, ...);

// Need synchronization HERE?

cudnnDestroyTensorDescriptor(xdesc);

I think cudnnXXXForward APIs are async, so we need synchronization before destroying tensor descriptors, it that right?

[spolisetty]

Hi,

I don’t think the sync is needed, by the end of cudnnXXXForward(..., xdesc, ...); all the information in the xdesc that is needed has been converted into kernel parameters, etc and sent to some driver buffer.
At that point, xdesc can be safely deleted.

Thank you.