I’m a beginner cuDNN developer and I’m trying to fuse operators for inference.
When going through the documentation (Developer Guide :: NVIDIA Deep Learning cuDNN Documentation), I learned that this might be done by using backend API. (Please correct me if I’m wrong. I’m not 100% sure because the “seem-to-be” relevant functions are in training library.)
Although this post provides a good high-level overview, I’m still not sure how to actually implement operator fusion.
For example, it seems like the following functions might be necessary.
Are these sufficient enough to get the right results?
Also, I’m not sure how to pass the input/output tensor information and detailed parameters for convolution, relu, etc.
I went through cuDNN sample codes to find the code example but unfortunately, I had no luck.
If anyone can provide any hint/feedback/idea/code sample, that will be greatly helpful.
Thank you in advance!