Looking through the documentation, there doesn’t seem to be any way to do pointwise division of two tensors. This is needed for example, in layer normalization where you divide by the standard deviation of each channel. Is there really no way to divide two tensors in cudnn other than copying to the CPU, doing the division, and copying back to the GPU?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Tensor division operation | 1 | 1650 | December 3, 2019 | |
| Multiple GPU processing with cuDNN | 0 | 718 | December 16, 2018 | |
| Tips for Optimizing GPU Performance Using Tensor Cores | 15 | 1008 | July 24, 2019 | |
| How to utilize tensorcore when designing the network | 1 | 427 | August 29, 2023 | |
| How to calculate deconvolution layer with cudnn? | 0 | 1343 | December 27, 2016 | |
| Tensor packing and cryptic errors | 6 | 968 | June 8, 2022 | |
| cudnnBatchNormalizationForwardTraining Results in batchNormOutputTensor with Same Large Negative Double | 2 | 2250 | February 3, 2020 | |
| Feature request : Cudnn Layer Normalization | 1 | 1395 | January 14, 2020 | |
| Tensor Descriptors and Computational Graph in cuDNN | 1 | 509 | April 10, 2020 | |
| cuDNN: Merging parallel layers? | 1 | 1052 | October 27, 2014 |