Looking through the documentation, there doesn’t seem to be any way to do pointwise division of two tensors. This is needed for example, in layer normalization where you divide by the standard deviation of each channel. Is there really no way to divide two tensors in cudnn other than copying to the CPU, doing the division, and copying back to the GPU?
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Tensor division operation | 1 | 1685 | December 3, 2019 | |
| cudnn v4: Divisive Normalization | 0 | 1128 | February 17, 2016 | |
| cudnn v3 cudnnDivisiveNormalization | 0 | 1126 | September 9, 2015 | |
| binary division in cuda | 0 | 380 | August 13, 2019 | |
| Division problem (weird behavior) | 23 | 18276 | November 15, 2010 | |
| cudnnDivisiveNormalization | 2 | 1153 | December 21, 2015 | |
| cudnnOpTensor missing ops vs TensorRT | 1 | 675 | July 9, 2018 | |
| Speed comparison of division compared to other arithmetic operations, perhaps something like clock cycles | 9 | 6600 | November 19, 2024 | |
| Double-single division | 2 | 2069 | May 15, 2008 | |
| How is dmeans calculated in the cudnnDivisiveNormalizationBackward | 2 | 657 | August 28, 2023 |