[CUDNN 8.1 conv2d] if we set alpha to 1.0 and set output as INT32, are there type conversions under the hood?

[ocwins.z]

With backend API, we can set input as INT8 and output as INT32.

In this case, does cuDNN still cast the conv’s output to FLOATs and multiply by alpha(which is 1.0), then cast the result to INT32 ?

Another question, in this diagram,

How is the “Clamp” to be done? If I don’t make it wrong, user doesn’t provide any information about this “Clamp”, so what should I expect for the “data output y”?

[AakankshaS]

Hi @ocwins.z ,
cuDNN doesn’t support INT32 output type, see Table 17 for all the supported configurations below API Reference - NVIDIA Docs

@ocwins.z

How is the “Clamp” to be done? If I don’t make it wrong, user doesn’t provide any information about this “Clamp”, so what should I expect for the “data output y”?

It’s clamping to the casted range and Round to Nearest Integer (nearest even for midways).

Thanks!

[ocwins.z]

@AakankshaS

cuDNN doesn’t support INT32 output type, see Table 17 for all the supported configurations below API Reference :: NVIDIA Deep Learning cuDNN Documentation

OK…

But I play around with your examples (conv_sample.cpp from cudnn-frontend), set output to INT32, and it passes.

====PADDING DIMENSIONS====
padded input dims are 1, 256, 10, 9
padded filter dims are 256, 256, 3, 3
padded output dims are 1, 256, 10, 9
x CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 3 Id: 120 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
y CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 3 Id: 121 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
w CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 3 Id: 119 Alignment: 16 nDims 4 Dim [ 256,256,3,3,0,0,0,0,0 ] Str [ 2304,1,768,256,0,0,0,0,0]
z CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 3 Id: 122 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
b CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 4 Id: 98 Alignment: 16 nDims 4 Dim [ 1,256,1,1,0,0,0,0,0 ] Str [ 256,1,256,256,0,0,0,0,0]
A CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 4 Id: 65 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
B CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 3 Id: 66 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
C CUDNN_BACKEND_TENSOR_DESCRIPTOR : Datatype: 4 Id: 67 Alignment: 16 nDims 4 Dim [ 1,256,10,9,0,0,0,0,0 ] Str [ 23040,1,2304,256,0,0,0,0,0]
ADD_ CUDNN_BACKEND_POINTWISE_DESCRIPTOR : Mode: 0 Math precision 4
BIAS CUDNN_BACKEND_POINTWISE_DESCRIPTOR : Mode: 0 Math precision 4
ACTV CUDNN_BACKEND_POINTWISE_DESCRIPTOR : Mode: 100 Math precision 4
CONV CUDNN_BACKEND_CONVOLUTION_DESCRIPTOR : Datatype: 4 Mode: 0 Num Dimensions: 2 PadLower [ 1,1,0,0,0,0,0,0,0 ] PadUpper [ 1,1,0,0,0,0,0,0,0 ] Dilation [ 1,1,0,0,0,0,0,0,0 ] Stride [ 1,1,0,0,0,0,0,0,0]
CONV_OP CUDNN_BACKEND_OPERATION : OpMode: 10 X 00000214A8BAE7F0 Y 00000214AD172320 W 00000214AD170460 B 0000000000000000 C 00000214AD172680 P 0000000000000000 alphabetaType 4 Alpha: 1 1 Alpha2: 1 1 Beta: 0 0
ADD__OP CUDNN_BACKEND_OPERATION : OpMode: 13 X 00000214AD172320 Y 00000214AD172BC0 W 0000000000000000 B 00000214AD171C00 C 0000000000000000 P 00000214AD172A40 alphabetaType 4 Alpha: 1 1 Alpha2: 1 1 Beta: 0 0
BIAS_OP CUDNN_BACKEND_OPERATION : OpMode: 13 X 00000214AD172BC0 Y 00000214AD171D80 W 0000000000000000 B 00000214AD171C60 C 0000000000000000 P 00000214AD1727A0 alphabetaType 4 Alpha: 1 1 Alpha2: 1 1 Beta: 0 0
ACTV_OP CUDNN_BACKEND_OPERATION : OpMode: 13 X 00000214AD171D80 Y 00000214AD170400 W 0000000000000000 B 0000000000000000 C 0000000000000000 P 00000214AD172C20 alphabetaType 4 Alpha: 1 1 Alpha2: 1 1 Beta: 0 0
CUDNN_BACKEND_OPERATIONGRAPH_DESCRIPTOR : has 34 engines.
Heuristic has 3 configurations
Plan tag: ConvFwd_Add_Add_ReluFwd_eng11
CUDNN_BACKEND_EXECUTION_PLAN_DESCRIPTOR : requires workspace 0
variantPack CUDNN_BACKEND_VARIANT_PACK_DESCRIPTOR : has 5 data pointers

I mark it as C here, “AFTERCONV_TENSOR” in the source code, it’s conv_op’s output.

@AakankshaS

It’s clamping to the casted range

Do you mean mapping (min, max) to (-128, 127), or mapping floats with saturation (for example, the range of INT16) to (-128, 127), or something else?

If the activation is RELU, the min values are zeros, and all zeros go to -128 ?

Thank you.

[AakankshaS]

Hi @ocwins.z ,
Apologies for the delay, are you still facing the issue?

[ocwins.z]

No. I made it clarified.

The term “clamp” on the image means that: value = min(127, max(-128, value)), it just “cut off” the value, but not “mapping” it from a larger range into a smaller range.

Sorry for the naive question.

But I still suggest you or someone else could answer to question like this. Not everyone are familiar with these terms especially people who never meet these terms in documents they had read. A brief explanation could make guys like me lives more easier.

Or am I the only fool?

(The reason I think it is a mapping: “clamp” - to hold sth tightly, or fasten two things together, with a clamp - the Oxford Dictionary, I think the value range is “sth” here. ) ATTENTION: WRONG!

I didn’t come here and answer it myself because I think I’m not right person to clarify it. If I gives wrong answer, I may mislead others.

so If I’m wrong, plz correct me for others. Thank you.