Modifying the RNN example to use CUDNN_DATA_INT8

AI & Data Science Deep Learning (Training & Inference) cuDNN
1

I tried modifying RNN_example.ce (in cudnn_samples_v7/RNN) changing all CUDNN_DATA_FLOAT to CUDNN_DATA_INT8, but I got several runtime errors when I tried to run it:

$ ./RNN_int8 20 2 512 64 0
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 285
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 302
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 309
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 310
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 326
cuDNN Error: CUDNN_STATUS_NOT_SUPPORTED RNN_example_int8.cu 328
CUDA Error: out of memory RNN_example_int8.cu 330
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 373
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 402
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 482
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 516
cuDNN Error: CUDNN_STATUS_BAD_PARAM RNN_example_int8.cu 541

For example, the error at line 309 occurs on this line:

cudnnErrCheck(cudnnSetFilterNdDescriptor(wDesc, CUDNN_DATA_INT8, CUDNN_TENSOR_NCHW, 3, dimW));

Looking at the API docs I don’t see any restriction on using CUDNN_DATA_INT8 here. I’m guessing that the format (CUDNN_TENSOR_NCHW) isn’t correct for this data type? Any tips on changing this example to use CUDNN_DATA_INT8? Is there existing code somewhere?

Thanks.

Phil

2

I’m thinking that the RNN model does not support using the CUDNN_DATA_INT8 datatype. Can anyone with source code access (at Nvidia) confirm?