is it normal for creating cudnn handle that takes about 1.5 second?

cuda version is 10.0, cudnn version is 7.4

Could be normal, depending on machine configuration and other software characteristics.

Thanks for reply! I have another question, in cudnnSetTensor4dDescriptor() API, CUDNN_DATA_INT32 is not support(bad parameters error) in my code, CUDNN_DATA_FLOAT could work, what’s the possible reason?

I think INT32 tensors would be fairly unusual. Many library operations such as CUBLAS and CUFFT that CUDNN depends on are not supported for integer data.

Anyway my suggestion would be to ask CUDNN specific questions on the CUDNN forum.