So, I kind of have a complicated problem. I have made some bindings in golang for cudnn. But cudnn doesn’t have a function I need in order to do the Adam optimization. Long story short. Everything seems to work until after I launch the kernel using cuLaunchKernel(). If I sync the stream I get an error “illegal memory access was encountered”. I guess my question is. When launching my own kernels using the driver api. Do I need to create a new context on a new host thread in order to get this to work?
I fixed the problem. When, I separated the code I was still running into the problem with cuLaunchKernal(). It turned out I wasn’t passing the device memory pointer incorrectly it. Just one symbol stole 8+ hours of my life.