Could you post minimal, complete (buildable and runnable) code please that reproduces the problem? I’d be happy to take a look.
Integer arithmetic in CUDA follows standard C/C++ specifications and there should be no result differences to CPU computation unless undefined behavior is invoked, which should not be the case here. Note that there is no rounding mode in integer computations, that is a concept that applies to floating-point computation. According to you original post, ‘a’, ‘b’, and ‘m’ are all of type ‘int’, so no type conversions take place.
What version of CUDA are you using? What is your exact nvcc command line? If you are currently doing a debug build try switching to release build, if you are currently doing a release build, try switching to a debug build. Does the observed behavior change?