As a project last year, I developed a compiler that allows use of NVIDIA CUDA GPUs from within Java, just by using Java annotations. The end result is far from production-ready, or bug free, but may be interesting to some of you. You can get the source code from http://code.google.com/p/java-gpu/.
I’m not going to be developing this any further, but can probably offer some guidance if anyone wants to modify it etc. I’d be interested if you find it of any value!