OptiX is a high-level GPU accelerated ray casting SDK. If you can express an algorithm by tracing rays, it should be possible to implement and accelerate that with OptiX.
The OptiX SDK comes with various examples explaining individual features it supports. Then there are the some OptiX Advanced Samples you can find on github. Link in one of the sticky posts on this sub-forum.
You need an NVIDIA GPU and the architectures supported by the current version are Kepler, Maxwell, Pascal, and Volta.
The newer the architecture, the higher-end the GPU, and the more VRAM, the better it will perform.
Wikipedia has a decoder for which GPU is on which NVIDIA board.
Please read the OptiX Release Notes for what the other prerequisites are to develop applications using OptiX.
Here are some links you should read and search through to get more information:
Ignore the OptiX version numbers in these. The current release is OptiX 5.0.1.
You can find many GTC presentations about OptiX for various usage cases here, including some which don’t do image synthesis: