Common sources for GPU(cuda) and CPU

Accelerated Computing CUDA CUDA Programming and Performance
1

Is it possible to maintain one sources base to compile for CPU or GPU(make choice using building system)? Are there any pitfalls for this approach?