How to Resolve State Initialization Limitations in MTGP32 Generator with CUDA 11

Accelerated Computing GPU-Accelerated Libraries
1

I am trying to generate random numbers using the MTGP32 generator in CUDA 11 because it is the fastest option. However, according to the documentation and example code, MTGP32 can only create up to 200 initial state values. Therefore, when parallelizing beyond 200 blocks, multiple threads in different blocks reference the same state values, resulting in some identical outputs. Are there any techniques to solve this problem?

2

Thank you for your question. Indeed. the cuRAND documentation is saying that 200 parameter sets have been pre-generated. But it also tells that it is possible to generate other parameter sets as described in the paper by Mutsuo Saito, “A Variant of Mersenne Twister Suitable for Graphic Processors”, Jun 2010. It will be non-trivial effort. Is there any specific reason why you need more than 200 blocks?

3

Thank you.

The most simple solution is generateing huge amount of random numbers using for loop in less than 200 blocks.
But now, one thread generates one random number.

I guess “for loop” solution is slower. How Host API generator does it?

4

I see. You can always generate as many random numbers as you want with 200 blocks using CUDA grid stride loops.

5

Is that the fastest method to use MTGP32??

6

Not necessarily. RNG rarely is a sole bottleneck. Performance needs to be considered by analyzing the entire workflow. Some people prefer Philox, some Mersenne Twister. And in many cases, it is all subjective.

One of potential gaps (again, it all depends on the entire workflow) of Mersenne Twister family is its lengthy state, which is hundreds of bytes. With MTGP32 you have 200 of such states. It’s quite a bit of pressure on memory system, and may be a performance limiting factor in your workflow.

In contrast Philox state is compact. There are no limits on the number of Philox instances you use to generate subsequences in parallel. And a cherry on top is super easy and effective way of jumping ahead in a sequence. I would certainly give a try to Philox. Statistically it is proven to be as good as Mersenne Twister.

1 Like
7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.