I tested H.264 video decoding with CUDA SDK example. It is quite fast. However, the decoding gives some bugs (wrong blocks on the picture) when number of reference frame is equal to 5. Furthermore, if this number is greater than 5, the display is nothing but a black screen.
Have you also experienced such case? Is there some explanation about this bug?