Can anyone tell me please, I just want to encode movie. how to use CUDA, which software I should?

I use E4600 + 9500GT in my system.
my purpose is encode my DVD movie to H.264 mkv or avi file.
and my (AVCHD) movie made from Sony FullHD Camera HDR-SR10E encode to DVD disc.

I try to use badaboom, but I don’t know what’s codec it used, the file type that output from encoding is “mp4”?
I try to use TMPGEnc 4.0 XPress, to encode AVCHD file to be DVD MPG, I have opened CUDA feature in TMPGEnc already. but It’s slower than encoding without CUDA. for 1 hour AVCHD file it use more than 3 hours to encode to DVD mpeg.
I try to use “TotalMedia Theatre 3” to playback movie file. I found there’re CUDA feature but I can’t open it when play AVCHD file or DVD disc. the only filetype that I can open CUDA feature is playing MPG file.
I don’t understand.

please tell me which program I should use for my purpose,

thank you,.