# Simple SVD for CUDA **URL:** \ **Category:** CUDA Programming and Performance\ **Created:** [July 21, 2008, 9:33pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678 "2008-07-21T21:33:00Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![Rocky24](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Rocky24](https://forums.developer.nvidia.com/u/Rocky24)\ **Post date:** [July 21, 2008, 9:33pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/1 "2008-07-21T21:33:00Z")
Hello all, I am new to CUDA and I am doing a research project to compare the power of GPU computing to the CPU for 3D reconstruction. The main algorithm I have to focus on is Singular Value Decomposition. I have searched up an down for SVD implemented using CUDA or CUBLAS but have yet to find anything. I attempted to take a step by step approach to writing my own but am stuck on how to implement the eigen values and vectors. I am hoping that perhaps there is someone who has a simple CUDA SVD , nothing too fancy, that they wouldnt mind letting me use to do the main part of my project which is the benchmarking. OR someone who knows enough to help me get my code where it needs to be. Thanks in advance!! ---
**Author:** ![Yao](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Yao](https://forums.developer.nvidia.com/u/Yao)\ **Post date:** [July 23, 2008, 7:21pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/2 "2008-07-23T19:21:51Z")
There is a technical report, “LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs” that talks about GPU LU and QR decomposition, and GPU-specific issues in terms of performance. Hope it helps. [url=“[http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-49.pdf](http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-49.pdf)”][http://www.eecs.berkeley.edu/Pubs/TechRpts...ECS-2008-49.pdf[/url]](http://www.eecs.berkeley.edu/Pubs/TechRpts...ECS-2008-49.pdf%5B/url%5D) Yao ---
**Author:** ![darkstorm](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@darkstorm](https://forums.developer.nvidia.com/u/darkstorm)\ **Post date:** [July 24, 2008, 2:18am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/3 "2008-07-24T02:18:21Z")
A 512 x 512 SVD on CUDA code, one-side jaccobi method, optimized memory access for GPU, faster than Intel MKL SGESVD, slower than SGESDD. A bidiagnalized input is suggested for better accuracy, output is U and W\*V. ZhangShu, DouHeng, supplementary issue of \<\>,ChengDu, 2009.7 [cusvd\_by\_ZhangShu.rar](https://forums.developer.nvidia.com/uploads/short-url/zk0niupQb7ysOXRtLk4kbGtxEAn.rar) (338 KB) ---
**Author:** ![senorbum](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@senorbum](https://forums.developer.nvidia.com/u/senorbum)\ **Post date:** [July 24, 2008, 2:36pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/4 "2008-07-24T14:36:17Z")
> [@](#): > > There is a technical report, “LU, QR and Cholesky Factorizations using Vector Capabilities of GPUs” that  talks about GPU LU and QR decomposition, and GPU-specific issues in terms of performance. Hope it helps. > > http://www.eecs.berkeley.edu/Pubs/TechRpts…ECS-2008-49.pdf > > Yao > > [snapback]414927[/snapback] In their timings they don’t include mem transfer. I don’t understand WHY people don’t include this, as it is something you NEED to do if you want to use the GPU. Just keep that in mind when looking at the results. ---
**Author:** ![E.D\_Riedijk](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@E.D\_Riedijk](https://forums.developer.nvidia.com/u/E.D_Riedijk)\ **Post date:** [July 24, 2008, 5:59pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/5 "2008-07-24T17:59:42Z")
> [@](#): > > In their timings they don’t include mem transfer. > > I don’t understand WHY people don’t include this, as it is something you NEED to do if you want to use the GPU.  Just keep that in mind when looking at the results. > > [snapback]415379[/snapback] Well just as an example, people might generate the input to the SVD on the GPU. I always prefer it when both performance numbers are shown, the ones including & excluding transfer. As a side note: Anybody have a gaussian elimination version for CUDA lying around? Now I let matlab calculate the inverse of a matrix I generate in CUDA & use the inverse in CUDA again, but I do not need the inverse in my algorithm. ---
**Author:** ![senorbum](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@senorbum](https://forums.developer.nvidia.com/u/senorbum)\ **Post date:** [July 24, 2008, 6:22pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/6 "2008-07-24T18:22:04Z")
> [@](#): > > Well just as an example, people might generate the input to the SVD on the GPU. I always prefer it when both performance numbers are shown, the ones including & excluding transfer. > > As a side note: Anybody have a gaussian elimination version for CUDA lying around? Now I let matlab calculate the inverse of a matrix I generate in CUDA & use the inverse in CUDA again, but I do not need the inverse in my algorithm. > > [snapback]415462[/snapback] At one point in the summer this would have helped me too. I wouldn’t mind taking a look at some code of Gaussian elimination, but don’t need it for anything. I get sad when I realize that I enjoy looking at linear algebra and related code now \/crying.gif’ class=‘bbc\_emoticon’ alt=‘:’(’ /\> ---
**Author:** ![E.D\_Riedijk](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@E.D\_Riedijk](https://forums.developer.nvidia.com/u/E.D_Riedijk)\ **Post date:** [July 24, 2008, 7:43pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/7 "2008-07-24T19:43:49Z")
I have just been reading about FLAME and there are some working notes suggesting that they have working code for CUDA for these kinds of operations. Tomorrow when at work I’ll dig deeper. I already had a CUDA algo in my head, but am afraid that is it quite sub-optimal ;) ---
**Author:** ![vvolkov](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@vvolkov](https://forums.developer.nvidia.com/u/vvolkov)\ **Post date:** [January 28, 2009, 6:52am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/8 "2009-01-28T06:52:02Z")
> [@](#): > > In their timings they don’t include mem transfer. > > I don’t understand WHY people don’t include this, as it is something you NEED to do if you want to use the GPU. Just keep that in mind when looking at the results. No, mem transfer is included in timings (if you are talking about my tech report). ---
**Author:** ![Bugfree](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Bugfree](https://forums.developer.nvidia.com/u/Bugfree)\ **Post date:** [October 15, 2009, 6:04pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/9 "2009-10-15T18:04:48Z")
> [@](#): > > Hello all, > > I am new to CUDA and I am doing a research project to compare the power of GPU computing to the CPU for 3D reconstruction. The main algorithm I have to focus on is Singular Value Decomposition. I have searched up an down for SVD implemented using CUDA or CUBLAS but have yet to find anything. I attempted to take a step by step approach to writing my own but am stuck on how to implement the eigen values and vectors. > > I am hoping that perhaps there is someone who has a simple CUDA SVD , nothing too fancy, that they wouldnt mind letting me use to do the main part of my project which is the benchmarking. OR someone who knows enough to help me get my code where it needs to be. > > Thanks in advance!! Hi, Did you have any luck in implementing SVD eventually… I am working on a similar project … Would be great if I could get some input… Thanks in advance!! ---
**Author:** ![vvolkov](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@vvolkov](https://forums.developer.nvidia.com/u/vvolkov)\ **Post date:** [October 15, 2009, 9:49pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/10 "2009-10-15T21:49:30Z")
> [@](#): > > Did you have any luck in implementing SVD eventually… I am working on a similar project … Would be great if I could get some input… Did you check CULAtools? They seem to have SVD: http://www.culatools.com/versions/basic There was also a relevant paper “Singular value decomposition on GPU using CUDA” by Lahabar and Narayanan in IPDPS’09. ---
**Author:** ![Jimmy\_Pettersson](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Jimmy\_Pettersson](https://forums.developer.nvidia.com/u/Jimmy_Pettersson)\ **Post date:** [October 16, 2009, 7:15am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/11 "2009-10-16T07:15:00Z")
Yes, here is the paper. Really good read. I’ve tested the culatools SVD and compared to some of the times shown in the paper. Culatools seems to give the same or slightly better results for larger sizes even though I have a GTX260 while they use a GTX280 in their paper… [SVD.pdf](https://forums.developer.nvidia.com/uploads/short-url/4fU9vq5RCHnhHUTbnczQrPmmyjt.pdf) (177 KB) ---
**Author:** ![Jimmy\_Pettersson](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Jimmy\_Pettersson](https://forums.developer.nvidia.com/u/Jimmy_Pettersson)\ **Post date:** [October 16, 2009, 7:17am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/12 "2009-10-16T07:17:29Z")
Oh yeah, i’ve also had a look at Mr. Volkovs gpu\_lapack, very cool stuff! ---
**Author:** ![Bugfree](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Bugfree](https://forums.developer.nvidia.com/u/Bugfree)\ **Post date:** [October 17, 2009, 8:03am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/13 "2009-10-17T08:03:23Z")
> [@](#): > > Oh yeah, i’ve also had a look at Mr. Volkovs gpu\_lapack, very cool stuff! Thanks for the prompt response guys… The thing is … I have written my own C code for QR factorisation and I am nearly done with my C code for SVD using QR factorization … So I am trying to implement this C code in CUDA… But I am running thru too many issues … even after thoroughly goin thru the programming guide and several examples… So I was wonderin, if there is a code out there that does not use the LAPACK library, which could be of some assistance… I am running short of time guys, I would really appreciate this… Many thanks… ---
**Author:** ![Jimmy\_Pettersson](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Jimmy\_Pettersson](https://forums.developer.nvidia.com/u/Jimmy_Pettersson)\ **Post date:** [October 17, 2009, 8:54am UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/14 "2009-10-17T08:54:57Z")
Hmm… check Mr. Volkovs first post here [url=“[The Official NVIDIA Forums | NVIDIA](http://forums.nvidia.com/index.php?showtopic=89084&mode=threaded&pid=573376)”][http://forums.nvidia.com/index.php?showtop...&pid=573376[/url]](http://forums.nvidia.com/index.php?showtop...&pid=573376%5B/url%5D) Ther is a link to his gpu\_lapack code there… ---
**Author:** ![Bugfree](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Bugfree](https://forums.developer.nvidia.com/u/Bugfree)\ **Post date:** [October 18, 2009, 3:48pm UTC](https://forums.developer.nvidia.com/t/simple-svd-for-cuda/4678/15 "2009-10-18T15:48:38Z")
> [@](#): > > Hmm… check Mr. Volkovs first post here http://forums.nvidia.com/index.php?showtop…&pid=573376 > > Ther is a link to his gpu\_lapack code there… Jim, Could you tell what is the best way I can write the following piece of code on the device… for(i=k;i\<=Row-1;i++) { tempp=i\*Row+j; Q[tempp]=Q[tempp]-2.0_tempt_R[i\*Col+k]; } I’ve tried putting idy in place of Col, but that doesnt work … I would like the above code to run in parallel as tempp changes… Any help would be really appreciated… Thanks in advance…