# Transfer data from host to device Transfer 10G **URL:** \ **Category:** CUDA Programming and Performance\ **Created:** [August 19, 2009, 12:39pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699 "2009-08-19T12:39:32Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 19, 2009, 12:39pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/1 "2009-08-19T12:39:32Z")
Hi What is the MAX bandwidth I can get with GTX 295? Can I improve it with external device (BUS, DMA, Special memory…)? I ask because I need to transfer 10G and to do Matrix-Vector Multiplication. Thanks Amir ---
**Author:** ![LSChien](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@LSChien](https://forums.developer.nvidia.com/u/LSChien)\ **Post date:** [August 19, 2009, 12:48pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/2 "2009-08-19T12:48:57Z")
> [@](#): > > Hi > > What is the MAX bandwidth I can get with GTX 295? > > Can I improve it with external device (BUS, DMA, Special memory…)? > > I ask because I need to transfer 10G and to do Matrix-Vector Multiplication. > > Thanks > > Amir you can use bandwidthTest.exe in SDK example to test, in my platform: GTX 295 host to device: 1.1 GB/s device to host: 1.7 GB/s ---
**Author:** ![Pimbolie1979](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Pimbolie1979](https://forums.developer.nvidia.com/u/Pimbolie1979)\ **Post date:** [August 19, 2009, 12:56pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/3 "2009-08-19T12:56:39Z")
You must buy a Mainboard with PCIe 2.0. Then you have a max datatransfer from 5000MByte per second (PC to GPU) and 5000MByte per second (GPU to PC). The PCIe Bus is a bidirectional Bus, so you can read and write data at the same time. You can buy a second GPU. So you can copy the first picture in the first GPU and the second picture in the second GPU. So you can copy 10000MByte per second. ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 19, 2009, 1:32pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/4 "2009-08-19T13:32:42Z")
> [@](#): > > You must buy a Mainboard with PCIe 2.0. > > Then you have a max datatransfer from 5000MByte per second (PC to GPU) and 5000MByte per second (GPU to PC). The PCIe Bus is a bidirectional Bus, so you can read and write data at the same time. > > You can buy a second GPU. So you can copy the first picture in the first GPU and the second picture in the second GPU. So you can copy 10000MByte per second. Hi Do you know how to get more B.W (Maybe special PICe)? Thanks Amir ---
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [August 19, 2009, 1:46pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/5 "2009-08-19T13:46:58Z")
> [@](#): > > Hi > > Do you know how to get more B.W (Maybe special PICe)? > > Thanks > > Amir You’ll always be limited by the PCIe of the board, I guess, since even if you have a very fast external device you need to connect it to the miserable x16 lanes you have on the motherboard. Does your calculation take a lot of time? If so you can try to move half the data, run the kernel async, copy the other half and than run another async kernel on the second half. That way you can save time and do things at the same time (calculate and transfer data). I think someone here refered to Nehalem as the fastest available machine - PCI wise eyal ---
**Author:** ![Laue](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Laue](https://forums.developer.nvidia.com/u/Laue)\ **Post date:** [August 19, 2009, 1:51pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/6 "2009-08-19T13:51:22Z")
> [@](#): > > Hi > > Do you know how to get more B.W (Maybe special PICe)? > > Thanks > > Amir PCIe (2.0) is a standardized port and the graphics hardware are manufactured for this standard. It doesn’t exists a “special” thing. You can overclock your system. But i think you a stable and reliable system is more desired… > I ask because I need to transfer 10G and to do Matrix-Vector Multiplication. > host to device: 1.1 GB/s 10G Host-\>Device / 5G (result back) Device-\>Host OK. it takes some time, but what’s the problem :) - you can’t calculate any problems in realtime [External Media](http://hqnveipbwb20/public/style_emoticons/<#EMO_DIR#>/smile2.gif "Media hosted on another site. Click to open in a new tab.") ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 19, 2009, 1:59pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/7 "2009-08-19T13:59:02Z")
> [@](#): > > PCIe (2.0) is a standardized port and the graphics hardware are manufactured for this standard. > > It doesn’t exists a “special” thing. > > You can overclock your system. But i think you a stable and reliable system is more desired… > > > I ask because I need to transfer 10G and to do Matrix-Vector Multiplication. > > > host to device: 1.1 GB/s > > 10G Host-\>Device / 5G (result back) Device-\>Host > > OK. it takes some time, but what’s the problem :) - you can’t calculate any problems in realtime [External Media](http://hqnveipbwb20/public/style_emoticons/<#EMO_DIR#>/smile2.gif "Media hosted on another site. Click to open in a new tab.") I need to get better results than CPU. ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 19, 2009, 2:00pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/8 "2009-08-19T14:00:43Z")
> [@](#): > > You’ll always be limited by the PCIe of the board, I guess, since even if you have a very fast external device > > you need to connect it to the miserable x16 lanes you have on the motherboard. > > Does your calculation take a lot of time? If so you can try to move half the data, run the kernel async, copy the > > other half and than run another async kernel on the second half. That way you can save time and do things at the same time (calculate and transfer data). > > I think someone here refered to Nehalem as the fastest available machine - PCI wise > > eyal What/Who is Nehalem? ---
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [August 19, 2009, 2:16pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/9 "2009-08-19T14:16:38Z")
> [@](#): > > What/Who is Nehalem? Nehalem are the new Intel chips. Look here at my (stupid) suggestion and what tmurray said: http://forums.nvidia.com/index.php?showtop…40&start=40 posts #44 - 47 how about the other question? how long does your kernel take? eyal ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 20, 2009, 8:24am UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/10 "2009-08-20T08:24:49Z")
> [@](#): > > Nehalem are the new Intel chips. > > Look here at my (stupid) suggestion and what tmurray said: > > http://forums.nvidia.com/index.php?showtop…40&start=40 > > posts #44 - 47 > > how about the other question? how long does your kernel take? > > eyal I talk about 268M and I run on Quadro FX 1700. The tranfer to GPU takes 95.5 (ms) → 2.8GB/s. ``` Kernel takes 26.6 (ms) ``` I want to buy GTX 295 so I believe I can get: Transfer (5GB/s) → 53.48 (ms). Kernel x30 0.88 (ms). What do you think? Amir ---
**Author:** ![eyalhir74](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@eyalhir74](https://forums.developer.nvidia.com/u/eyalhir74)\ **Post date:** [August 20, 2009, 8:38am UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/11 "2009-08-20T08:38:18Z")
> [@](#): > > I talk about 268M and I run on Quadro FX 1700. > > The tranfer to GPU takes 95.5 (ms) → 2.8GB/s. > > ``` > Kernel takes 26.6 (ms) > > ``` > > I want to buy GTX 295 so I believe I can get: > > Transfer (5GB/s) → 53.48 (ms). > > Kernel x30 0.88 (ms). > > What do you think? > > Amir Well, 5GB/s is probably optimistic but in anycase the ratio between the kernel and transfer is still very high. Any way you can save data from being copied to the GPU? for example, I had three arrays, one was sqrt of the other I found that it was better to calculate it on the fly than to pass it as input to the GPU. Maybe if you can elaborate more on what is the data you move and what does your kernel do, someone will have idea as to how to improve this ratio… edit: BTW - did you try pinned memory? eyal ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 20, 2009, 2:56pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/12 "2009-08-20T14:56:05Z")
> [@](#): > > Well, 5GB/s is probably optimistic but in anycase the ratio between the kernel and transfer is still very high. > > Any way you can save data from being copied to the GPU? for example, I had three arrays, one was sqrt of the other > > I found that it was better to calculate it on the fly than to pass it as input to the GPU. > > Maybe if you can elaborate more on what is the data you move and what does your kernel do, someone will have > > idea as to how to improve this ratio… > > I heard about “on the fly” solution. > > Can you give me some words about that? > > edit: BTW - did you try pinned memory? > > Using cudaMallocHost? > > Yes > > Amir > > eyal ---
**Author:** ![mfatica](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mfatica](https://forums.developer.nvidia.com/u/mfatica)\ **Post date:** [August 20, 2009, 3:03pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/13 "2009-08-20T15:03:13Z")
The FX1700 is a PCI-e gen2 card. You will not see faster PCI-e transfer using the GeForce. PCI-e transfer speed depends on the MB/chipset. ---
**Author:** ![CapJo](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@CapJo](https://forums.developer.nvidia.com/u/CapJo)\ **Post date:** [August 20, 2009, 9:14pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/14 "2009-08-20T21:14:34Z")
> [@](#): > > The FX1700 is a PCI-e gen2 card. > > You will not see faster PCI-e transfer using the GeForce. > > PCI-e transfer speed depends on the MB/chipset. Intel’s upcoming Lynnfield CPUs will have PCIe 2.0 integrated in the CPU itself and might therefore achieve higher bandwidth and certainly lower latencies. Lynfield Overview @ anandtech ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 21, 2009, 10:34am UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/15 "2009-08-21T10:34:41Z")
> [@](#): > > Intel’s upcoming Lynnfield CPUs will have PCIe 2.0 integrated in the CPU itself > > and might therefore achieve higher bandwidth and certainly lower latencies. > > Lynfield Overview @ anandtech Do you know how much bandwidth we can get with this chip and GTX295? ---
**Author:** ![avidday](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/avidday/32/59219_2.png) [@avidday](https://forums.developer.nvidia.com/u/avidday)\ **Post date:** [August 21, 2009, 10:57am UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/16 "2009-08-21T10:57:55Z")
> [@](#): > > Do you know how much bandwidth we can get with this chip and GTX295? It hasn’t been released yet. There are no performance numbers. But it will be less that 6.4 Gb/s because that is all that is practically achievable under PCI-e 2.0 with 16 lanes. ---
**Author:** ![YDD](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@YDD](https://forums.developer.nvidia.com/u/YDD)\ **Post date:** [August 21, 2009, 1:19pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/17 "2009-08-21T13:19:14Z")
Technical point on the GTX295… don’t the cards split the bus, so that each only really has x8 bandwidth? When I had a GX2, I think I found something like this was happening. ---
**Author:** ![avidday](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/avidday/32/59219_2.png) [@avidday](https://forums.developer.nvidia.com/u/avidday)\ **Post date:** [August 21, 2009, 1:45pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/18 "2009-08-21T13:45:12Z")
The GTX295 switches the bus, so can can get close to the full available PCI-e bandwidth to one of the GPUs if the other one is idle. WIth both going, it will be reduce to slightly less than half per GPU. ---
**Author:** ![YDD](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@YDD](https://forums.developer.nvidia.com/u/YDD)\ **Post date:** [August 21, 2009, 3:14pm UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/19 "2009-08-21T15:14:59Z")
OK - that’s good to know. The GTX295 is like one cable from an S1070 :) ---
**Author:** ![Amir\_Baruh](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Amir\_Baruh](https://forums.developer.nvidia.com/u/Amir_Baruh)\ **Post date:** [August 23, 2009, 7:21am UTC](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699/20 "2009-08-23T07:21:58Z")
> [@](#): > > The GTX295 switches the bus, so can can get close to the full available PCI-e bandwidth to one of the GPUs if the other one is idle. WIth both going, it will be reduce to slightly less than half per GPU. Ok… I want to buy a new machine (~2200$). I want the GTX295. I understood that there are some kinds of PCIex16. Which of them do you suggest? Thanks Amir [Next page](https://forums.developer.nvidia.com/t/transfer-data-from-host-to-device-transfer-10g/11699.md?page=2)