GPU recommendation

AI & Data Science Data Processing
#1

Hi everybody,

I hope the section is correct. I’m a PhD student and with my research group we’re starting to work on Machine Learning. I’m writing here for some advice on the hardware choice, as we have little expertise in this field and we need to set up a dedicated lab.

We will work on training of neural networks. We will use Tensorflow and Pytorch but we won’t work on images. Our budget is up to 2k$, possibly slightly less. Up to now I think our options are going either for a Jetson Xavier AGX or more simply for a PC with a good graphic card. To sum up, my questions are the following:

  • Is there any clear advantage in using a Jetson Xavier instead of a normal PC?
  • Which kind or series of graphic card are the most suitable for this kind of task? (training NN, no images)
  • Is the CUDA technology implemented and exploitable in all these solutions?

Thanks in advance for your help!

#2

hello, this is the question we all ask ourselves, we who have a limited budget. Already do not think of a single computer but of a cluster computer, you divide the acquisition costs, but you gain in efficiency, you will have to learn how to manage Linux, so Ubuntu, for the rest the node is ryzen 3/5, 8 GB of RAM, a terabyte hard drive. the GPU you need 1650/650 super at least, you need two nodes and a server .
To manage all of this, plus datasets, models, frameworks, go to https://cnvrg.io/

1 Like
#3

Thanks @devhci for the answer! So yes, if I get it right you basically suggest to invest in 2 smaller devices and to link them, distributing the workload. I’ll have a look to cnvrg.io, looks interesting

#4

I found it hard to express myself: not a server + a worker, but a server + two or more workers if you can, you have to think in “kubernetes” mode, which has become the alpha and omega of the deployment, certainly, it is far from our usual playgrounds, but it is so in the datasciences. Once again, do not buy right away, the prices are too high and it is speculative.

1 Like
#5

I’m going to tackle this question from a different angle:

  • Which kind or series of graphic card are the most suitable for this kind of task? (training NN, no images)

What’s your lab’s current intended model and training dataset size? How about your future? Some non-image, state of the art models are starting to require minimum 8GB. This is from earlier this year, but a good reference point on what memory sizes you may want to look for as this trend continues: https://lambdalabs.com/blog/choosing-a-gpu-for-deep-learning/.

What is your timeframe for buying? I don’t know if it is immediate, but I do know that this late in the Christmas rush is a tough time to buy highly desired items on the cheap (if at all), considering your budget. You may be able to still score some great deals now on other system builder components, like an extremely fast SSD or RAM, or even a GPU ready/equipped machine. A few things on my personal list are going to have to be 2021 items.

Pytorch support for CUDA 11.1 (https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel_20-11.html) opens up 30xx series nicely, which have larger memory sizes for far less money than the previous generation. That may give you more runway for the future. If your lab’s requirements are lesser, or you can’t get your hands on a 30xx in by your deadline, my thoughts are that the 20xx series is quite capable and won’t disappoint for now as your lab gets started.

1 Like
#6

Hi @tdyerNV , thanks for the reply. I’ll try to answer your points:

  1. We’re going to train a NN firtsly on a single satellite and then to a fleet of at most 10 agents. A simulation on 6 agents is 8 MB large. I guess that working on batch of simulations the size will increase quite easily. 500 simulations would mean 4 GB, right? Does this make sense to you or am I overestimating?

  2. My timeframe is not immediate. I think I’ll try to proceed in February. I reckon this isn’t the best period to buy.

  3. I was looking at the 30xx series as well, more specifically on the 3070/80. As for CPU, SSD and RAM I still have to worry about that, I mean, first I wanted to understand a couple of things on GPUs.

#7

according to the latest rumors, the 30xx series would only be available around February / March at best, not to mention the COVID which may further delay the on-sale. For the record, I am also stuck for the activation of my cluster. Yes, 4 GB is small, but we must also be able to do what we can, if Nvidia put the 3060 at the price from 1650, that would give startups a good boost

1 Like