I’m trying to estimate how many NICs/GPUs would be required for a particular data ingest, transform, and distribution application.
It’s a similar application to the HoloHub example NVIDIA Holohub (nvidia-holoscan.github.io) and I’d like to get a better of idea of scalability and bandwidth limitations. What would expected maximum throughput for a ConnectX-7 NIC and A30 GPU? Would they achieve the 400 Gbps limit of the X-7?
Hi @ben.stensland – this is a great question, and thanks for posting on the forums.
The Network Radar Pipeline you that referenced uses the Advanced Networking Operator (ANO) that’s available in Holohub. The goal of this operator is to provide both high bandwidth and low latency packet transfers from the NIC to the GPU, abstracting away much of the implementation detail.
We have worked with a number of customers who are able to combine I/O with compute at 200Gbps on an IGX system (CX7 and A6000 dGPU). We have also shown that 400Gbps to GPU is possible with the ANO, but doing this requires additional optimizations.
Without knowing more information about your application and specific requirements (e.g. latency, ability to split data feeds, etc), I think you could safely plan for 200Gbps per 1 GPU for a low latency GPU-accelerated real time pipeline.
Of course, we’re always here to talk through specifics about your own application.
Thanks for the quick response! In our scenario, we have several hundred incoming data streams, each carrying about 1.6 Gbps of I/Q which is decimated and processed on a per-stream basis (e.g. embarrassingly parallel). These reduced streams (~0.01 Gbps) are then sent back out to another data sink on the network. From ingest to client data sink, latency should be <0.1s.
Would you expect ~126 streams per CX7/A600 dGPU pair? How intensive are the optimizations to achieve the 400 Gbps rate?
Hey @ben.stensland – Looking closer at the A30 GPU, you’re limited to 16 lanes of PCIe Gen4 (64GB/sec bi-directional), meaning peak bandwidth in one direction would be 32GBps * 8 b/B = 256Gbps. That said, we’d love to chat with you more in-depth about your application and system design. I’ll send a note to you with my contact information, and we can take the conversation offline.
Hey @adamt I’ve tried sending you a couple of emails to set-up a call. Are they maybe getting stuck in a filter? The originating email would start with ben.stensland@ . Thanks
Oh no! Following up in DM, @ben.stensland