The Backbone Nobody Sees
Copyright: Sanjay Basu A Deep Dive into HPC Communication Protocols and the Networks That Make Everything Possible There is a persistent illusion in the world of high-performance computing, and it goes something like this. People assume the hard part is the compute. The GPUs, the CPUs, the accelerators. They obsess over teraflops and petaflops, over CUDA cores and tensor cores, over clock speeds and memory bandwidth. And they are not wrong to care about these things. But they are looking at only half the picture. Maybe less than half. The truth, which anyone who has spent serious time with large-scale systems will confirm, is that the network is where the magic happens. Or, more precisely, where the magic falls apart when you get it wrong. You can have the most powerful GPUs money can buy, but if they cannot talk to each other with sufficient speed and low enough latency, you might as well be running a very expensive space heater. The history of HPC is, in many ways, the history ...