At the Ignite developer conference, Microsoft unveiled their latest Azure HBv5 virtual machines (VMs), which are powered by four EPYC processors specially customized by AMD with HBM3 memory. The HBv series of Azure VMs are designed to deliver substantial memory bandwidth, a critical factor for High-Performance Computing (HPC), which Microsoft identifies as the primary bottleneck in this field. Previously, Microsoft utilized AMD's Milan-X and Genoa-X processors, which featured 3D V-Cache technology to tackle bandwidth challenges. However, with the introduction of the HBv5 VMs, Microsoft has evidently shifted its approach.
According to Tom's Hardware, a single Azure HBv5 VM boasts 450GB of HBM3 RAM and 352 Zen 4 cores clocked at 4GHz, offering double the interconnect (IF) bus bandwidth of a typical EPYC processor. However, synchronous multithreading (SMT) is disabled. Additionally, it features 800Gb/s Nvidia Quantum-2 InfiniBand for network switching. Collectively, the four processors in a single HBv5 VM provide up to 6.9TB/s of memory bandwidth, which is nine times that of an HBv4 VM equipped with Genoa-X CPUs and 20 times that of an HBv3 VM with Milan-X CPUs.
The HBM3 memory utilized on these custom processors functions similarly to 3D V-Cache, but instead of serving as an L3 cache, it operates as a massive L4 cache with significantly more capacity and bandwidth. While this is beneficial, it doesn't match the latency of a direct L3 cache like the 3D V-Cache. However, for the workloads intended for this product, bandwidth is paramount, thus making latency a secondary concern.
Each of the four CPUs comprises a total of 352 cores, distributed as 88 cores per CPU. It's plausible that each CPU itself consists of 96 cores, with 8 cores reserved for managing virtual machines. Furthermore, this custom processor from Microsoft appears to be relatively similar to the MI300C, which is essentially the MI300A APU minus the CDNA 3 graphics, featuring just the Zen 4 CCD. Both variants are 96-core CPUs equipped with HBM3 RAM and can reach speeds of up to 3.7GHz, paralleling the CPUs deployed in the HBv5 VMs.
