SK Hynix has announced that it had developed the industry’s first 12-layer 24GB HBM3 memory stacks that offer both high density and extreme bandwidth of 819 GB/s. The 12-Hi HBM3 products maintain the same height as the company’s existing 8-layer HBM3 products, meaning that they are easy to deploy.
SK Hynix’s 24GB HBM3 known good stack die (KGSD) product places twelve 16Gb memory devices connected using through silicon vias (TSVs) on a base layer with a 1024-bit interface. The device features a 6400 MT/s data transfer speed and therefore the whole 24GB HBM3 module offers a bandwidth of 819.2 GB/s.
Depending on the actual memory subsystem, such modules can enable 3.2 TB/s – 4.915 TB/s of bandwidth for 96GB of memory over a 4096-bit or 144GB of memory over a 6140-bit interface, respectively. To put the numbers into context, Nvidia’s H100 NVL — the most advanced HBM3 implementation to date — features 96GB of memory with 3.9 TB/s of bandwidth for each of its two GH100 compute GPUs.
Stacking 12 layers of HBM DRAM on top of each other is challenging for several reasons. Firstly, it is hard to drill some 60,000 or more TSV holes through the package to connect all twelve layers. Secondly, 12-Hi HBM DRAM packages cannot get physically higher than 8-Hi HBM KGSDs (typically, 700 – 800 microns, 720 microns in the case of Samsung), so it will be extremely complicated (if possible at all) to install such HBM3 KGSDs next to a CPU or a GPU that have a fixed height. To that end, DRAM makers like SK Hynix need to either reduce thickness of an individual DRAM layer without sacrificing yields or performance (which brings in a slew of challenges) or reduce the gap between layers as well shrink the base layer.
SK Hynix says that that to build its 12-Hi HBM3 product with the same height as its 8Hi HBM3 device, it used its Advanced Mass Reflow Molded Underfill (MR-MUF) encapsulation technology that involves mass reflow (MR) chip attach to shrink the base layer and molded underfill (MUF) processes to reduce die-to-die spacing.
SK Hynix has delivered samples of its 24GB HBM3 product to several eagerly anticipating customers. The product is currently undergoing performance evaluation and is set to enter mass production in the second half of the year.