SK hynix has unveiled a new high-bandwidth memory (HBM) technology that dramatically improves the heat dissipation problem, the biggest hurdle facing artificial intelligence (AI) chips. The new technology, called "iHBM," embeds cooling elements inside the package, overcoming the structural limitations of conventional HBM, whose performance degrades due to heat generated during massive computing operations. SK Group Chairman Chey Tae-won will meet again with Nvidia CEO Jensen Huang early next month, leveraging the new technology to strengthen their HBM alliance.
SK hynix on the 26th unveiled the "iHBM" technology, which features an "Integrated Cooling Element (ICE)" inside the HBM package. The ICE uses silicon material with high thermal conductivity to act as a kind of "cooling tower" that discharges internally generated heat to the outside. The iHBM places the ICE at the bottommost "die-to-die physical layer (D2D PHY)," where heat concentrates, forming a separate heat-dissipation pathway. This reduces thermal resistance by more than 30% compared with existing HBM, enabling stable operation even under high-temperature, high-load conditions.
Nvidia's next-generation AI accelerator, Rubin Ultra, boasts computing performance of about 100 quadrillion operations per second, or 100 petaflops. This is expected to consume up to 230 kilowatts (kW) of power per rack, about 64% more than the current Blackwell GPU (up to 140 kW per rack). Nvidia has also disclosed development plans for "Feynman," the successor to Rubin Ultra. For existing HBM to support Feynman's performance, the heat issue arising from computing operations must be resolved. SK hynix has presented iHBM as the solution.
The industry has evaluated iHBM as a technology that solves the chronic heat problem without disrupting existing design flows. Choi Jae-hyouk, a professor in the Department of Electrical and Computer Engineering at Seoul National University, said, "AI chips have evolved by shortening the distance of the data pathway (D2D PHY) between the internal computing unit (SoC) and the memory (HBM) to reduce power consumption during high-speed signal transmission." He added, "In this process, an empty space without core (memory) dies remains above the data pathway, and the idea of erecting a cooling pillar there to dissipate heat is a very excellent attempt."
SK hynix plans to secure mass-production capability by utilizing already-proven packaging processes. The application of iHBM technology makes HBM manufacturing processes more complex. To address this, SK hynix will produce iHBM using the wafer-level packaging (WLP) process based on "Mass Reflow Molded Underfill (MR-MUF)," which is applied to existing products.
In this case, mass production can be established immediately without introducing new equipment, securing stable production and yields. Using existing processes also enhances compatibility with customers' existing system-in-package (SiP) designs, allowing big tech customers to immediately adopt iHBM without design modifications.
SK hynix plans to introduce iHBM starting from the 8th-generation HBM (HBM5) to meet the demanding requirements of next-generation graphics processing units (GPUs) from its key customer, Nvidia. Chairman Chey plans to meet CEO Huang at "GTC Taipei 2026," held in conjunction with Computex in Taipei, Taiwan on the 1st of next month, to closely discuss advances in AI chips and HBM technology. Chey is also expected to meet TSMC Chairman C.C. Wei to strengthen the trilateral alliance among SK hynix, Nvidia and TSMC.
