From London: as Australians woke this week, a relatively quiet announcement out of Boise, Idaho was quietly reshaping the economics of artificial intelligence infrastructure. Micron Technology confirmed on 3 March that it had begun shipping customer samples of the world's first 256GB SOCAMM2 LPDRAM module, a product built on the company's new monolithic 32Gb LPDDR5X die. The memory chip market does not generate the same headlines as a central bank rate decision, but for the operators of AI data centres, this development matters as much as any interest rate cut.
The new modules allow AI system builders to configure servers with 2TB of LPDDR memory across eight memory channels from a single CPU. That figure represents a one-third increase in capacity compared to Micron's previous 192GB SOCAMM2 modules. Put plainly, a server that once required multiple processors to hold a very large AI model in memory can now manage that task from a single socket, with implications for both hardware costs and energy bills.
The new modules offer roughly 66 per cent better power efficiency compared to standard RDIMMs, and they are compatible with the liquid cooling architectures increasingly required in high-density AI server racks. SOCAMM2 consumes one-third of the power of equivalent RDIMMs while using only one-third of the physical footprint, improving rack density and reducing the total cost of ownership. For the operators of large-scale data centres, those two factors, power draw and floor space, represent the dominant ongoing cost pressures of the AI buildout.
Micron's internal testing cites 2.3 times faster time-to-first-token for long-context LLM inference and three times better performance per watt in standalone CPU high-performance computing workloads. Those results are based on real-time inference using the Llama3 70B model at FP16 quantisation with 500,000-token context length and 16 concurrent users, comparing 0.12 seconds of latency for the 2TB configuration against 0.28 seconds for 1.5TB. Micron's own testing always warrants scrutiny, but independent validation of similar performance claims in adjacent product categories has broadly held up.
The SOCAMM2 form factor is the product of a partnership between Nvidia and memory makers Micron, Samsung, and SK Hynix. The original SOCAMM standard was designed by Nvidia, but the company reportedly encountered thermal problems running the modules at high density. Nvidia's chief executive Jensen Huang brought the memory manufacturers in, resulting in the SOCAMM2 specification with growing density and lower power consumption.
The power efficiency argument is worth taking seriously on its own terms, quite apart from performance. The energy intensity of AI data centres has become a genuine political flashpoint across Europe and increasingly in Australia, where state governments are fielding planning applications for large-scale compute campuses. A memory technology that claims to cut per-rack power draw by two-thirds, if those claims bear out at commercial scale, would meaningfully change the cost and carbon calculus of those projects.
There are legitimate reasons to be measured in enthusiasm. Micron is sampling, not mass-producing. The company plans to achieve higher-volume SOCAMM2 production in line with customer launch schedules, a phrase widely understood to mean alignment with Nvidia's own roadmap for next-generation AI server products. The gap between a sampling announcement and broad commercial availability can stretch for many months, and pricing for leading-edge memory has risen sharply; UBS has raised its price target for Micron to US$475, highlighting strengthening pricing dynamics in the DRAM and NAND sectors, with shortages expected to persist into 2028.
Micron continues to play a leading role in the JEDEC SOCAMM2 specification definition and maintains technical collaborations with system designers to drive industry-wide improvements in power efficiency. Once SOCAMM2 becomes an official JEDEC memory specification, other DRAM manufacturers will be able to launch competing modules, which should eventually bring prices down and widen access beyond the largest hyperscalers.
For Canberra, the implications are indirect but real. Australia's ambitions in AI infrastructure, from the federal government's investment framework through to state-level data centre precincts, depend on the cost trajectory of the underlying hardware. A genuine step-change in memory efficiency at the component level is the kind of development that, over a two-to-three-year product cycle, filters through into the economics of locally hosted compute. Whether Australian policy settings are agile enough to capture that opportunity, rather than simply import its benefits via overseas-owned facilities, is the more pointed question.