The center of gravity in artificial intelligence (AI) competition is shifting rapidly from "training" to "inference." The era of training — teaching bigger models with more data — has passed, and the era of inference, which measures how quickly and accurately AI answers our queries, has arrived. Nvidia CEO Jensen Huang recently declared "an inflection point for AI inference" by incorporating inference-dedicated technology (Rubin CPX) into the company's next-generation Vera Rubin platform — a symbolic illustration of this shift.
The global AI inference chip market is projected to grow at an average annual rate of more than 30 percent through 2030, yet no dominant player has yet emerged. This represents an opportunity for Korean companies, which have already built up proprietary neural processing unit (NPU) technology and system capabilities. Victory in this new battleground will no longer be decided by a single chip, but by the competitiveness of the entire infrastructure system encompassing servers, data centers and networks. The inference era is defined not by raw computing power, but by domain-specific "specialization," "power efficiency" and "low latency."
While the conventional graphics processing unit (GPU) excels at massive parallel computation, the NPU — on which Korean companies are focused — delivers far higher inference throughput per watt, thanks to a design specialized for the neural network computations AI uses to see, hear and make judgments. As a result, a division of roles is solidifying in which GPUs handle training while NPUs handle real-time response generation. A robust foundation for AI inference infrastructure is completed only when this is combined with Compute Express Link (CXL), which optimizes data movement between chips, and the data processing unit (DPU), which intelligently handles data movement and security between servers.
The inference market is now expanding beyond massive data centers into "on-device terminals" in our daily lives. In particular, as demand surges for so-called "physical AI" — in robotics, AI manufacturing and autonomous driving, which interact directly with the real world — the role of AI chips that make instant judgments on site has become more important. Fortunately, Korean companies are producing meaningful results. Rebellions and FuriosaAI are entering domestic and overseas data center markets, while DeepX and Mobilint are moving closer to unicorn status in the edge segment.
But an industry cannot be built on the strong performance of a handful of companies alone. Scale-up of NPUs, advancement of CXL and DPUs, and a "full-stack optimization" development strategy that links them through large-scale testbeds and open system software must be pursued in parallel. Support must also be provided so that these technologies can build a base of real-world validation and be linked with public-sector demand. The government has also launched its AI chip industrial strategy. The key is the execution to translate this momentum into results on the ground.
AI chips in the inference era are a national strategic asset and a matter of security. AI inference is directly connected to industries and services including manufacturing, finance, healthcare and transportation. Because supply chain disruptions or technology controls can instantly translate into chaos on the industrial front line, relying solely on foreign platforms is risky. Without independence in core inference infrastructure, AI sovereignty cannot be fully preserved. Only when Korea's AI models and network capabilities are combined with a solid AI chip ecosystem to form "comprehensive competitiveness" will the nation be able to stand as a leader, rather than a follower, in the AI inference era.
