Writing

Field notes · October 20, 2026

Projected Edge Inference Speeds for 27B Models in 2028

A community analysis estimates that Intel's forthcoming Razor Lake-AX platform could achieve approximately 46 tokens per second on dense 27-billion-parameter models by 2028, based on extrapolations from current edge hardware benchmarks.

Projected Edge Inference Speeds for 27B Models in 2028
Intel Confirms Razor Lake-AX 700 TOPS Launch in 2028

A recent discussion on the r/LocalLLaMA forum presents a rough estimate regarding the future inference capabilities of Intel’s announced Razor Lake-AX processor. According to WCCFTech, Intel has confirmed that the Razor Lake-AX architecture will deliver 700 TOPS (tera operations per second) at its planned launch in 2028. The cited source explicitly states that "Razor Lake-AX's 700 TOPS will far exceed the NVIDIA Jetson AGX Orin platform, which offers 275 TOPS." Drawing on this comparison, a contributor to the forum thread calculates the expected token generation speed for a dense 27-billion-parameter language model running locally on this hardware. The calculation relies on an observed baseline: the NVIDIA Jetson AGX Orin currently achieves approximately 18 tokens per second (TPS) when running a 27-billion-parameter model, a figure referenced via a shared conversation log. By applying a linear scaling factor—multiplying the baseline 18 TPS by the ratio of the new platform's compute capacity to the old platform's capacity (700 divided by 275)—the author arrives at an estimated throughput of roughly 46 TPS. The post further contextualizes this projection by noting that 46 TPS represents approximately double the performance of AMD’s current Strix Halo architecture, and observes that AMD’s anticipated Medusa Halo platform is also expected to double Strix Halo's performance. These details were published in a thread containing eighteen comments.

Intel Razor Lake Ax in 2028 ruff estimate about 46 TPS on dense 27b

The theoretical mechanism underpinning this estimate is the relationship between raw computational throughput, measured in TOPS, and the autoregressive decoding speed of large language models. During the decoding phase of text generation, a model must perform a forward pass through its neural network layers for every single token produced. For dense transformer architectures, the computational cost of generating each token scales predictably with the number of parameters and the precision of the weights used. Therefore, assuming memory bandwidth does not become a bottleneck, increasing the available integer or floating-point operations per second should proportionally decrease the time required to generate each token. The linear extrapolation used in the forum post—treating the jump from 275 TOPS to 700 TOPS as a direct multiplier for tokens per second—reflects this basic arithmetic relationship between compute availability and generation speed.

However, the limits of this evidence are substantial, and the projection must be treated as a heuristic rather than a guaranteed specification. First, the linear scaling assumption ignores the memory bandwidth wall. Autoregressive decoding of large language models is frequently memory-bound rather than compute-bound. A 27-billion-parameter model requires significant memory to store its weights, and if the memory bus of the Razor Lake-AX platform cannot feed data to the compute units fast enough, the additional TOPS will remain underutilized. Second, the baseline figure of 18 TPS on the Jetson AGX Orin is drawn from an external conversation link rather than a peer-reviewed benchmark, introducing uncertainty regarding the specific quantization level, software stack, and context length used during that test. Third, architectural differences between NVIDIA’s GPU-based approach and Intel’s NPU or hybrid CPU-GPU designs mean that TOPS are not universally fungible; the efficiency with which different architectures translate theoretical operations into actual matrix multiplications varies widely. Finally, the estimate is explicitly labeled a "ruff estimate" by its author, projecting hardware performance four years into the future based on preliminary announcements.

Despite these evidentiary limitations, the trajectory described in the source material carries significant implications for teaching and learning with artificial intelligence, provided the projected hardware capabilities materialize. Currently, deploying a 27-billion-parameter model locally yields speeds that are functional but often sluggish for real-time interactive dialogue. Eighteen tokens per second is slightly faster than average human reading speed, but it introduces perceptible latency during extended educational exchanges. If edge devices in 2028 can reliably produce 46 tokens per second, the interaction paradigm shifts fundamentally. At 46 TPS, text generation significantly outpaces student reading speed, eliminating the mechanical friction of waiting for a local AI tutor to formulate responses. This transforms local AI from a batch-processing tool into a genuinely conversational partner capable of sustaining rapid, iterative pedagogical dialogues without reliance on cloud infrastructure.

This shift toward high-speed, local inference directly impacts the feasibility of privacy-preserving educational technology. Schools and districts are increasingly constrained by data protection regulations that complicate the use of cloud-hosted generative AI. Running sophisticated 27-billion-parameter models entirely on local hardware circumvents the need to transmit student queries, drafts, and personal data to external servers. If the hardware required to do this smoothly becomes standard in educational devices by 2028, institutions could deploy highly capable, domain-specific AI tutors that operate entirely within the school's physical network. Furthermore, the parity suggested between Intel’s projected Razor Lake-AX and AMD’s anticipated Medusa Halo indicates that this capability will likely be competitive and widespread across hardware vendors, rather than locked behind a single proprietary ecosystem. For instructional designers, the assurance of low-latency, offline AI means that interactive learning modules, real-time formative assessment agents, and dynamic simulation environments can be built with the expectation that the underlying language model will respond instantaneously, regardless of the institution's internet connectivity. The progression from 18 to 46 tokens per second on edge devices thus represents more than a hardware milestone; it marks the threshold at which fully autonomous, private, and responsive AI tutoring systems become technically viable for widespread classroom deployment.

Source: Intel Confirms Razor Lake-AX 700 TOPS Launch in 2028

Read the original