Blog
Engineering notes.
Explainers from the team that runs the GPUs: how LLM inference is served, and where model design is heading.
Articles
-
LLM Inference Parallelism, Explained with Animations
An animated guide to LLM inference parallelism: data, tensor, pipeline and expert parallelism, and the attention designs (MHA, GQA, MQA, MLA) behind them.
Read the article -
Next-Concept Modeling (NCM), Explained with Animations
How a language model can write a paragraph in ~12 parallel steps instead of hundreds: a coarse-to-fine hierarchy of discrete codes, one level at a time.
Read the article