Posts Tagged "llm"

The token gap

Last October, ten billion lifetime tokens earned developers a public tribute at OpenAI DevDay. This June, one Umans AI user ran nearly eleven billion tokens in a single day. What that says about token demand, compute supply, and the serving efficiency that has to close the gap between them.

Tokenomics, behind the scenes

Put the exact same model on GPUs and its token production cost can vary by multiples depending on how you serve it. A walk through the economics of a token: the frontier, the dollars it turns into, the levers that move it, and the claim our measurements let us make.

DeepSeek V4 Pro DSpark: the model isn't ready, the architecture is

We served DeepSeek V4 Pro for two days. What we wanted was hands on the architecture: attention that scales almost linearly instead of quadratically, and a speculative decoder that adapts to load. Both delivered. The preview checkpoint did not.

GLM-5.2 NVFP4: fast, cheap, and not worth serving

We put an NVFP4 build of GLM-5.2 in front of real users for four days, at 200+ tokens per second. Users called it intoxicating. We retired it anyway: if the tokens are not useful, they are not worth serving, no matter how efficiently we can produce them.

GLM-5 vs Kimi-K2.5: Long-context serving at scale

Why GLM-5 is a real step forward compared to GLM-4.7 for serving long-context coding agents, and why we still keep Kimi-K2.5 as the default for the best experience.