OpenAI says its own models optimised Astra Ultrafast's inference on Nvidia GPUs

OpenAI used its own models to tune the inference software that serves GPT-6 Astra Ultrafast on Nvidia hardware, two of its executives say on the record in a post Nvidia published on October 1. "We used our internal models to optimize inference on NVIDIA GPUs, and NVIDIA's programmability helped us deliver the acceleration behind Astra Ultrafast," says Uday Ruddarraju, chief technology officer of compute at OpenAI.

Ultrafast is served from Nvidia Blackwell GPUs, according to Nvidia.

Ultrafast is the fast service tier for GPT-6 Astra. Nvidia says it "offers up to 8x faster token generation than the Astra Standard mode", runs on Nvidia Blackwell GPUs, and is "available now in the OpenAI API and to eligible ChatGPT Work and Codex users".

The 8x is a vendor figure for one thing only

That number is Nvidia's, published on Nvidia's own blog by Dion Harris, and it covers token generation rather than the time a request takes from end to end. No benchmark, prompt, model size or batch configuration is given with it, Nvidia does not say which Blackwell part the comparison ran on, and the "up to" is doing the usual work. Nvidia's artwork for the post shows a Standard run at 17.0 seconds against 6.4 seconds for Ultrafast, which is not the 8x figure and comes with no description of the task.

OpenAI's own description is narrower in a useful way. Its API changelog entry of September 29 says the tier exists "to reduce the time between generated output tokens", which is the gap between words appearing rather than the wait before the first one.

Price has already been set against it: Ultrafast costs $300 per million output tokens, six times the standard rate, so the speed is bought rather than handed over.

What OpenAI's engineers claim their models did

Philippe Tillet, inference lead at OpenAI, puts the work on the GPUs themselves. "NVIDIA's deep investment in tooling and documentation has enabled us to make our models exceptionally good at programming Blackwell and Rubin GPUs," he says, adding that "Astra can turn that knowledge into high-performance kernels that make NVIDIA hardware compelling across the full frontier of latency, throughput and cost."

Nvidia's own framing of it is that "OpenAI is using its own models to help refine the inference software running on NVIDIA GPUs, taking advantage of the platform's programmability to test and implement improvements", and that the work continues after a model is deployed.

Both quotes are about one company's inference stack on one vendor's accelerators, after what Tillet credits to Nvidia's own tooling and documentation. Neither says anything about how much of the stack was written this way, how the output was reviewed, or what share of the speed-up came from it rather than from the hardware, and nobody outside the two companies has measured any of it.

Who can actually reach the tier

For developers, Ultrafast is the model "gpt-6-astra" called with a service tier of "ultrafast" in the Responses API, and OpenAI says it is "available to API customers, subject to rate limits, with global processing and US data residency. EU and other regional inference residency aren't supported." A team in Germany that needs inference to stay in the EU is therefore not served by it, whatever the generation speed. Nvidia adds the practical case for it: faster generation "can shorten coding agents' edit-test-debug cycles" and cut the wait between tool calls, which is where a developer would notice the difference rather than in a single answer.

Tillet names Rubin, Nvidia's generation after Blackwell, alongside it, though Nvidia names only Blackwell as serving Ultrafast. Neither company has published the test behind the 8x, and OpenAI has not said whether the same model-written optimisation work feeds its cheaper tiers.