Black Forest Labs' 7B FLUX 3 Action tops Nvidia's own robot leaderboard at 42.9%
Black Forest Labs, the lab behind the FLUX image models, has put out a 7B open-weights robot-control model that now sits first on Nvidia's own robotics leaderboard at 42.9%. That is 515 of 1,200 trials for FLUX 3 Action, where Nvidia's best entry, the 16B Cosmos3-Nano-Policy, is fifth on 36.8%, or 441 of 1,200, on Nvidia's RoboLab-120 leaderboard as checked on September 28.
Black Forest Labs labels this footage FLUX 3 Action playing games. Game recordings account for 19.55% of the samples in the action-focused midtraining phase. Image: Black Forest Labs
Nvidia's own GR00T N1.6, a 3B vision-language-action model, does far worse: 13th with 7.2%, or 87 of the 1,200 trials, on the same leaderboard.
RoboLab-120 is a simulation benchmark of 120 generated tasks, put to a policy that gets camera frames, robot joint state and a text instruction, with each instruction worded three ways from vague to specific. The overall table scores 1,200 trials per policy. Black Forest Labs calls FLUX 3 Action a World Action Model, meaning it predicts future video frames and robot actions together rather than actions alone, according to the lab's own technical report.
Two checkpoints, two different scores
The numbers need keeping apart. The fast single-step checkpoint scores 38.3% on RoboLab, which the report compares with 36.8% for Cosmos 3 Nano, while the slower guidance-distilled checkpoint reaches 42.2% in the report, and the leaderboard lists its submission at 42.9%.
Speed is the pitch for the fast one. Black Forest Labs reports a 1.34x to 2.28x better real-time factor in FP8 than Pi0.5 on workstation and datacentre GPUs, and it predicts 2.13 seconds of motion per pass against 1.0 second for Pi0.5.
A fifth of the action training came from PC gameplay
The training mixture is the odd part, and it is where an image lab's habits show. Of all the samples in the action-focused midtraining phase, 19.55% come from game recordings and 13.54% from egocentric video with hand-pose annotations, with 14.03% from handheld grippers and 15.93% from teleoperation across 14 embodiments.
Game actions get their own 64-dimensional action space. Two dimensions encode mouse movement on the x and y axes, two encode the left and right mouse buttons, and the remaining 60 encode keyboard buttons, the report says.
A blind run on a real arm completed 28 of 30 attempts
The only hardware result is a third-party evaluation that Black Forest Labs reports itself. Positronic Robotics ran the policy on a Franka arm in its own lab across ten DROID tasks, three attempts each, alongside Cosmos3-Nano, DreamZero and Pi0.5, with the same 240-second window and the same scoring for every model.
The operator did not know which model was driving each attempt, and every attempt was filmed. FLUX 3 Action completed 28 of its 30 attempts.
The cost comparison in the report is the part a developer will read twice. GPT-6 Astra at maximum reasoning effort solves every task in a separate benchmark, Black Forest Labs says, but each success costs $13.47 and 16 minutes on average, while FLUX 3 Action on its own costs $0.09 and under two minutes per success on an H200 rented at $3 an hour. Letting Astra delegate to FLUX 3 Action and step in only when needed solves 90% of episodes at $8.77 per success.
Nothing here is a product. The weights are on Hugging Face under the FLUX Kommunity License, the model was uploaded on September 22, and Black Forest Labs announced it on September 23 as an open-weights 7B model in first place on RoboLab. The lab is blunt about the ceiling: at 42.9% the model fails most of the benchmark, and it writes that "there are still tasks that all policies, ours included, fail to solve on their own".
A leaderboard place is a leaderboard place, and this one was taken by a lab that makes pictures.
