Eleven AI models graded AR scenes, and the best tracked human raters at 0.87
Six Duke University researchers set eleven commercial vision-language models the job of grading how right augmented reality content looks, and the best of them ordered scenes almost the way people did, at a Spearman rank correlation of 0.8695. The version on arXiv went up on September 30 and is a preprint, though its arXiv entry says the work is to be published at VRST 2026, the ACM symposium on virtual reality software and technology that runs in Sendai from November 16 to 18.

A library illustration of glowing letters falling towards a circular mirror. The paper names neither the eleven models it tested nor the hardware its AR scenes were captured on.
That figure is about agreement on order, not accuracy. A Spearman rank-order correlation asks whether two raters sort the same set from worst to best in the same sequence, and 0.8695 says the model's ranking lined up closely with the humans'. It says nothing about whether the model ever matched anyone's actual score.
Placement, scale and shadows, rated by people first
The paper, by Elias Rotondo, Lin Duan, Yanming Xiu, Sangjun Eom, Conrad Li and Maria Gorlatova, introduces a benchmark called RateAR: AR images and videos "collected across diverse scenes and environmental conditions", judged on object placement, scale and shadow consistency. Human agreement on those judgements is reported at ICC(2,5) of .90 or better, which the authors describe as good-to-excellent reliability.
The reason to want a machine doing it is cost. AR head-mounted displays "contend with constrained scene geometry, spatial jitter, and temporal instability", the abstract says, and user studies, the usual way to find out whether a scene looks wrong, have a "cost, diminishing scalability, and inflexibility" that "pose bottlenecks during iterative application design". A model that ranks scenes the way a test panel would could sit in the loop while a developer is still moving objects around.
AR is being put to work in educational systems, healthcare delivery and risk-mitigation protocols, the authors say. Those are the settings where a badly anchored object stops being a demo problem, and where nobody wants to pay for a test panel on every iteration.
Then 21 people tried the automatic fix
The second half of the work builds an automated AR content adjustment system on top of the ratings and puts it in front of a user study of 21 participants. "More than 90% of participants found that the system improved placement and size coherence of virtual content," the abstract says. That is a stated preference from 21 people, which is not the same as a measurement, and the authors report it as such.
RateAR is the authors' own benchmark, built for this work, and it carries video as well as still images. The human ratings behind it are the yardstick: every model score reported here measures how closely the machine tracked what the people said, which is also the ceiling on what the benchmark can tell anyone.
An ablation found that the team's contextual prompting aligned better with human ratings than the other prompting strategies they tried, "while balancing introduced complexity cues". Which eleven models were tested, and which one reached 0.8695, the abstract does not say. Neither does it name the headset or glasses the AR scenes were captured on.
Vision-language models are being pointed at what a camera sees for people rather than for developers: Google launched Guided Vision in Gemini Live, saying it does not replace a white cane. Grading a developer's scene is a lower-stakes job, and a preprint with 21 participants behind its user study is where it currently stands.