The Strongest Model Was the Least Safe: Reviewing ReactHuman

Source Document

Yizhan Li, Jianxin You, Mengyang Xiong, Yinhuan Chen, Zicheng Zhao, Dekun Wu, Dongqing Zhang, Bang Liu, "ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs", arXiv:2609.10895 [cs.RO], submitted 2026-09-09, DOI 10.48550/arXiv.2609.10895, CC BY 4.0. Affiliations: Université de Montréal, Mila – Quebec AI Institute, McGill University, McMaster University, Meta Platforms. Benchmark dataset released (Hugging Face: Alan123/reacthuman-benchmark-scaled).

This is a preprint without peer review, classed cs.RO. Most authors are at Canadian universities and institutes (Université de Montréal, Mila, McGill, McMaster), and funding is credited to Canada's NSERC Discovery Grant, a public research fund. One co-author is affiliated with Meta Platforms — a disclosed conflict of interest, though none of the seven evaluated models is a Meta model, which limits the incentive to skew results. The full text was verified against a snapshot (collected 2026-09-11T23:22:12Z, labeled 2026-09-12 KST — two days before today). Session egress was blocked even to a control domain (example.com) after three retries at 10-minute intervals, and today's own snapshot collection failed twice (automatic and manual) with arXiv API HTTP 429, so the most recent untruncated snapshot was used instead.

Study Overview

The research question is singular: can a multimodal LLM (MLLM) convert its understanding of a sudden physical hazard — a slipping plate, a falling knife — into an immediate, safe action? Existing benchmarks either probe physical understanding passively through video question answering, or evaluate only deliberate, long-horizon tasks such as navigation and tidying, leaving this question unanswered.

The method is a "freeze-and-predict" protocol. The Genesis physics engine simulates an event at 240Hz and freezes at the decision moment; the model outputs a structured reaction plan made of intent, confidence, a walking command, and a hand trajectory. The plan is not graded on paper — it is executed by a simulated humanoid (Unitree G1), producing a physical outcome (catch or miss, dodge or hit). Evaluation spans 17 sudden-event families, over 1,000 reproducible scenes, and 14 adversarial objects whose appearance contradicts their physics (a foam anvil, a steel apple, and others). On the 306-scene evaluation set the Catch/Dodge/No_Action label split is 141/156/9, so a majority-class baseline (always Dodge) scores 51.0%.

Key Results

Seven MLLMs (Claude Opus 4.8, GPT-5.5, Gemini 2.5 Flash, Kimi K2.6, Qwen3-VL-235B, Qwen3-VL-30B, Gemma-3-27B) were evaluated under identical conditions via OpenRouter. Across 306 scenes × 7 models, four decisions were lost to API errors, leaving 2,138 analyzed.

Three patterns stand out. First, safety-rule violations on scenes whose correct response is Dodge average 35.9% (23–66% by model), and most mistakes (253 of 392, 64.5%) are simply freezing in place. Second, each model has a fixed disposition — Claude Opus 4.8 catches well (67% on catch scenes) but engages 34% of dodge scenes it should avoid, landing last on safety (64.7%), while the other six models skew toward retreat instead. Third, scale does not buy safety: the 27B Gemma-3-27B ranks second on accuracy (62.4%) and first on safety (88.2%), beating the flagship Claude Opus 4.8.

Majority voting beats the average model but not the best single model — the models share the same blind spots. Across 280 adversarial-object decisions, no model ever questioned an object's appearance-defying material.

Credibility Assessment

What earns trust: ground truth is deterministic from 240Hz rigid-body simulation with no subjective labeling, scenes are fully reproducible from a seed, and every plan is physically executed — surfacing failures like "picked the right action, missed by hand" that a multiple-choice test cannot reveal. Caveats: this is a pre-peer-review draft; rigid-body simulation excludes deformation and shattering; decisions are single-shot with no mid-execution replanning, as the authors themselves note. The Action–Intent Alignment metric is scored by keyword matching, which may be coarse; the adversarial-object test rests on only 280 decisions; and the evaluated models are point-in-time API snapshots, a limitation the authors also flag. The finding that "bigger is not safer" runs against the common observation that scale helps text-based harm-refusal safety — but that measures a different axis (embodied reactive safety), so this looks like a different capability rather than a direct contradiction.

Related Work (Academic Cross-Check)

• Chow et al. (2025). PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding — Prior work. ReactHuman's own related-work section calls this its "closest neighbor"; it measures physical understanding via multiple-choice VQA. ReactHuman swaps the answer for an executed plan, surfacing the "right action, unreachable hand" failure for the first time.
• Sferrazza et al. (2024). HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation — Prior work. Evaluates whole-body humanoid control across 31 tasks, all deliberate and long-horizon. ReactHuman holds controller quality fixed and isolates only the decision layer for sudden, time-critical reactions.
• Tung et al. (2023). Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties — Prior work. Passively tests inference of latent properties like mass and friction from observation. ReactHuman checks whether that inference actually translates into a safe, physically executed action.

All three measure "understanding" through question-answering or property inference; ReactHuman instead executes to check whether that understanding becomes safe action. The fixed dispositions, binary speed perception, and reach errors found here are failure modes the prior methods could not have observed — an extension of this line of work, not a contradiction of it.

Reviewer's Judgement

First, that most safety violations (39.5% of 410) are simple freezing is, in this reviewer's reading, the most operationally important finding. Models fail by deferring a decision rather than actively choosing the wrong one, so mitigation should look less like better reasoning and more like a timeout design: switch to a predefined safe default if no confident plan arrives in time.

Second, "picked the right action, missed by hand" reads as a split between an MLLM's linguistic judgment and the geometric computation robotic control actually needs. That 49% of these misses happened because the robot stood beyond arm's reach (0.8m away) means the model reached the correct conclusion but never converted it into an executable trajectory. Treating MLLM output as intent only, with reachability checked by a separate geometry layer, looks like the safer design.

Third, that Claude Opus 4.8 — a model that ranks highly on general capability evaluations — scores lowest on safety here argues against using general capability rankings as a proxy for physical-deployment safety. Where safety is on the line, a dedicated reactive-safety evaluation should be its own gate.

Putting It to Work

• Decision timeouts with a safe default — if no confident plan arrives within a time budget, switch to a predefined safe action (e.g., retreat) instead of freezing.
• Pre-execution geometric feasibility checks — separately compute distance and reach for a hand trajectory or walking command before executing it, filtering out plans that are verbally correct but physically impossible.
• Stop treating speed as binary — inject speed and time-remaining as explicit numeric values so gradual hazards register as events.
• Test for appearance-driven physics errors — where material or weight is observable, check separately whether the model can override a visual prior with it.
• Separate capability and safety gates — require a dedicated reactive-safety evaluation before deployment where physical safety is at stake, independent of general capability rankings.

Conclusion

ReactHuman is the first benchmark to put a number on the gap between "understood" and "safely executed." The finding that the strongest model was the least safe, that most safety violations come from deferred decisions, and that a correct choice can still miss by hand — none of this shows up in a multiple-choice test; all of it only appeared once plans were physically executed. Given the pre-peer-review status, some coarse metrics, and modest sample sizes, treat the numbers as a directional signal and run your own reactive tests before deployment. For the operating principle of gating capability and safety separately, see Passing a Capability Benchmark Is Not a Safe-Deployment Signal.

References

• ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs — arXiv abstract (source)
• Full HTML text of the same paper — used to verify tables and figures (via snapshot)
• PhysBench — related-work source
• HumanoidBench — related-work source
• Physion++ — related-work source
• Passing a Capability Benchmark Is Not a Safe-Deployment Signal — sunny34.com blog