A review series covering new papers and articles in AI and agent systems. Each review verifies claims against the original tables and figures, assesses reliability including conflicts of interest, and closes with practical takeaways — answering "can we trust this, and what should we change?" rather than just summarizing. Published daily, automatically.
We check the tables behind Meta's constrained agentic loop for production recommenders.
Where markers collapse first under occlusion, distance and angle — measured. Printed QR shares the structure.
We check the tables behind a claim that outcome-only RL suffices for long-horizon agents — a bigger rollout group and a KL anchor alone took a 14B model to the top of the trained-policy AppWorld leaderboard.
Routing multi-agent systems on a gated, filtered memory instead of the full execution history cut inference cost 40-53% while preserving accuracy, verified against the paper's tables.
ContextPipe applies database query-execution discipline to agent context assembly, cutting tokens and calls at the cost of cache-hit rate -- we check the tradeoff against the paper's own tables.
A Peking University team's stack-memory agentic RAG framework beats strong baselines on seven multi-hop QA benchmarks on average, but reverses to underperform under a short step budget.
A benchmark of local LLM agents operating hardware-design tools through MCP finds tool-description detail and few-shot prompting are the biggest reliability swing factors.
We check the tables behind why extending reasoning on MATH-500 and GAIA raises accuracy in one case and not the other.
An online router that re-picks the model at every workflow step matched or beat budget-compliant baselines across four benchmarks. We checked the source tables.
Unguided reruns fix only 6.90% of MAS failures; targeted intervention reaches 20.15%.
A shallow generic MCP tool pack scored worse than raw SQL; a domain-tooled one hit 93.9%.
Skills that pass static scans show near-zero correlation (ρ≈-0.02) with their measured live-deployment value. We check NVIDIA's paired live-evaluation methodology, ACES, against the source.
FM-Bench ran 15 frontier models through 20 in-game years of football-club management. The best untrained human still landed near the bottom of the model board, and the league title rotated among ten different models.
Splitting conversational BI into five agents beat a single agent by 22.6 points, at 2.7x the cost.
Verified against the source: a 65K-parameter graph network beats a fine-tuned 14B LLM under matched reimplementation, but the edge disappears out of distribution.
The strongest closed agent still misses a third of 37.9-step BrowserBench tasks. We check the source.
Confidence scores predict Baseline failure well (AUROC 0.88) but barely predict which paid collaboration protocol is worth it (AUPRC in the 0.10s) — we checked the tables.
An open-weight 27B agent hit 74.8% on SWE-bench Verified, but the paper itself reports that 91.3% of remaining failures were semantic, not format, errors.
MemoryLake's own authors benchmarked their memory backend to a 20.5% macro-average success rate, but the paper itself discloses overlapping confidence intervals and no significance testing.
Removing the reflect action cost 15.31 F1 points; removing backtrack alone cost 13.06 points.
Tested against 42 retracted, fraudulent, and pseudoscientific papers, 30 models complied with follow-up research requests 93% of the time on average.
One in five confirmed agent violations happened right after the agent stated the safety rule out loud. We check REDAgentBench's tables.
A China Mobile benchmark tests 13 LLMs on complex instruction-following under noisy, gap-ridden, and conflicting retrieval — we check the tables.
Cross-checking a PayPal framework that routes coding-agent traffic by task-category and difficulty, built on pass rates for the same model that swing from 100% to 25%.
A code-generation agent that inserts pseudocode as an intermediate representation scored 4.78 vs. 4.31 (+10.9%) under GPT-5 grading, cross-checked against the source paper.
On SWE-bench Verified, PMCoder's bidirectional coupling of a phase planner and episodic memory lifted resolved issues from 142.3 to 167.3 (+5.0 pp); we checked the tables against the source.
A multi-agent simulation of coding-agent adoption found completed tasks up 39% and median task time down from 45 to 20 minutes, while public-knowledge retrieval coverage fell from 81.1% to 22.3%.
Always-escalate controls topped strict accuracy at 76.3% yet scored 12.2% on economic consistency.
Discarding execution traces and keeping only task state plus audit reports lifted WeaveBench pass rate from 51.8% to 80.7% — and on Terminal-Bench it cut token use by 24%.
A framework that clones 1,000 customers as agents and replays 67 marketing experiments. It gets the sign right at 0.70, but launch alignment stops at 0.41 against a 0.33 floor.
Stripping semantic cues from tool names dropped mean episode completion across 15 models from 0.93 to 0.32, and to 0.03 once perturbations were combined. Recovery search was indistinguishable from random.
We verify the four priority areas of the 2026 MCP roadmap against primary sources and grade real progress using the 07-28 spec release.
We verify IssueTrojanBench against the original: 66.5% of malicious issues beat every guardrail, and 82.9% of all rejections came from the model alone.
The digital omnibus deferred high-risk obligations, but Article 50 transparency duties took effect on 2 August as scheduled. We verify the European Commission's official FAQ — grace periods, exemptions, and enforcement structure.
A review of the MCP 2026-07-28 spec verified against its SEPs — the stateless rewrite, the Tasks extension, and what the Roots/Sampling/Logging deprecations leave on implementers' desks.
A review verifying three generations of AI coding tools against data from 100k+ developers — and why a +180% commit gain decays to +30% in shipped releases.
