Source Document
Muhammad Rafay Azhar, Yuhang Zhou, Gilbert Jiang, Yuchen Wang, Rahul Sharma, Matthew DeSousa, Jiayi Liu, Xin Guo, Lizhu Zhang, Xiangjun Fan, "CORAL: An LLM-Native Harness for Production Recommender Systems", arXiv:2609.02730 [cs.CL], submitted 2026-09-02, DOI 10.48550/arXiv.2609.02730. Affiliation: Meta AI.
This is a preprint without peer review, closer to an industry case report than an academic study. The conflict of interest is stark: all 10 authors work at Meta, and the two production services under test are Meta's own — no external audit or independent replication is possible. We verified the full text against a snapshot taken 2026-09-05T22:15:32Z: this session's egress proxy blocked every outbound domain including arXiv (a control fetch to example.com failed identically), so we used a copy of the source HTML that GitHub Actions had already fetched.
Study Overview
Production recommenders carry many hand-tuned parameters — retrieval budgets per source, ranking weights, serving and caching policy — and the optimal setting keeps moving as content and behavior shift. Human-run experiment cycles are too slow to keep up, so the system drifts from its best operating point.
CORAL (Constraint-Optimized Recommender via an Agentic Loop) puts an LLM in that loop. On a 3-day cadence (k=3), each cycle the agent observes the prior window's signals, pulls past decisions and their measured effects from a three-store memory (observation, assessment, decision; m=3 cycles retained), and proposes a new configuration. The proposal is never applied directly — a separate numerical optimizer projects it onto the budget-feasible set first — and the A/B-measured outcome is written back to memory, so the policy adapts in context with no retraining. The harness was validated on two different social-platform surfaces: a continuous decision (retrieval-budget allocation across sources) and a discrete one (serving-treatment selection per user segment).
Key Results
The first case study (a video-recommendation service's retrieval-budget allocation) redeployed the same loop over three successive rounds, each measured with its own A/B experiment. R1 is a zero-shot proposal from a single window of statistics; R2 shifted budget more aggressively and overcorrected; R3 is the converged, deployed configuration after several further cycles.
| Metric | R1 (zero-shot) | R2 | R3 (converged) |
|---|---|---|---|
| Watch time | +0.13% | neutral | +0.15% |
| Sessions (all users) | neutral | neutral | +0.16% |
| Sessions (largest market) | neutral | neutral | +0.77% |
The authors read the non-monotonic R1→R2→R3 progression — a regression in R2 — as evidence the loop is working as intended: it corrects the next decision based on the measured effect of the last one, rather than winning every round. R3's gain came from reallocation alone, at no additional serving cost. A separate, segment-specific allocation for new, low-signal users lifted their sessions by +0.23% (a different condition, not directly comparable to the table above).
The second case study — serving-capacity allocation on a different service — is a discrete choice problem. Round 1 cut serving cost substantially on a subset of segments ("millions of USD" in annualized savings, no exact figure given); round 2 widened the change to the remaining segments, growing the savings by 44% with engagement statistically unchanged. Both studies report only "A/B experiments spanning millions of users," so sample sizes, confidence intervals, and the significance threshold behind "significant" vs. "neutral" are not stated.
| Case study | Calls per cycle | Input tokens/call | Output tokens/call |
|---|---|---|---|
| Retrieval-budget | ~10 | ~1,500 | ~2,500 |
| Serving-capacity | ~8 | ~2,000 | ~2,500 |
The paper states these figures are estimated from prompt and payload sizes, since the pipelines do not log token usage. Cumulative tokens across a deployment are on the order of 106, putting end-to-end inference cost at an estimated "tens of U.S. dollars" per deployment at representative frontier-model pricing. Because calls per cycle are fixed regardless of traffic, the paper emphasizes that cost is independent of service scale — unlike per-user billing.
Credibility Assessment
What earns trust: these are real production A/B experiments, not simulations, so the measured effects reflect real user behavior; and the same harness worked across two services and two decision types (continuous and discrete), which argues against a trick overfit to one system. Reporting R2's regression rather than hiding it also adds credibility.
What to weigh: every author and every system under test belongs to Meta, a conflict of interest that rules out external replication or audit. Absolute figures ("millions of users," "millions of USD") are kept vague, making it hard to judge how large the relative figures (0.16%, +0.23%, 44%) actually are. No significance threshold or confidence interval is given for "significant" vs. "neutral." The cycle cadence (k=3 days) and memory horizon (m=3 cycles) are stated as "sensible defaults" rather than tuned, with no ablation showing what changing them would do. Both case studies are the same class of decision — resource allocation — so, as the paper itself notes, whether the approach transfers to qualitatively different levers such as ranking logic remains untested. The work has not been peer-reviewed.
Contrary/contextual evidence: the related-work section cites similar industrial agent-harness efforts, including AgentX (agent-driven self-iteration of industrial recommenders, arXiv:2606.26859) and Nova (a verification-aware agent harness for architecture evolution, arXiv:2606.27243). We did not independently verify those papers, but that several companies appear to be pursuing similar directions makes it less likely CORAL's result is a one-off specific to Meta.
Reviewer's Judgement
First, the authors present the non-monotonic R1→R2→R3 progression as evidence the loop "learns," but the paper never states what specifically changed in the agent's reasoning between R2 and R3. From the table alone, we cannot distinguish memory-based attribution actually working from the agent simply trying another allocation within the same budget and getting lucky.
Second, the claim that "cost is independent of scale" is true by design given fixed calls per cycle, but the cost figures here are estimates from prompt size, not measurements. Operational overhead that doesn't show up in pure token pricing — retries, tool-failure handling, monitoring — is likely missing, so teams evaluating adoption should measure rather than budget directly against the "tens of dollars" figure.
Third, deltas like 0.16%, 0.23%, and +0.77% are plausibly only detectable with statistical significance at Meta's traffic volume. A much smaller service adopting the same harness has no guarantee of measuring a comparable gain with comparable confidence, so applying these results at a different scale calls for caution.
Putting It to Work
- Separate proposal from budget enforcement — don't apply the LLM's allocation directly; only the value a deterministic optimizer projects onto the budget-feasible set gets deployed.
- Structure memory into three layers — observation, assessment, and decision stores kept separate so each cycle has a clear basis for attribution.
- Set cadence and memory horizon from observability, not intuition — pick defaults based on how fast effects surface and how quickly outcomes go stale, then adjust from measurement.
- Gate every change on A/B — even an autonomous loop shouldn't hand supervision to automated guardrails without per-cycle A/B verification.
- Verify cost by measurement — keep the per-cycle-fixed cost design, but log actual token usage instead of estimating from prompt size.
Conclusion
CORAL's contribution isn't a new algorithm but the structure itself: automating, under human supervision, an experiment-and-adjust cycle that used to take engineers weeks, using an LLM with memory paired with a deterministic optimizer. That the same harness improved engagement on one service and serving efficiency on another is evidence against a one-off trick. Given the conflict of interest, the vague absolute figures, the missing ablations, and a validation scope limited to one class of decision, teams should re-verify with their own A/B before porting this to a different organization or decision type. For codifying budget and stop rules directly into a loop, see Budget and Circuit-Breaker Design for a Callbot Loop, which carries the idea into a different domain.