ICLR 2027 · Submission35229 · Rebuttal 1 · Reviewer 3Rua

3Rua：怎么读、怎么回、要跑哪些实验

规划稿 v1（2026-11-08）。实验一个都还没开跑，表里的数全是占位；写死的数只来自投稿版 PDF（页码 p.N 指它）和审稿原文。算力和时间的估算来自 .note 里 0814–0818 的 RL 步时记录，没有重新测。

Rating 3 · Weak accept · 信心 34 条批评 = 同一个问题：功劳归谁实验未开跑 · 值为占位算力待定（今晚 7 台全有主）

How to read 3Rua

一眼看懂

TA 的立场

唯一的 weak accept。夸了四点：机制可证伪、对照设计、理论保留残差项、成本透明。四条批评全是“把功劳拆清楚”：BAPO 里是分组还是 decision reward；TTO 里是 ranking 还是多采样；world model 是不是说大了；理论里的量有没有量过。

我们的总方针

方法层的事实澄清写硬（Prop. 2 里没有“候选之间的 KL”；belief 里有 state summary）。结果层用新实验回答，不用解释代替数据。claim 收窄，但 “world model” 一词保留，正文给出精确定义。TA 问的每一个 “can you report …” 都给一张表。

回法图例（rebuttal skill 六种，按优先级）

① 直接澄清审稿人读错或漏看，给页码
② 已有证据论文里已有的结论
③ 最近工作差别
④ 新实验结果用占位，不编数
⑤ 有限让步只认具体那一点
⑥ future work

Item by item

逐条理解（W1–W4 / Q1–Q4）

W1 / Q1 · BAPO 没被单独拆开

④ 新实验⑤ 有限让步TE

TA 原话

“Fixing the prediction loss and decision reward while only changing the advantage construction would settle this.” “How much of the +1.8–3.0 remains?”

+1.8–3.0 在哪

p.7 §3.2 最后一句：training with BAPO adds 1.8, 2.6, and 3.0 points on top of the belief state at 7B, 14B, and 32B。就是 Table 1 里 ANTE 减 ANTE (w/o BAPO)：27.2−25.4、46.2−43.6、57.2−54.2。TA 问的是：把 decision reward 留下、只把“按 belief 分组的步级 advantage”换掉以后，这 1.8/2.6/3.0 还剩多少。

你的方案

decision reward 不进步级 advantage，而是按步平均后加进 outcome reward，再算 trajectory-level 的 GRPO advantage；7B/14B/32B 都跑，多 run 报 std。方向对：这正是 TA 说的 “same prediction loss, same decision reward, only the advantage construction changed”。两个细节要定：

细节 1 · 尺度

α=0.05、γ=0.0625、κ=0.02（Table 9），而任务奖励是 0/1。若按“各步 r_t 取平均、原样加进 R”，加进去的量只有 ±0.02～0.05，episode advantage 又是 mean-only 不归一化（Table 9），这个对照臂和 w/o BAPO 几乎分不开，审稿人会说“decision reward 根本没被给机会”。建议：用论文自己的量 W_i,1 = Σ_t r_i,t（整条轨迹的 decision return），在 task 内做和 Eq. 5 相同的标准化（floor α+γ、ω=½、clip ±2），然后广播到每一步：A_i = (R_i − mean R) + ω·std(W_i,1)。这样两臂“信号相同、尺度相同”，唯一的差别就是 credit 落在整条轨迹还是落在同 forecast 的那一步。取 mean 还是 sum 只差一个轨迹长度因子，用 sum 和 BAPO 的 W 定义一致。

细节 2 · 第二个对照

TA 质疑的字面是 “the claimed novelty of grouping by belief”。你的臂回答“步级 credit 有没有用”；还有一个更贴字面的臂：保留步级项，只把 V̂/Q̂ 换成 p.31 已定义的 fallback（task 内 W 的均值），即“步级 credit 但不按 belief 分组”。算力够就 14B 加这一臂，不够就只做你那一臂（已经能回答 Q1）。

多 run

TA 在 W2 里要的 “variance over training seeds” 是训练 seed。按 .note 记录，7B RL ≈40 min/步、14B ≈35–48 min/步（EFA），103 步一趟 ≈3–3.5 天，32B 更久（0902 记录 32B 评 198 题就要 ~7 h）。11/18 前三个尺度各多 seed 做不完，见时间线的三档方案。

要你定

对照臂公式（细节 1 的 a 还是 b）；要不要加细节 2 的臂；32B 跑不跑；seed 走“14B 双臂各 2 seed”还是“缩短 schedule 的配对局”。

W2 / Q2 · TTO 增益贴噪声，缺同触发点的选择对照

④ 新实验① 直接澄清EL

TA 原话

“no control at the same trigger positions, candidate count, and budget using random or simple selection” “with variance over training seeds”

TA 拆的两件事

（i）噪声：TTO 行 28.6/48.6/57.8 没有 ±，+1.4/+2.4/+0.6 里 7B 和 32B 的增量都小于 ANTE 自己的 s.e.（1.7/1.1，p.7）。（ii）归因：增益到底来自“按 belief 排序选候选”，还是来自“多采了 3 个候选、换掉了 greedy 那一步”本身。只有在同一组触发点、同样 K、同样预算下换一个不看 forecast 的选法，才能分开。

你的方案 + 补充

TTO 跑 3 次报 mean 和 error bar、算 CI；K 候选数 1/3/5/7；三个尺度都跑。都对。补三点：
① 触发点：触发规则是 abstain 或预测 FAIL（Algorithm 3），由 greedy 草稿决定，所以各臂在分叉之前触发位置天然相同，这就是 TA 要的 “identical trigger positions”。
② 选法对照（同触发、同 K、同温度）：(a) Eq. 23 按 belief 排序；(b) 在 K+1 个候选里随机挑；(c) 不看 forecast 的简单规则“有跑测试的候选就选它，否则保留草稿”（BwwJ B10 也点名了这条）；(d) 关掉 TTO。(b)(c) 采样数和 (a) 一样，预算天然匹配，TA 要的 “identical budget” 不用另造。
③ K：5 和 7 三个尺度都跑同意；但 32B 一次 500 题评测按 0902 记录要 ~17 h，所以 32B 的 K=5/7 只跑 1 个 seed，7B/14B 跑 3 个。

seed 是什么

我们的“跑三次”= 候选采样的 seed（候选按 temperature 1.0 采，Appendix H），每次换 seed 就换一组候选，这是 TTO 自己的方差。TA 要的 “training seeds” 是不同 RL 训练 run 的 checkpoint，我们没有。回复里要明说 ± 的来源是 evaluation seeds，训练 seed 只在 A1 的 14B 双臂跑出来时才有；不要含糊。另外 Table 1 主表 ± 是 temperature 0 的 3 次评测，它们之间什么在变（环境/vLLM 的不确定性）你还没告诉我，回复里要讲清。

budget 怎么报

照 Figure 4 的口径：B_tok = 每题 prompt+生成 token（含被丢弃的候选）相对关掉 TTO 的倍数，从 call counts 实测；再加每题 LLM 调用数和墙钟。随机选和简单规则两臂实测 B_tok 应与 Eq. 23 臂接近，这就是“budget 匹配”的证据。TTS-4（4.42×）只作参考列。

显著性

每臂对 (d) 做逐题配对：500 题 × 3 seed，bootstrap 出均值差的 95% CI，并报 McNemar（0823 预注册就用它）。CI 不含 0 才写“显著”。

要你知道

0823 那次 7B n=100 预注册 probe（台账 T4）里没有任何 TTO 配置显著优于关掉它（主臂 −2；july 规则 +4 后被 500 题全量反号为 −3.4）。所以 E2 要事先备好“增益不显著”的分支：Table 1 的 TTO 行换成实测均值 ±，p.8–9 的 TTO 句子按实测改；TTO 的定位改成 “belief 触发的重采样，用来诊断不确定步”。另外旧 harness 的打分器多一项 −2·will_break，和 Eq. 23 不一致，发射前要对齐成 Eq. 23；候选采样 seed 当时没有透传（SWE_BV_SEED），要补。

W3 / Q4 · “world model” 说大了；OOD 读得太因果

① 直接澄清⑤ 有限让步④ 新实验WE

TA 原话

“a result predictor, not a state model of the environment” “Would you consider narrowing the ‘world model’ framing to execution-result prediction, and qualifying the cross-domain causal claim accordingly?”

两层意思

第一层（命名）：被执行检验的只有 {PASS, FAIL, NA}×{exit 0/1}，所以 TA 说这是 result predictor。你的理由“belief 里有 repo/env 的 state summary”在输出层面是事实（p.2：summary + forecast + decision；p.2 结论句 “what the agent has established, what the code does now, what the next command will change”），但 summary 在 RL 里没有梯度、也没被任何东西检验（p.4 “the summary tokens receive no gradient, since nothing checks a summary”）。所以写法是：belief 三部分如实写；被验证、被度量的是 forecast；“world model” 定义成“policy 内部对下一条命令执行结果的 belief”，范围说清，词保留。RL 文献里 world model 本来就常指“预测动作后果”（Dreamer 预测 reward 和观测，VAGEN/RWML 预测下一状态/结果），我们预测的是 harness 读出的那一层结果，这是可辩护的；“omniscient over both time and environment state”（p.1 摘要）不可辩护，修订版已删，回复里要说“已删”而不是“没写过”。

cross-domain causal claim 是什么

p.8：The two rows share the backbone, trajectories, and training procedure and differ only in the belief and its signals. The gap therefore isolates what the belief changes in general reasoning. 加上 p.2 ❹ “improves average performance by 6.2 percentage points over a matched RL baseline”。这就是“跨域因果主张”：把 OOD 上 +6.2（MMLU-Pro +10.4）归因为 belief 训练。TA 说 “a single belief intervention” 撑不起这么强的读法，因为：每格只跑了一次、没有误差棒；对照只有 Reactive，而 Reactive 还是唯一没有 format 项和 budget warning 的臂（p.31–32），两边的差不只 belief。“qualify accordingly” = 把 “isolates” 改成 “is consistent with”，补干净对照和区间。

你猜的那个问题

“任务 OOD、和 belief 无关，为什么能提升？” O.5（p.43）给的是机制假设：belief 训练出来的模型在纯 CoT 里会先构造再 trace 函数自己验证，Reactive 模拟一次就停。这当解释可以写，当证据不行；回复里写成 hypothesis，并和新补的 Retrospective-32B OOD 对照一起给。arXiv 版 O.5 有没有更长的版本我没在 .overleaf 里搜到（两个 clone 里都没有这一节的 tex），要用得先确认在哪一版。

要你定

标题 “Learning Internal World Models” 保留（正文精确定义）还是改；补不补 Retrospective-RL-32B 的 11 个 OOD benchmark（只评测不训练，约 1 个 job 1 天）。

W4 / Q3 · 理论里的量没实测，界成立不代表紧

① 直接澄清④ 新实验⑤ 有限让步LEW

TA 原话

“Proposition 2 depends on inter-candidate KL, a mutual-information term, and the point-prediction error ζ; none of these are estimated empirically” “estimate the extra terms … on a small sample”

先纠一处

Prop. 2（p.4，Eq. 2）里只有每个候选自己的 δ_k = KL(p_k‖q_k)：候选 k 的真实执行结果分布 p_k 对它自己写下的 forecast 分布 q_k，不是“候选之间的 KL”。互信息项在 Eq. 17（p.24），连接训练时的 forecast divergence 和候选 divergence；ζ 在 Corollary 3 / Eq. 16（p.23）。“三个量都没估”这一点 TA 说得对，直接认。

理论上“紧不紧”是什么意思

Eq. 16：E[v_k* − v_k̃] ≤ U·E[√(2 max_k δ_k)] + 2ζ，U=4（Eq. 23 的分值范围）。左边是真实选择遗憾（最好候选的真实分值减去我们选中的那个的真实分值），最大才 4。右边只要 max δ_k ≥ 0.5 nats，第一项就 ≥ 4，界就空了（vacuous）。所以在真实 trigger 步上，这个界很可能是“成立但不信息”；它的价值在于指出该降哪个量（forecast divergence 和 point-forecast 的离散），不在于数值紧。回复里要提前把这个说出来，不要等审稿人算。

怎么估（小样本）

取 n=50–100 个 trigger 步（优先 14B 或 32B 的 TTO 日志）。
ζ：只要每个候选 value token 上的概率分布 q_k（无需执行）：ζ̂ = mean_步 max_k |u(ŷ^(k)) − E_{q_k}[u]|。日志没存 logprob 的话，用 policy 对候选文本 teacher-forcing 重算一遍 value token 的 softmax。
δ_k：要执行没被选中的候选。沙箱是确定性的，p_k 是点质量，δ_k = −log q_k(y_k)，即 forecast 给真实结果的 NLL。做法：按轨迹前缀在 R2E-Gym 容器里重放到 trigger 步，再分别执行 K+1 个候选各一次。
MI 项：Eq. 17 是恒等式，右边 I(a;y|c) 用同一批样本的 (候选动作, 结果) 对做插件估计，样本小时只报量级。
界 vs 遗憾：同一批步上算 Eq. 16 右边和真实遗憾 u(y_k*) − u(y_k̃)，报比值。

你要的 synthetic env

一个只有一步的选择问题：K+1 个候选，各自有已知的结果分布 p_k（定义在 Y = {PASS,FAIL,NA}×{0,1} 上，分值 u 用 Eq. 23），forecaster q_k = p_k 加受控扰动使 KL 落在给定 δ；选法用 Cor. 3 的 plug-in（从 q_k 抽一个点预测取 u）或 Prop. 2 的 believed value。Monte Carlo 算精确遗憾，与界并排，扫 δ ∈ {0.01, 0.05, 0.2, 0.5} × K ∈ {1, 3, 7}。它回答的是“界在哪个区间有信息”，不是“我们的模型落在哪”；后者由真实小样本回答。CPU 上几分钟，可以先做。

要你定

TTO 日志里有没有候选 belief 和 value-token logprob；trigger 步的容器能不能按前缀重放（决定 δ_k 做不做得了）。

Experiments and tables

实验规划：设计、读法、预期（表格在下一节 English tables）

这里只放每个实验怎么设计、怎么读、预期什么；表格一律英文，放在下一节，形状就是最后贴进回复的样子。占位表示待实测。E = 只评测，T = 要训练，L = 只分析日志。

E1 · BAPO isolation（T + E）答 A1 · 兼答 BwwJ B3/B4、zcGj C5 · 表 = Table R1

设计：同一 SFT 起点、同 prediction loss、同 decision reward 与 (α, γ, κ)、同数据与 schedule，只换 decision 信号进 advantage 的方式。Control B = 整条轨迹的 decision return W_i,1 在 task 内按 Eq. 5 同样标准化（floor α+γ、ω=½、clip ±2）后广播到每步；Control C（可选）= 保留步级项，只把 V̂/Q̂ 换成 task 内 W 的均值（p.31 的 fallback），即“步级 credit 但不按 belief 分组”。w/o BAPO 行同批补评 3 次拿 ±。

读法：Table R1 里 ANTE − Control B 就是“把 credit 锚在同 forecast 的那一步”单独值多少。若 Δ 接近 0，诚实写法是 “most of the BAPO gain is carried by the decision return itself”，p.7–8 那句 “a large part of the gain comes from BAPO rewarding each step” 相应改。预期不写：memory 里 32B 的 bapo 对只 belief 在 hit rate 上是真分离（0.44→0.63 对冲高回落），但 decision 位上两臂几乎不分离，所以这一臂怎么落我不猜。

E2 · TTO：同候选选法对照、K 扫描、赢家正确率（E + L）答 A2 · 兼答 BwwJ B9/B10 · 表 = Tables R2a, R2b, R2c

设计：触发规则（abstain 或预测 FAIL）由 greedy 草稿决定，所以各臂在分叉前触发位置相同；K=3、温度相同；只换选法：(a) Eq. 23 排序，(b) K+1 里随机，(c) test-first（有跑测试的候选就选，否则留草稿，不看 forecast），(d) 关掉。(b)(c) 采样数与 (a) 相同，预算天然匹配，B_tok 按 Figure 4 口径实测。每臂 3 个候选采样 seed，报 mean ± s.e.、对 (d) 的逐题配对 95% bootstrap CI 和 McNemar。K ∈ {3, 5, 7}，32B 的 5/7 单 seed（一次 500 题 ≈17 h）。

读法：(a) − (b) = ranking 本身；(b) − (d) = 多采样换掉 greedy 那一步；(c) 是 BwwJ 隐含的排序“任何 check ≻ 任何 edit”。K 扫描的预期（你的判断）：不涨分只涨成本，真是这样就写 “returns saturate at K = 3 while B_tok grows roughly linearly in K”。

预期与成因：(b) 持平或略降，因为 3/4 的概率把 greedy 换成 T=1.0 单采样本（0823 probe：bo4 单遍 16/22/21 对 greedy 25）；(c) 很可能 ≈ (a)，因为 Eq. 23 下只要有预测 PASS 的 check 它就赢，两者只在“只有预测 FAIL 的 check”或“没有 check 候选”时不同；(a) − (d) 可能不显著：0823 的 7B n=100 预注册 probe 里没有任何 TTO 配置显著优于关掉（主臂 −2），印值 +1.4 / +0.6 也小于 s.e.。M1 可能不高：0823 校准诊断显示 world model 系统性乐观、少报失败，O.1 step 7 是一例；M1 − M1′ 才是 “ranking 选到了更准的 forecast” 的证据。若 (c) ≈ (a) 且 (a) − (d) 不显著：TTO 改写为 belief-triggered resampling，单独报触发器的价值（触发步里真实失败的富集；0823 离线测量 7B 1.7× / 14B 2.5× / 32B 3.1×，重算后再用）。

E3 · Prop. 2 / Cor. 3 的量：真实小样本 + synthetic（L + 小规模 E + CPU）答 A4 · 兼答 BwwJ B7 · 表 = Tables R3a, R3b

真实样本（Table R3a）：n = 50–100 个 trigger 步（优先 14B / 32B 的 TTO 日志）。ζ 只要候选 value token 的概率分布，不执行；δ_k 要把没选中的候选也执行一次（沙箱确定，p_k 是点质量，δ_k = −log q_k(y_k)），做法是按前缀在容器里重放到 trigger 步；MI 项用同批 (候选动作, 结果) 对做插件估计，只报量级；最后在同批步上算 Eq. 16 右边与真实遗憾的比值。

synthetic（Table R3b）：单步选择问题，K+1 个候选、已知 p_k、Eq. 23 分值、q_k 受控扰动到给定 δ、Cor. 3 的 plug-in 选法，Monte Carlo 算精确遗憾，扫 δ × K。

预期：真实步上的界多半空（U=4，max δ_k ≥ 0.5 nats 第一项就 ≥ 4）；synthetic 会显示只有 δ 很小时界才有信息。回复里提前说出来，理论定位为“指出该降哪个量”，p.24 那句 “formal side of the empirical finding (§3.4)” 删。

E4 · OOD 的干净对照：Retrospective-RL-32B 补 11 个 benchmark（E）答 A3 · 兼答 BwwJ B2 · 表 = Table R4

Table 3 口径（greedy CoT，无 scaffold、无 belief protocol），只补 Retrospective-RL-32B 这一列，逐题 bootstrap 给 ANTE − Retrospective 的区间。AIME’25 只有 30 题、AMC23 40 题，一题就是 3.3 / 2.5 个点，所以 +16.7 / +7.5 必须带区间说；MMLU-Pro 600 题的 +10.4 才是能站住的那个。p.8 的 “isolates” 改 “is consistent with”。

Rebuttal-ready tables

Tables as they will be posted in the reply to Reviewer 3Rua (English)

Final form, one table per request. Plain numbers are from the submitted PDF; dashed orange cells are placeholders to be filled only from the runs. Each table has its Markdown source for OpenReview underneath.

Table R1 · BAPO isolation on SWE-bench Verified (pass@1, %)

All rows start from the same SFT checkpoint and share the prediction loss, the decision reward r_t with coefficients (α, γ, κ), the data, the schedule and the budget of Table 9; they differ only in how the decision signal enters the advantage. ± is the s.e. over 3 evaluation runs of one checkpoint, as in Table 1.

Variant	Decision reward	Credit	Anchor	7B	14B	32B
ANTE w/o BAPO (Table 1; now re-evaluated 3×)	no	episode	none	25.4 ± x.x	43.6 ± x.x	54.2 ± x.x
Control B (new): trajectory-level decision return. W_i,1 = Σ_t r_t, standardized within the task with the same ω and clipping as Eq. 5, broadcast to every step	yes	episode	none	xx.x ± x.x	xx.x ± x.x	xx.x ± x.x †
Control C (optional): step-level credit with the task-mean anchor (the fallback of p.31)	yes	step	task	—	xx.x ± x.x	—
ANTE (BAPO; Table 1)	yes	step	forecast × decision	27.2 ± 1.7	46.2 ± 0.9	57.2 ± 1.1
Δ ANTE − Control B (the part of +1.8 / +2.6 / +3.0 attributable to anchoring by forecast)				+x.x	+x.x	+x.x
Training seeds per row				1 +n	1 +n	1

† reported if the 32B run completes within the discussion period; otherwise the cell says so.

Markdown for OpenReview

**Table R1. BAPO isolation on SWE-bench Verified (pass@1, %)** All rows start from the same SFT checkpoint and share the prediction loss, the decision reward r_t with coefficients (α, γ, κ), the data, the schedule and the budget of Table 9; they differ only in how the decision signal enters the advantage. ± is the s.e. over 3 evaluation runs of one checkpoint, as in Table 1.

| Variant | Decision reward | Credit | Anchor | 7B | 14B | 32B |
|---|---|---|---|---|---|---|
| ANTE w/o BAPO (Table 1; now re-evaluated 3×) | no | episode | none | 25.4 [± x.x] | 43.6 [± x.x] | 54.2 [± x.x] |
| Control B (new): trajectory-level decision return. W_{i,1} = Σ_t r_t, standardized within the task with the same ω and clipping as Eq. 5, broadcast to every step | yes | episode | none | [xx.x ± x.x] | [xx.x ± x.x] | [xx.x ± x.x] † |
| Control C (optional): step-level credit with the task-mean anchor (the fallback of p.31) | yes | step | task | — | [xx.x ± x.x] | — |
| ANTE (BAPO; Table 1) | yes | step | forecast × decision | 27.2 ± 1.7 | 46.2 ± 0.9 | 57.2 ± 1.1 |
| Δ ANTE − Control B (the part of +1.8 / +2.6 / +3.0 attributable to anchoring by forecast) |  |  |  | [+x.x] | [+x.x] | [+x.x] |
| Training seeds per row |  |  |  | 1 [+n] | 1 [+n] | 1 |

† reported if the 32B run completes within the discussion period; otherwise the cell says so.

Table R2a · ANTE-TTO against forecast-free selection rules on identical candidates (SWE-bench Verified pass@1, %)

The trigger rule (abstain or predicted FAIL, Algorithm 3), K = 3 and the sampling temperature are fixed; only the choice among the K+1 candidates differs, so arms (b) and (c) sample exactly as many candidates as (a). Each cell: mean ± s.e. over 3 candidate-sampling seeds (paired Δ against (d) with a 95% bootstrap CI over the 500 tasks; measured token budget B_tok relative to (d), as in Figure 4).

Selection rule at identical triggers	7B	14B	32B
(d) Greedy draft, no resampling (ANTE, Table 1)	27.2 ± 1.7 (reference; 1.00×)	46.2 ± 0.9 (reference; 1.00×)	57.2 ± 1.1 (reference; 1.00×)
(b) Uniform random pick among the K+1 candidates	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)
(c) Test-first: a test-running candidate whenever there is one, otherwise the draft	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)
(a) Belief ranking, Eq. 23 (ANTE-TTO)	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)	xx.x ± x.x (Δ [ , ]; 1.xx×)
Reference: TTS-4, best-of-4 full trajectories (Figure 4)	B_tok 4.42×	—	—

(a) − (b) is the contribution of the ranking itself; (b) − (d) is the contribution of resampling at the trigger steps; (c) is the ordering the reviewer's analysis implies (any check above any edit).

Markdown for OpenReview

**Table R2a. ANTE-TTO against forecast-free selection rules on identical candidates (SWE-bench Verified pass@1, %)** The trigger rule (abstain or predicted FAIL, Algorithm 3), K = 3 and the sampling temperature are fixed; only the choice among the K+1 candidates differs, so arms (b) and (c) sample exactly as many candidates as (a). Each cell: mean ± s.e. over 3 candidate-sampling seeds (paired Δ against (d) with a 95% bootstrap CI over the 500 tasks; measured token budget B_tok relative to (d), as in Figure 4).

| Selection rule at identical triggers | 7B | 14B | 32B |
|---|---|---|---|
| (d) Greedy draft, no resampling (ANTE, Table 1) | 27.2 ± 1.7 (reference; 1.00×) | 46.2 ± 0.9 (reference; 1.00×) | 57.2 ± 1.1 (reference; 1.00×) |
| (b) Uniform random pick among the K+1 candidates | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) |
| (c) Test-first: a test-running candidate whenever there is one, otherwise the draft | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) |
| (a) Belief ranking, Eq. 23 (ANTE-TTO) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) | [xx.x ± x.x] ([Δ [ , ]]; [1.xx×]) |
| Reference: TTS-4, best-of-4 full trajectories (Figure 4) | B_tok 4.42× | — | — |

(a) − (b) is the contribution of the ranking itself; (b) − (d) is the contribution of resampling at the trigger steps; (c) is the ordering the reviewer's analysis implies (any check above any edit).

Table R2b · Candidates per trigger, K (SWE-bench Verified pass@1, %)

Same trigger rule and temperature as Table R2a; K = 1 is the greedy draft with no resampling. Each cell: mean ± s.e. over 3 candidate-sampling seeds, and the measured B_tok.

K	7B	14B	32B
1 (off)	27.2 ± 1.7 · 1.00×	46.2 ± 0.9 · 1.00×	57.2 ± 1.1 · 1.00×
3	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×
5	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x · 1.xx× ‡
7	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x · 1.xx× ‡

‡ single candidate-sampling seed at 32B for K = 5 and 7 (one 500-task evaluation at 32B takes about 17 hours).

Markdown for OpenReview

**Table R2b. Candidates per trigger, K (SWE-bench Verified pass@1, %)** Same trigger rule and temperature as Table R2a; K = 1 is the greedy draft with no resampling. Each cell: mean ± s.e. over 3 candidate-sampling seeds, and the measured B_tok.

| K | 7B | 14B | 32B |
|---|---|---|---|
| 1 (off) | 27.2 ± 1.7 · 1.00× | 46.2 ± 0.9 · 1.00× | 57.2 ± 1.1 · 1.00× |
| 3 | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] |
| 5 | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x] · [1.xx×] ‡ |
| 7 | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x] · [1.xx×] ‡ |

‡ single candidate-sampling seed at 32B for K = 5 and 7 (one 500-task evaluation at 32B takes about 17 hours).

Table R2c · What the selected candidates forecast and how they turned out (K = 3, Eq. 23 arm)

M1 and M1′ are counted from the evaluation logs on conclusive steps (realized PASS or FAIL). M2 executes all K+1 candidates from a replay of the trajectory prefix on a subsample of trigger steps; the best set contains the candidates with the highest realized Eq. 23 score, and chance is the expected hit rate of a uniform pick on the same candidate sets. 95% CIs by cluster bootstrap over tasks.

Quantity	7B	14B	32B
Trigger steps per trajectory (mean) / share of triggers at which the draft was replaced	x.x / xx%	x.x / xx%	x.x / xx%
M1: forecast of the executed winner matched the realized outcome	xx% [ , ]	xx% [ , ]	xx% [ , ]
M1′: the same share for the greedy drafts that were replaced	xx% [ , ]	xx% [ , ]	xx% [ , ]
M2: selected candidate in the best set, replayed subsample (n = 100 trigger steps; all-tie steps excluded and reported separately)	xx% vs chance xx%; ties xx%	xx% vs chance xx%; ties xx%	xx% vs chance xx%; ties xx%

Markdown for OpenReview

**Table R2c. What the selected candidates forecast and how they turned out (K = 3, Eq. 23 arm)** M1 and M1′ are counted from the evaluation logs on conclusive steps (realized PASS or FAIL). M2 executes all K+1 candidates from a replay of the trajectory prefix on a subsample of trigger steps; the best set contains the candidates with the highest realized Eq. 23 score, and chance is the expected hit rate of a uniform pick on the same candidate sets. 95% CIs by cluster bootstrap over tasks.

| Quantity | 7B | 14B | 32B |
|---|---|---|---|
| Trigger steps per trajectory (mean) / share of triggers at which the draft was replaced | [x.x / xx%] | [x.x / xx%] | [x.x / xx%] |
| M1: forecast of the executed winner matched the realized outcome | [xx% [ , ]] | [xx% [ , ]] | [xx% [ , ]] |
| M1′: the same share for the greedy drafts that were replaced | [xx% [ , ]] | [xx% [ , ]] | [xx% [ , ]] |
| M2: selected candidate in the best set, replayed subsample (n = [100] trigger steps; all-tie steps excluded and reported separately) | [xx%] vs chance [xx%]; ties [xx%] | [xx%] vs chance [xx%]; ties [xx%] | [xx%] vs chance [xx%]; ties [xx%] |

Table R3a · The terms of Proposition 2 and Corollary 3 on real trigger steps (K = 3, U = 4)

δ_k = KL(p_k ‖ q_k) is the divergence of candidate k's executed outcome law from its own forecast; in a deterministic sandbox it reduces to −log q_k(y_k) and needs one execution of each candidate from a replay of the trajectory prefix. ζ is the point-forecast spread of Corollary 3 and needs only the value-token probabilities. The bound is the right-hand side of Eq. 16; the regret is the realized u(y_{k*}) − u(y_{k̃}) on the same steps.

Quantity	14B	32B	How obtained
Trigger steps, n	n	n	subsample of the Table R2a logs
mean over steps of mean_k δ_k (nats)	x.xx	x.xx	execute each candidate once
mean over steps of max_k δ_k (nats)	x.xx	x.xx	same
ζ̂ = mean over steps of max_k \|u(ŷ^(k)) − E_{q_k} u\|	x.xx	x.xx	value-token probabilities, no execution
Î(a; y \| c), the information term of Eq. 17 (plug-in estimate, order of magnitude)	x.xx	x.xx	the same (candidate, outcome) pairs
Realized regret E[u(y_{k*}) − u(y_{k̃})]	x.xx	x.xx	executed outcomes and Eq. 23 scores
Bound, Eq. 16: 4 · E√(2 max_k δ_k) + 2 ζ̂	x.xx	x.xx	the two rows above
Bound / regret (a value above 4 means the bound is vacuous on these steps)	x.x	x.x

Markdown for OpenReview

**Table R3a. The terms of Proposition 2 and Corollary 3 on real trigger steps (K = 3, U = 4)** δ_k = KL(p_k ‖ q_k) is the divergence of candidate k's executed outcome law from its own forecast; in a deterministic sandbox it reduces to −log q_k(y_k) and needs one execution of each candidate from a replay of the trajectory prefix. ζ is the point-forecast spread of Corollary 3 and needs only the value-token probabilities. The bound is the right-hand side of Eq. 16; the regret is the realized u(y_{k*}) − u(y_{k̃}) on the same steps.

| Quantity | 14B | 32B | How obtained |
|---|---|---|---|
| Trigger steps, n | [n] | [n] | subsample of the Table R2a logs |
| mean over steps of mean_k δ_k (nats) | [x.xx] | [x.xx] | execute each candidate once |
| mean over steps of max_k δ_k (nats) | [x.xx] | [x.xx] | same |
| ζ̂ = mean over steps of max_k \|u(ŷ^(k)) − E_{q_k} u\| | [x.xx] | [x.xx] | value-token probabilities, no execution |
| Î(a; y \| c), the information term of Eq. 17 (plug-in estimate, order of magnitude) | [x.xx] | [x.xx] | the same (candidate, outcome) pairs |
| Realized regret E[u(y_{k*}) − u(y_{k̃})] | [x.xx] | [x.xx] | executed outcomes and Eq. 23 scores |
| Bound, Eq. 16: 4 · E√(2 max_k δ_k) + 2 ζ̂ | [x.xx] | [x.xx] | the two rows above |
| Bound / regret (a value above 4 means the bound is vacuous on these steps) | [x.x] | [x.x] |  |

Table R3b · Synthetic selection problem: where the bounds are informative

K+1 candidates with known outcome laws p_k on Y = {PASS, FAIL, NA} × {0, 1} and the score u of Eq. 23 (U = 4); each forecast q_k is a controlled perturbation of p_k with KL(p_k ‖ q_k) = δ; the selector uses the plug-in point forecast of Corollary 3. Regret is computed exactly by Monte Carlo. Each cell: regret / Proposition 2 bound / Corollary 3 bound.

Forecast divergence δ (nats)	K = 1	K = 3	K = 7
0.01	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx
0.05	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx
0.2	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx
0.5	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx	x.xx / x.xx / x.xx

This table answers where the bound is informative, not where the trained model lies; Table R3a answers the latter.

Markdown for OpenReview

**Table R3b. Synthetic selection problem: where the bounds are informative** K+1 candidates with known outcome laws p_k on Y = {PASS, FAIL, NA} × {0, 1} and the score u of Eq. 23 (U = 4); each forecast q_k is a controlled perturbation of p_k with KL(p_k ‖ q_k) = δ; the selector uses the plug-in point forecast of Corollary 3. Regret is computed exactly by Monte Carlo. Each cell: regret / Proposition 2 bound / Corollary 3 bound.

| Forecast divergence δ (nats) | K = 1 | K = 3 | K = 7 |
|---|---|---|---|
| 0.01 | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] |
| 0.05 | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] |
| 0.2 | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] |
| 0.5 | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] | [x.xx / x.xx / x.xx] |

This table answers where the bound is informative, not where the trained model lies; Table R3a answers the latter.

Table R4 · Out-of-distribution results at 32B with the matched Retrospective baseline (Table 3 protocol)

Greedy chain-of-thought with no agent scaffold and no belief protocol, as in Table 3. Retrospective-RL-32B is the baseline that carries the format term and the budget warnings (Appendix G); the Reactive-RL and ANTE columns are the submitted Table 3. The last column is a paired bootstrap interval over items.

Benchmark	Reactive-RL (Table 3)	Retrospective-RL (new)	ANTE (Table 3)	ANTE − Retrospective [95% CI]
CruxEval-O	83.6	xx.x	85.4	+x.x [ , ]
CruxEval-I	72.2	xx.x	78.8	+x.x [ , ]
LCB-exec	91.4	xx.x	94.2	+x.x [ , ]
BBH-hard	87.2	xx.x	90.9	+x.x [ , ]
BBH-state	88.6	xx.x	91.7	+x.x [ , ]
MATH500	94.8	xx.x	97.2	+x.x [ , ]
AIME'25 (30 items)	63.3	xx.x	80.0	+x.x [ , ]
AMC23 (40 items)	90.0	xx.x	97.5	+x.x [ , ]
Minerva	38.6	xx.x	46.0	+x.x [ , ]
MMLU-Pro (physics, chemistry, biology; 600 items)	72.8	xx.x	83.2	+x.x [ , ]
SciBench	66.0	xx.x	71.4	+x.x [ , ]
Average	77.1	xx.x	83.3	+x.x [ , ]

Averages of the Table 3 columns: 77.1 and 83.3 (+6.2). With 30 and 40 items, one item moves AIME'25 by 3.3 points and AMC23 by 2.5.

Markdown for OpenReview

**Table R4. Out-of-distribution results at 32B with the matched Retrospective baseline (Table 3 protocol)** Greedy chain-of-thought with no agent scaffold and no belief protocol, as in Table 3. Retrospective-RL-32B is the baseline that carries the format term and the budget warnings (Appendix G); the Reactive-RL and ANTE columns are the submitted Table 3. The last column is a paired bootstrap interval over items.

| Benchmark | Reactive-RL (Table 3) | Retrospective-RL (new) | ANTE (Table 3) | ANTE − Retrospective [95% CI] |
|---|---|---|---|---|
| CruxEval-O | 83.6 | [xx.x] | 85.4 | [+x.x [ , ]] |
| CruxEval-I | 72.2 | [xx.x] | 78.8 | [+x.x [ , ]] |
| LCB-exec | 91.4 | [xx.x] | 94.2 | [+x.x [ , ]] |
| BBH-hard | 87.2 | [xx.x] | 90.9 | [+x.x [ , ]] |
| BBH-state | 88.6 | [xx.x] | 91.7 | [+x.x [ , ]] |
| MATH500 | 94.8 | [xx.x] | 97.2 | [+x.x [ , ]] |
| AIME'25 (30 items) | 63.3 | [xx.x] | 80.0 | [+x.x [ , ]] |
| AMC23 (40 items) | 90.0 | [xx.x] | 97.5 | [+x.x [ , ]] |
| Minerva | 38.6 | [xx.x] | 46.0 | [+x.x [ , ]] |
| MMLU-Pro (physics, chemistry, biology; 600 items) | 72.8 | [xx.x] | 83.2 | [+x.x [ , ]] |
| SciBench | 66.0 | [xx.x] | 71.4 | [+x.x [ , ]] |
| Average | 77.1 | [xx.x] | 83.3 | [+x.x [ , ]] |

Averages of the Table 3 columns: 77.1 and 83.3 (+6.2). With 30 and 40 items, one item moves AIME'25 by 3.3 points and AMC23 by 2.5.

Tables R2a–R2c are shared with the reply to Reviewer BwwJ (there as F2, F2b, F2c).

Wording

写作修改清单（修订版 PDF 要动的句子）

Location (submitted PDF)	Current wording	Change
p.1, last sentence of the abstract	“… world model that is omniscient over both time and environment state.”	Delete (already removed in the working revision); the reply says “removed”, not “never written”.
p.1 abstract / p.2 contribution ❶	“its world model’s accuracy climbs …” / “learning an internal world model within the policy”	Define the scope: “an internal world model over the execution outcome of the agent’s next action”; “world model accuracy” → “forecast hit rate on conclusive steps”.
p.2 §1, belief definition	“a brief state summary, an outcome prediction, and a trust or abstain decision”	Keep; add that execution verifies the prediction and that the summary is trained in Stage I only (consistent with p.4).
p.7–8 §3.2	“so a large part of the gain comes from BAPO rewarding each step instead of giving every step the same final reward”	Rewrite from the Δ of Table R1: credit only what the isolation supports.
p.8 §3.3	“The gap therefore isolates what the belief changes in general reasoning.”	“The gap is consistent with the belief training changing general reasoning; Table 3 reports one greedy run per model, and Reactive also lacks the format term (Appendix G). Table R4 adds Retrospective.”
p.8, end of §3	“consistent ANTE-TTO gains show that learned beliefs are both predictive and actionable (Proposition 2; Figure 5; Table 1)”	Rewrite from Table R2a; drop “actionable” if the gain is not significant.
p.9 §3.5	“Together with the ANTE-TTO gains of Table 1, this is the empirical side of Proposition 2 … raises success at every scale.”	Rewrite from Tables R2a and R3a; if the bound is vacuous, Proposition 2 “identifies the quantity to reduce”.
p.24 Appendix D.5	“this is the formal side of the empirical finding that ANTE-TTO helps as the world model becomes accurate (§3.4)”	Delete: §3.4 contains no such analysis, and the cross-scale hit rates 58 / 63 / 69 do not move with the TTO gains.
Table 1	TTO row without ±; no isolation row	Add the Control B row; replace the TTO row by the measured mean ± s.e. over 3 seeds and state K and what the seeds are.
Title	“Learning Internal World Models …”	Your call; if kept, the definition in §1 must match it.

Draft response

回复稿（英文，按 ICLR 每条评论 ≤ 5,000 字符拆成三条）

结构照 rebuttal skill：每条先一句直接回答，再 2–4 句带出处的证据，最后一句说明影响。方括号 [INSERT …] 是等实验的占位，不编数。引用审稿人原话只引最短的那句。没有道歉句，没有预设对方反驳。

Comment 1 / 3 · W1–W2, Q1–Q2

Thank you for the careful reading. The four weaknesses ask one fair question, which part of each gain belongs to which component. We answer each with a controlled measurement where one is needed, and with a wording change where the text claims more than the design supports.

W1 / Q1, BAPO isolation. The +1.8 / +2.6 / +3.0 points (Table 1, ANTE minus ANTE w/o BAPO at 7B / 14B / 32B) measure the decision reward and the belief-anchored advantage together, because the w/o BAPO variant removes both (§3.1, p.7). We are running the control you describe: the same SFT checkpoint, prediction loss, decision reward r_t and coefficients (α, γ, κ), data, schedule and budget (Table 9); the only change is where the decision signal lands. In the control, the decision return of the whole rollout, W_{i,1} = Σ_t r_{i,t}, is standardized within the task with the same ω and clipping as Eq. 5 and broadcast to every step, instead of being compared within the forecast groups G(ỹ) and G(ỹ, d). The difference between this control and ANTE is therefore attributable to anchoring credit by the predicted outcome. [INSERT: Table R1, 7B and 14B, pass@1 mean ± s.e. over 3 evaluation runs; Δ = ANTE − control.] [INSERT: 32B row, or the statement that the 32B run did not finish within the discussion period.] [INSERT if run: a second control that keeps the step-level term but replaces the group baselines by the task-level mean of W (the fallback already defined on p.31), which removes the belief key while keeping per-step credit.] We will add the control row to Table 1 and rewrite the attribution on p.7–8 to credit only what the isolation supports.

W2 / Q2, ANTE-TTO. We agree that the TTO row needs its own error bars and a selection control, and we are adding both. The trigger rule is fixed (abstain or predicted FAIL, Algorithm 3) and is decided by the greedy draft, so every arm resamples at the same positions until the arms diverge; K and the sampling temperature are also fixed. Only the choice among the K+1 candidates changes: (a) the belief ranking of Eq. 23; (b) a uniform random pick; (c) a forecast-free rule, take a test-running candidate if there is one, otherwise keep the draft; (d) the draft itself, no resampling. Arms (b) and (c) sample exactly as many candidates as (a), so they are budget-matched by construction; we report the measured token budget B_tok of every arm as in Figure 4. Each arm is run with 3 candidate-sampling seeds, and we report mean ± s.e. together with a paired 95% bootstrap interval on the per-task difference against (d) over the 500 tasks. [INSERT: Table R2a at 7B / 14B / 32B.] We also vary K ∈ {3, 5, 7} [INSERT: Table R2b]. On seeds: the ± in Table 1 is over evaluation runs of one checkpoint, as its caption states, and the new ± is over candidate-sampling seeds; training-seed variance is [INSERT: available at 14B from the isolation runs above / not available within the discussion period], and the revision will say which. Whatever these runs show replaces the TTO row of Table 1 and the sentences on p.8–9 that describe its gains.

Comment 2 / 3 · W4, Q3

W4 / Q3, the quantities in Proposition 2. One clarification on what the bound contains: its only data-dependent quantities are the per-candidate divergences δ_k = KL(p_k ‖ q_k) between the executed outcome law of candidate k and the forecast written for that same candidate (Eq. 2, p.4); the mutual-information term is in Eq. 17 (p.24), which relates the training-time divergence to the candidate divergence; ζ is the point-forecast spread of Corollary 3 (Eq. 16, p.23). You are right that none of the three is estimated in the paper. We are estimating them on a sample of trigger steps: ζ from the forecast distribution q_k read off the value-token probabilities of each candidate, which needs no execution; δ_k by executing each unselected candidate from a replay of the trajectory prefix in the sandbox, which for a deterministic environment reduces to −log q_k(y_k); and the right-hand side of Eq. 16 against the realized regret u(y_{k*}) − u(y_{k̃}) on the same steps, with U = 4. [INSERT: Table R3a, n trigger steps, mean and max δ_k, ζ, MI term, bound, realized regret.] Because the score range is 4, the first term alone exceeds the largest possible regret once max_k δ_k ≥ 0.5 nats, so the bound is informative only for well-calibrated forecasts; to show where that regime lies we add a synthetic selection problem with known outcome laws, sweeping the forecast divergence and K [INSERT: Table R3b]. We will report both results as they come out. If the bound is loose on real steps, the revision presents Proposition 2 and Corollary 3 as identifying which quantity the training reduces, not as a tight guarantee, and removes the sentence on p.24 that calls Corollary 3 the formal side of a finding in §3.4, since §3.4 contains no such analysis.

Comment 3 / 3 · W3, Q4

W3 / Q4, the term "world model" and the cross-domain claim. The belief state has three parts (p.2, §2.1): a summary of what the agent has established and what the code does now, a forecast of the outcome of the next command (test PASS / FAIL / NA and exit status), and a trust / abstain decision. You are right that only the forecast is verified by execution and that every accuracy number in the paper measures the forecast; the summary is trained in Stage I and receives no RL gradient (p.4). The revision states this in the abstract and §1, defines the model as an internal world model over the execution outcome of the agent's next action, removes "omniscient over both time and environment state" (already removed in our working revision), and writes "forecast hit rate" where the submission says "world model accuracy". We keep "world model" in the sense used by VAGEN and RWML, a predictor of the agent-visible consequence of an action, with that scope stated.

On the cross-domain claim: Table 3 compares one ANTE-32B run with one Reactive-RL run, and p.8 says the gap "isolates what the belief changes". That is stronger than the design supports, because Reactive is also the baseline trained without the format term and budget warnings (Appendix G, p.31–32) and each number is a single greedy run. We are adding Retrospective-RL-32B, the matched baseline that has those components, on the same eleven benchmarks, with paired bootstrap intervals over items, and we rewrite the sentence as "is consistent with the belief training changing general reasoning". [INSERT: Table R4.] The traces in Appendix O.5 stay as an illustration of the mechanism we hypothesize, a predict-then-verify habit carried into plain chain-of-thought, not as evidence for the size of the effect.

[INSERT after launch: which of the runs above are in progress and when each is expected; nothing is promised that is not running.]

Needs your decision

要你拍板

Control B 的公式：(a) 各步 r_t 平均后原样加进 R（你的原话）；(b) W_i,1 在 task 内标准化、同 ω 和 clip，再广播（我的建议，尺度和 BAPO 对齐）。要不要在 14B 加 Control C。
尺度与 seed：7B、14B 全 schedule 各 1 个 8 节点 job，≈3–3.5 天；32B 要 16 节点 ≈5–7 天，11/18 前能不能完是赌。训练 seed 两条路：14B 双臂各再跑 1 个 seed（再加 2 个 job，3.5 天）；或 14B 双臂各 2 seed 的 50 步配对局（4 个 job，1.5 天，schedule 短于论文但两臂等预算，回复里如实标）。
TTO 矩阵的大小：7B/14B 全矩阵（4 臂 × 3 seed + K 5/7 × 3 seed ≈ 18 次评测/尺度）；32B 按 ~17 h/次只能做 (a)(b)(d) × 3 seed + K=5/7 单 seed ≈ 8 次 ≈ 6 天单 job，要不要砍到 2 seed。
发射前 harness 要改的三处：打分器对齐 Eq. 23（旧版多 −2·will_break）；加 random / prefer-test 两种选法；候选采样 seed 透传（0823 登记 SWE_BV_SEED 没透传）。改完先 20 题冒烟。
TTO 的两个分支事先说好：实测值不论方向都替换 Table 1 的 TTO 行和 p.8–9 的句子；增益不显著时 TTO 改写为 belief 触发的重采样诊断。0823 的 7B probe 是这个方向的先验。
OOD：补不补 Retrospective-RL-32B 的 11 个 benchmark（1 job 约 1 天）；标题改不改。
Table 1 主表 ± 的来源：temperature 0 的 3 次评测之间什么在变，回复里要用一句话讲清。
日志：TTO 日志有没有存候选 belief / 分数 / logprob；trigger 步容器能不能按前缀重放。决定 E3a 做多少。
发帖顺序：skill 的建议是先回打低分的（BwwJ、zcGj 都是 2），3Rua 的回复等 E1/E2 有数再发；还是先发写作层的回答、数据后补（ICLR 允许多条评论）。

Compute and timeline

算力与时间线（11/08 → 11/18）

池子现状（19:40Z）

7 个 8 节点 job 全有主：agm 6 台（T16/T17 在跑 + 3 台给 beacon 今晚的 T18–T20 预热），Yireeb 1 台（cd18615a，每天最多 1 个 job）。全部 ≈11/09 14:30 ET 到租。rebuttal 的实验要起新 job；show-capacity 19:36Z 还有 33/98 节点空。

步时依据（.note 0814–0818）

7B RL ≈40 min/步（18–46）；14B ≈35–48 min/步（EFA 生效时；TCP 87–101）；103 步 ⇒ 7B ≈3 天，14B ≈3–3.5 天，含每 24 h 跳箱 ~40–65 min。32B：0902 记录 198 题评测 ~7 h；训练步时按 16 节点未记录，按 p5 ~100 min/步外推 ≈5–7 天。

需要的新 job（8 节点）

7B ctrl ×1、14B ctrl ×1、（14B seed 或配对局 +2）、32B ctrl ×2（16 节点）、TTO 评测 7B/14B ×1、TTO 32B ×1、OOD Retro-32B ×1。同时在飞最多 ≈6–8 个 job，每 24 h 续一次。

Date (PT)	Work	Depends on
Sun 11/08	Decide the Control B formula, the size of the TTO matrix and the 32B go / no-go; three harness fixes plus a 20-task smoke test	Items 1–5 above
Mon 11/09	Launch RL: 7B control, 14B control (and the 32B control); launch the 7B TTO matrix; run the synthetic Table R3b	New jobs available
Tue 11/10	14B TTO matrix; OOD for Retrospective-32B; small-sample estimates for Table R3a (container replay)	Log availability (item 8)
11/11–12	RL running (hop boxes every 24 h); 32B TTO; count the Table R2c log statistics
Fri 11/13	7B / 14B controls finish; evaluate each 3×; fill Table R1	RL launched on 11/09
Sat 11/14	Final replies to all three reviewers; revised PDF (wording list plus the new tables)	Tables R1–R4 filled
Sun 11/15	Post the replies and upload the revision (three days of discussion left)
11/16–18	Follow-ups; post the 32B control if it finishes

依据说明：本页没有读代码和实验日志；RL 步时与 TTO 基建状态引自 .note 与 memory 的 0814–0923 记录（beliefvalidator v4 da711902、rolling_shard_bv4 4b12e98c、SWE_BV_TRIGGER/K/TEMP 参数），发射前要重新核一遍文件还在不在。

ICLR 2027 · Submission35229 · Rebuttal 2 · Reviewer BwwJ

BwwJ：逐句拆解、他怎么想、我们把他带进哪个框架

规划稿 v1（2026-11-08）。他的每一句都对照投稿版 PDF 核过（页码 p.N 指它）；实验一个都没开跑，表里的数全是占位。参考了 Kangrui 的回复草稿、Jifan 的 hit-rate 拆分方案（Downloads 里那页）、rebuttal skill 的回法与语气规则，以及 11/06 读过的 10 个 OpenReview rebuttal 线程。

Rating 2 · Weak reject · 信心 2给低分的主因写在 Strengths 栏（预算、对照、单次训练）Weaknesses + Q1 + Q2 = B5–B10实验未开跑 · 值为占位

Reviewer model

他是谁、他怎么想

他的心智模型

“world model = 在动作之前预见动作的后果。” 他拿这个定义去对照我们的 schema：forecast 只说“本步要跑的 check 结果 + exit”，edit 步恒 NA，所以“改之前预见改动效果”这件事方法做不到；TTO 的分值等价于“偏好预测 PASS 的 check”；hit rate 上升可能来自送分 check；理论是理想化的标准界；标题/摘要/Figure 1 都说过头。每一步推演都对得上论文原文。

他的证据风格（信心 2）

不靠领域直觉，靠论文定义逐条推演，算术全对（Eq. 23 的 4/2/1、O.1 step 7 的反事实）。这类审稿人被什么说服：可核对的事实、一个操作性定义、他点名的对照实验如实跑出来。不被什么说服：形容词、“我们相信”、把他的问题说成误解。

要把他带进的框架（六句）

① anticipation 的单位是 edit → check，不是 edit 本身。② forecast 预测的是一个还不存在的结果：check 还没跑，写下预测，执行再裁决。③ “能不能预见自己改动的效果”= post-edit check（Type A）的 hit rate，这是 headline 指标。④ terminal 的 schema 每一步（含 edit）预测整局结果，TTO 在那里确实按预测给候选 fix 排序（Figure 14 右）。⑤ TTO = belief 触发的重采样；ranking 的贡献用同候选对照量出来。⑥ world model 定义收窄、omniscient 删、理论定位为 motivation 并列出实现差异。

Framing, and its limits

“春秋笔法”的度：能做什么，不能做什么

可以做

先定义再回答：把他的问题改写成我们能精确回答的版本（他问“能不能预见 edit 的效果”，我们答“edit 之后、看到结果之前，对 check 结果的预测准不准”）。
认事实、不认结论：他列的每条 schema 事实都认，然后说“我们对后果的读法不同，原因是…”。
顺序和强调：先讲 edit→check 的原理和 Type A 指标，再讲 Figure 1 重画；TTO 先讲触发再讲 ranking。
用他自己的话：他说 “Only the first type shows whether the agent can foresee the effect of its own edits”，我们接着说“同意，所以 Type A 是修订版的 headline”。

不能做（会反噬）

争辩 B/C 类 check “并不简单”（他的点是 edit foresight，B/C 本来就证明不了）。
说 Figure 18(a) 已排除 mix shift（总占比稳定与 mix 变化相容，而且 7B 那条其实在降）。
直接说他 misunderstood（Dyna-Think 的教训；用 “we see why Figure 1 invites this reading”）。
对照跑出来接近还硬说方法必要（Dyna-Think 被 AC 拿那张表否了）。
承诺修订版 PDF 里没有的东西、把“会做”当证据（Long-context SWE RL 被拒的原因）。
编数、暗示不存在的证据、拿 Table 12 当 RL 过程证据（它只是一个 checkpoint 的轨迹级判分）。

他给 2 分的主因不在这页拆的文字里

他把主批评写在 Strengths 栏：评测预算只写了 ANTE（B1）、headline 对的是无 format 项的 Reactive（B2）、小尺度上 forecast ≈ memory（B3）、BAPO 增益来自单次训练（B4）。B2/B3 由 Retrospective 并列 + E4 OOD 补评回答，B4 由 E1 isolation + seed 回答；B1 取决于各行实际评测预算，你定路径后我再写那一段，本页只留占位。

Sentence by sentence

逐句拆解（左：他的原话；右：对不对、他要什么、我们怎么动）

✓ 成立 · ◐ 部分成立 · ✗ 不成立。回法标签同 rebuttal1。

Critical Weaknesses

The claims go well beyond what the method actually does.

◐ 对标题/摘要/Figure 1 成立，对方法本身不成立。他要的是把 claim 收回到方法能做的事。动作：认具体句子并给出替换措辞（⑤），同时用下面的 edit→check 框架守住“anticipation”这个词（①）。

The title, the abstract and Figure 1 all present the agent as foreseeing an edit's effect before it makes the edit.

✓ Figure 1 题注原话 “only ANTE forecasts the edit's effect and lets execution settle it before committing”（p.2）；p.2 正文 “anticipates that the proposed edit would break other tests, chooses an alternative fix”。动作：Figure 1 用真实轨迹重画（F3），摘要/p.2 改成 “forecasts the outcome of the check that follows each edit”。

In Figure 1, the belief written before the edit compares two possible fixes. It predicts that changing the function's output would pass the target test but break other code that calls the function, while adding a default value keeps all tests passing, so the agent picks the second fix.

✓ 他读图读得准（Forecast A / Forecast B 面板）。这是示意图，不是轨迹，但 schema 产不出这种 forecast，所以不辩。动作：认，“a two-fix comparison the repository schema does not produce”（Kangrui 原句）。

The method as defined cannot produce such forecasts.

◐ repository harness 上成立（dev 步 target_test 恒 NA，p.47）；terminal harness 上不成立：check field 预测 hidden verifier 的整局结果，每步都写，edit 也写（Appendix J），Figure 14 右就是候选 fix 按预测排序。动作：认 repo，指出 terminal（① 结构区分），不夸大：Table 2 没有 TTO 行，terminal 的 TTO 数要新跑（F5）。

The forecast that execution checks has only two discrete fields: the result of the check run in the current step, and whether the command exits cleanly.

✓ p.2–3 的定义。动作：不争。把“离散、可机检、每步结算”当精确性来讲，而不是当狭窄来辩。

An edit step is always forecast as "no verdict" plus the exit code, and the summary gets no training signal in RL.

✓ p.47；p.4 “the summary tokens receive no gradient, since nothing checks a summary”。动作：认；顺手把 summary 的角色说清：Stage I 训、RL 不训、它是 forecast 的写作上下文 c_i,t（Eq. 1）。

So the forecast written before an edit never says whether the edit will make the tests pass or break other tests.

✓ 字面成立，这是他的核心推论。动作（引导点一）：不否认字面，换单位。edit 之后、看到任何后果之前写下的那个 check forecast，才是“对 edit 效果的预测”；它预测的结果还不存在，然后被执行裁决（Eq. 3、Eq. 18）。Type A 的 hit rate 就是这件事的度量（F1）。

The agent only forecasts a test result later, when it runs the tests, and by then the code has already been changed.

✓ 事实对，结论不同。沙箱里改动可逆（O.2 step 25 就用 git stash 撤回再重跑），不可逆的是 submit，而协议规则 5 要求 finish 前必须重跑验证（App. P）。“已改”≠“已承诺”。动作：一句话说清（①）。

For the same reason, ANTE-TTO cannot rank candidate edits by their effect on the tests.

◐ repo 上成立（两个 edit 只差 exit 预测，B9）；terminal 上能（Figure 14 右）。动作：认 repo；TTO 的定位改成“belief 触发的重采样 + 在 check 候选间排序”，ranking 的贡献用同候选对照量出来（F2）。

The rising hit rate may also partly come from easy checks on scripts the agent writes itself.

◐ 机制上可能：BAPO 的步级 credit 只落在 conclusive check 步（Eq. 5），组按 forecast 和 decision 分、不按 check 类型（Eq. 4），trusted 命中一律 +α。是否发生没测。动作：跑 F1 拆分，加 base-rate 控制（④）。

The theory applies standard bounds under idealized assumptions that do not cover the algorithm as implemented.

◐ 论文自己写了理想化（p.19）和实现差异三处（p.21）。“标准”成立，“不覆盖”成立。动作：引用自己的话、定位为 motivation、主文明说（②），配 3Rua A4 的小样本估计。

Calling this an internal world model, and the abstract's stated aim of one that is "omniscient over both time and environment state," goes far beyond this evidence.

✓ omniscient 删（Overleaf 已删，回复说“已删”）。world model 给操作性定义：policy 内部对下一动作执行结果的 belief，被执行验证（WMA 先例：定义 + 与最近工作的区别，不辩名字）。动作：⑤ + ①。

Question 1 · hit rate 的来源

The paper's key evidence that the world model gets better is the rising forecast hit rate on conclusive steps (Figure 5(a)). At 32B it goes from 44% to 69% during RL.

✓ 准确（摘要、§3.5）。

These checks are tests that the agent chooses to run, including reproduction scripts it writes itself.

✓ App. P 的 CHECKING 步含 python reproduce_xxx.py。

The protocol also asks the agent to make each script print its own PASS or FAIL verdict (Appendix P).

✓ rule 4 原文 “MUST decide the outcome INSIDE the script and print a single final line VERDICT: PASS or VERDICT: FAIL”。

Some of these forecasts are easy. One example is predicting that a bug-reproduction script fails before any fix. Another is predicting that a script re-run with no code change gives the same result as last time.

◐ 不用 code 知识就能猜的两类（B 修前复现、C 原样重跑）。SFT 模型在 O.3 里 B 类连错两次，所以“easy”不绝对，但不要拿这点争（他的点是 edit foresight）。动作：在数据里报每类的 best-constant 和 rule baseline，让数字说话。

The paper's own example of the belief sharpening during RL (Appendix O.3) is of the first kind: the improved forecast is that the reproduction script fails on the still-unfixed code.

✓ 比他说的更糟：RL 前后 forecast 完全相同（FAIL, exit_ok false, abstain），变的是复现脚本；两步都 abstain。动作：认，O.3 改标为 B 类，补上 step 25 修后重跑的 forecast，再加一个 Type A 例子（O.4 step 21 现成：32B RL rollout 对自己刚打的 fix 预测 FAIL 并 trust，执行应验）。

BAPO also gives credit to the action at a step whose trusted forecast comes true, so the policy may even learn to run more such easy checks.

◐ 机制在（Eq. 5/18，组不分类型）；是否发生 = F1 的 share 曲线 + w/o BAPO 对照。

Figure 18 shows that the share of scored forecasts stays stable, but not what kinds of checks they are.

✓ 总占比稳定与 mix 变化相容；而且 Figure 18(a) 的 7B 那条在降，回复里不要写 “holds”。

Could you split the hit rate in Figure 5(a) by type of check: checks run right after a code edit, reproduction runs before any fix, and re-runs with no code change in between? Could you also show how the share of each type changes during RL?

做（F1）。他的三类有重叠（修前的原样重跑既是 B 又是 C），所以要给精确标注规则：先判 C 再判 B，A 再拆 A-rerun / A-new。多给 base-rate 控制（MCC、best-constant），否则 Type A 命中率会因为 fix 变好而被动上升。

Only the first type shows whether the agent can foresee the effect of its own edits.

✓ 同意，而且就用这句接过去：Type A 是修订版的 headline（引导点二）。

Question 2 · TTO 的打分与对照

ANTE-TTO picks among candidates with a fixed score (Eq. 23). A forecast of PASS gets 3 points, "no verdict" gets 1, FAIL gets 0, and a predicted clean exit adds 1.

✓ p.32–33。

Under the protocol, edits and views are always forecast as "no verdict", so they score at most 2, while a check forecast to pass scores 3 or 4. So any check predicted to pass beats every edit, and two different edits can only be told apart by their predicted exit code.

✓ 算术对（repo harness）。动作：认，不绕。

In other words, the score favours optimistic forecasts: a useful check that is correctly expected to fail ranks below an unverified edit.

✓ check 预测 FAIL + clean exit = 1 < edit NA + clean exit = 2。他隐含的替代排序是“任何 check 排在任何 edit 前面”，那正是对照规则 (c) test-first。动作：认算术，说明 PASS ≻ NA ≻ FAIL 的意图（预测能解决的优先），然后把他的排序当对照跑（引导点三：把批评变成我们表里的一行）。

The paper's own SWE example shows this (Figure 14 and Appendix O.1). At step 7, the drafted edit scored 2. The winning candidate ran the reproduction script and scored 4 because it forecast PASS, based on the wrong belief that the bug was not there. The script in fact failed.

✓ 全对（p.33、p.39–40，论文自己写 “refutes its abstained forecast”）。动作：这个例子从“TTO 的成功案例”改为“重采样让错误 forecast 被执行揭穿”的例子，或换例子。

Had the forecast been correct (FAIL with a clean exit), this candidate would have scored only 1. The rule would then have kept the unverified edit that the paper says ANTE-TTO avoided.

✓ 反事实算术对。动作：认；它正说明 ranking 的贡献必须用对照量，不能靠案例。

The TTO gains in Table 1 are also small (+1.4, +2.4 and +0.6 points) and have no error bars. At 7B and 32B they are smaller than ANTE's own run-to-run standard error.

✓ p.36 的 s.e. 1.7 / 0.9 / 1.1。动作：3 seed 的 ± 和配对 CI（F2）。

On the same candidates, how does ANTE-TTO compare with simple rules that ignore the forecast? Examples are picking a test-running candidate whenever there is one, or picking at random.

做（F2 的 (b) random、(c) test-first）。test-first 的意思：K+1 个候选里只要有一个动作是跑测试/复现脚本，就选它（多个就选第一个），没有就保留 greedy 草稿；完全不看 forecast。它模拟的是 Eq. 23 实际在做的事（预测 PASS 的 check 永远赢 edit），所以是最尖锐的对照：若 (c) ≈ (a)，forecast 在选择里就没贡献。

How often is the winning candidate's forecast correct?

做，而且要报两个量（下面 M1/M2）：他问的是 M1（被执行的赢家的 forecast 事后对不对，日志直接数）；你想报的“选出了好的候选的比例”是 M2，要在小样本上把 K+1 个候选都执行一遍才知道谁好。

Please report these with error bars.

✓ 每个数带 cluster bootstrap（按 task 重采样）的 95% CI；配对差用 McNemar 的不一致对。

Experiments, expected results, and why

实验规划：设计、预期结果和成因（表格在下一节 English tables）

这里只放设计、预期与成因（中英文）；表格一律英文，放在下一节，形状就是最后贴进回复的样子。

F1 · hit rate 按 check 类型拆分（L：只分析日志；小规模容器重放）答 Q1 · B6 · 表 = Tables F1, F1b, F1c

Labeling rule (apply in order, first match wins; key = check identity × source state)	Type	What a code-blind rule can guess
The same check already ran on the same source state	C re-run	The answer: the previous result
The source state equals the base commit (including after git stash)	B pre-fix	The issue says the target behaviour is broken → FAIL
The same check ran earlier on another source state	A-rerun	The result before the edit; whether the edit flipped it requires understanding the code
Otherwise	A-new	Nothing

check 身份 = 归一化命令 + 它运行的脚本/测试文件的内容 hash（改了 reproduce_issue.py 就算新 check）；源码状态 = 对 base commit 的 diff（非测试文件 + 包内新文件）的 hash，agent 的临时脚本不算源码。这正好把 O.3 step 24 判成 B（脚本变了、源码没变）。边界案例、伪代码和 ≥95% 人工一致性的 pilot 都在 Jifan 那页 §3b，发射时照搬。结果表 = Table F1（每类 share / hit / best-constant / MCC，外加 Type A “改坏了”的召回和 +25 的 shift-share 分解）；对照 = Table F1b（w/o BAPO 的 mix，若 mix 只在 BAPO 下移动就是步级 credit 造成的）；probe set = Table F1c（同一批 held-out context 逐 checkpoint 只做前向，最干净的 “forecaster 本身变没变好”，建议做）。

Possible reading	Type-A MCC	Mix term	Mechanism	Lead sentence of the reply
1 · Edit foresight improved	rises	small	The prediction loss trains the forecast on every verifiable step (Eq. 3); the model learns to judge the effect of its own edits	The Type-A curve and the decomposition: “most of the rise is post-edit accuracy”
2 · A real gain plus a selection effect	rises	large	Forecasts improve and the agent also runs more predictable checks (BAPO’s incentive on the choice of check)	The Type-A number first; the BAPO incentive as a stated limitation
3 · The rise is mix and/or base rate	flat	large	+α does not depend on how hard the check is; predictable checks earn above-average W in their group and the policy gradient favours them	Narrow the claim (Jifan’s 4b template); consider the mitigation: no decision reward and no anchor group for C steps
4 · Flat on-policy, probe set improves	flat	·	Forecasts improved but post-edit checks became harder (more full-suite runs)	The probe set (Table F1c) plus a breakdown by check scope

我们事先不知道落在哪一行（memory 里只有总 hit rate 0.44→0.63 单调升、NA 率 0.47→0.23 单调降，没有按类型的数）。四行都要有现成的句子，跑出来按行填。

F2 · TTO：同候选对照 + 赢家正确率 + 误差棒（E + L）答 Q2 · B9/B10 · 表 = Tables F2, F2b, F2c（与 rebuttal1 的 R2a–R2c 同一批跑）

表的形状就是 Kangrui 画的那张（Table F2），加 B_tok、配对 CI 与不一致对（Table F2b）和 M1′。四个臂的候选集完全相同（同触发点、同 K=3、同温度），只换选法。

Metric	How it is computed	Source	Question it answers
M1 winner-forecast correctness (the reviewer’s question)	Among executed winners, the share whose check-field forecast matched the realized outcome; conclusive winners only, NA reported separately; cluster bootstrap over tasks	TTO evaluation logs: candidates, scores, chosen index and next observation at every trigger step	Whether the win rests on an accurate forecast
M2 selection quality (your question: how often the chosen candidate was a good one)	On about 100 trigger steps, replay the trajectory prefix in the container and execute all K+1 candidates; the best set has the highest realized Eq. 23 score; report P(selected ∈ best set), the all-tie steps separately, and the chance rate \|best\| / (K+1) on the same candidate sets	Container replay plus the candidate texts in the logs	How much better than random the ranking picks on steps where the candidates differ; the same executions give δ_k for Reviewer 3Rua
M3 trajectory level (“a trajectory whose chosen action succeeded”)	The 2×2 of greedy vs TTO on the same tasks: resolved under TTO only / under greedy only / both / neither; the discordant pairs feed McNemar. A second layer by whether a draft was ever replaced, with the selection-bias caveat	Per-task results of the two arms and the switch flags in the logs	How many tasks the replacements saved and how many they broke

预期与成因：random 持平或略降（3/4 的概率把 greedy 换成 T=1.0 单采样本，0823 probe 里 bo4 单遍 16/22/21 对 greedy 25）；test-first 很可能 ≈ Eq. 23（算术决定：只要有预测 PASS 的 check 它就赢，两者只在“只有预测 FAIL 的 check”或“没有 check 候选”时不同）；TTO − off 可能不显著（0823 的 7B n=100 预注册 probe 没有任何配置显著优于关掉，印值 +1.4 / +0.6 也小于 s.e.）；M1 可能不高（world model 系统性乐观、少报失败，O.1 step 7 是一例），M1 − M1′ 才是 ranking 选到更准 forecast 的证据。若 test-first ≈ Eq. 23 且增益不显著：回复如实写，TTO 改写为 “belief-triggered resampling”，单独报触发器的价值（Table F2c 最后一行；0823 离线测量 7B 1.7× / 14B 2.5× / 32B 3.1×，重算后再用）。

F3 · Figure 1 重画（W）答 B5 · 和 zcGj C2 的字号配色一起改

Panel	Now (a schematic the schema cannot produce)	Redrawn from a logged trajectory
Top: the same task under the three agents	The belief compares two fixes before the edit and predicts that A breaks the callers while B passes everything	ANTE applies a fix → writes the forecast before running the check (for example O.4 step 21: FAIL, trusted) → execution confirms it → revises the fix → forecasts again → PASS confirmed → submits. Reactive and Retrospective keep their “submit unchecked / record after the fact” contrast
Caption	“only ANTE forecasts the edit’s effect and lets execution settle it before committing”	“ANTE forecasts the outcome of the check that follows each edit, before running it; execution settles the forecast before the agent commits”
Source trajectory	·	Preferably a 32B A-new step from the F1 labeling whose FAIL forecast was confirmed; O.4 step 21 as the fallback; every quotation in the figure must be traceable to the trajectory

F4 · Kangrui 的点 (2)：verified belief 是否左右下一步（E，只生成不执行，可选）表 = Table F4

做法：取约 500 个已记录的 check 步，把上下文里上一个已结算 forecast 翻转（PASS↔FAIL），greedy 重新生成下一步，数动作变化率（工具 + 目标文件 + 归一化命令不同），对照是翻转一个无关 token。便宜（纯生成），证明 “belief 被用上了”（Eq. 1 里 action 以 belief 为条件），但不证明 “belief 准”，放附录别当主证据。Kangrui 的另一半 “推理时抹掉 belief block 看 resolve rate 掉多少” 不建议：会掉，但原因混杂（协议外输入、格式漂移），审稿人会说是 distribution shift。

F5 · Kangrui 的点 (3)：terminal harness 上按预测给候选 fix 排序（E，可选）表 = Table F5

这是唯一 “候选 edit 按预测整局结果排序” 的场景（Figure 14 右）；Table 2 没有 TTO 行，Kangrui 的 “[+Z] on Terminal-Bench” 要新跑（tmax-9B，3 个 benchmark × 3 seed，加 random 对照）。预期不敢写：terminal 的 check 字段是整局预测、只经 decision reward 训练（App. J），它对候选 fix 的判别力没测过。做了就如实报；没做就把那句从回复里删掉。

Strengths 栏的四条（B1–B4）在哪答

Item	Reviewer’s words	Where it is answered	Status
B1 evaluation budget	“A larger evaluation budget is stated only for ANTE … part of the gap comes from budget”	The placeholder paragraph in Comment 4	Depends on the actual evaluation budget of each Table 1 row; waits for your decision; no defence is drafted here
B2 headline against Reactive	“measured against the only baseline trained without the format reward and budget warnings”	Retrospective reported beside Reactive in the abstract and §3.2; Table R4 adds Retrospective to the OOD comparison; Table B2	Wording + evaluation
B3 forecast ≈ memory at the small scales	“add almost nothing at the two smaller scales”	Stated as such (+0.2 and +0.2); the 32B gap 54.2 vs 52.4; the w/o BAPO row re-evaluated 3× (Table R1)	Wording + evaluation
B4 a single training run	“BAPO’s extra few points come from a single training run”	Table R1: the isolation control plus training seeds at 14B	Training (compute in the rebuttal1 tab)

Rebuttal-ready tables

Tables as they will be posted in the reply to Reviewer BwwJ (English)

Final form. F1–F1c answer Q1; F2–F2c answer Q2 and are the same runs as Tables R2a–R2c for Reviewer 3Rua; F4 and F5 are optional; B2 restates the comparison concerns of the Strengths field with the submitted numbers. Dashed orange cells are placeholders to be filled only from the runs.

Table F1 · Hit rate by type of check during RL (32B; 7B and 14B in the new appendix)

Every conclusive step of the logged RL rollouts is labeled by the check it runs (normalized command plus a content hash of the script or test file) and the source state (hash of the diff against the base commit over non-test files, new package files included): C if the same check already ran on the same source state; B if the source state equals the base commit, including after git stash; A otherwise, split into A-rerun (the same check ran earlier on a different source state) and A-new. Start and end are the first and last bins of 10 RL steps. 95% CIs by cluster bootstrap over tasks.

Type of check	Share of conclusive steps (start → end)	Hit rate (start → end)	Best constant forecast (end)	MCC (start → end)
A-rerun: the same check re-run after an edit	xx% → xx%	xx → xx	xx	.xx → .xx
A-new: a check run for the first time on edited code	xx% → xx%	xx → xx	xx	.xx → .xx
B: reproduction run before any fix	xx% → xx%	xx → xx	xx	.xx → .xx
C: re-run with no code change in between	xx% → xx%	xx → xx	xx	.xx → .xx
All conclusive steps (Figure 5(a))	100%	44 → 69	—	—

Recall on post-edit checks that fail (forecast FAIL and realized FAIL, the case of Figure 1): xx% → xx%. Shift-share decomposition of the +25 points: a from accuracy on post-edit checks, b from pre-fix checks and re-runs, m from the change in mix (Δh = Σ_k avg(s_k)·Δh_k + Σ_k avg(h_k)·Δs_k, an identity). On 30 hand-labeled trajectories the labeling rule agrees with manual labels on xx% of steps.

Markdown for OpenReview

**Table F1. Hit rate by type of check during RL (32B; 7B and 14B in the new appendix)** Every conclusive step of the logged RL rollouts is labeled by the check it runs (normalized command plus a content hash of the script or test file) and the source state (hash of the diff against the base commit over non-test files, new package files included): C if the same check already ran on the same source state; B if the source state equals the base commit, including after git stash; A otherwise, split into A-rerun (the same check ran earlier on a different source state) and A-new. Start and end are the first and last bins of 10 RL steps. 95% CIs by cluster bootstrap over tasks.

| Type of check | Share of conclusive steps (start → end) | Hit rate (start → end) | Best constant forecast (end) | MCC (start → end) |
|---|---|---|---|---|
| A-rerun: the same check re-run after an edit | [xx% → xx%] | [xx → xx] | [xx] | [.xx → .xx] |
| A-new: a check run for the first time on edited code | [xx% → xx%] | [xx → xx] | [xx] | [.xx → .xx] |
| B: reproduction run before any fix | [xx% → xx%] | [xx → xx] | [xx] | [.xx → .xx] |
| C: re-run with no code change in between | [xx% → xx%] | [xx → xx] | [xx] | [.xx → .xx] |
| All conclusive steps (Figure 5(a)) | 100% | 44 → 69 | — | — |

Recall on post-edit checks that fail (forecast FAIL and realized FAIL, the case of Figure 1): [xx% → xx%]. Shift-share decomposition of the +25 points: [a] from accuracy on post-edit checks, [b] from pre-fix checks and re-runs, [m] from the change in mix (Δh = Σ_k avg(s_k)·Δh_k + Σ_k avg(h_k)·Δs_k, an identity). On [30] hand-labeled trajectories the labeling rule agrees with manual labels on [xx]% of steps.

Table F1b · Type mix with and without BAPO (32B)

The same labeling applied to the rollouts of ANTE w/o BAPO, which has the belief protocol and the prediction loss but no decision reward and no step credit. If the mix moves only under BAPO, the step credit is the cause.

Type of check	ANTE share (start → end)	ANTE w/o BAPO share (start → end)	Checks per trajectory, ANTE (start → end)
A (rerun + new)	xx% → xx%	xx% → xx%	x.x → x.x
B	xx% → xx%	xx% → xx%	x.x → x.x
C	xx% → xx%	xx% → xx%	x.x → x.x

Markdown for OpenReview

**Table F1b. Type mix with and without BAPO (32B)** The same labeling applied to the rollouts of ANTE w/o BAPO, which has the belief protocol and the prediction loss but no decision reward and no step credit. If the mix moves only under BAPO, the step credit is the cause.

| Type of check | ANTE share (start → end) | ANTE w/o BAPO share (start → end) | Checks per trajectory, ANTE (start → end) |
|---|---|---|---|
| A (rerun + new) | [xx% → xx%] | [xx% → xx%] | [x.x → x.x] |
| B | [xx% → xx%] | [xx% → xx%] | [x.x → x.x] |
| C | [xx% → xx%] | [xx% → xx%] | [x.x → x.x] |

Table F1c · Fixed probe set: the forecaster scored on the same contexts at every RL checkpoint (if run)

Held-out conclusive contexts outside the RL pool, with history and reasoning teacher-forced up to the value tokens and the realized outcome fixed; each checkpoint gives the PASS and FAIL token probabilities. Forward passes only, no generation and no execution. This removes the agent's choice of checks entirely.

Type of context	Items	Log-loss (step 0 → 100)	AUROC (step 0 → 100)	MCC (step 0 → 100)
A post-edit	≥300	x.xx → x.xx	.xx → .xx	.xx → .xx
B pre-fix	200	x.xx → x.xx	.xx → .xx	.xx → .xx
C re-run	200	x.xx → x.xx	.xx → .xx	.xx → .xx

Markdown for OpenReview

**Table F1c. Fixed probe set: the forecaster scored on the same contexts at every RL checkpoint (if run)** Held-out conclusive contexts outside the RL pool, with history and reasoning teacher-forced up to the value tokens and the realized outcome fixed; each checkpoint gives the PASS and FAIL token probabilities. Forward passes only, no generation and no execution. This removes the agent's choice of checks entirely.

| Type of context | Items | Log-loss (step 0 → 100) | AUROC (step 0 → 100) | MCC (step 0 → 100) |
|---|---|---|---|---|
| A post-edit | [≥300] | [x.xx → x.xx] | [.xx → .xx] | [.xx → .xx] |
| B pre-fix | [200] | [x.xx → x.xx] | [.xx → .xx] | [.xx → .xx] |
| C re-run | [200] | [x.xx → x.xx] | [.xx → .xx] | [.xx → .xx] |

Table F2 · ANTE-TTO against selection rules that ignore the forecast, on the same candidates (SWE-bench Verified pass@1, %)

Same trigger steps (abstain or predicted FAIL), the same greedy draft plus 3 sampled alternatives, the same temperature; only the choice of the winner changes. Mean ± s.e. over 3 candidate-sampling seeds; B_tok measured as in Figure 4. M1 and M1′ count, on conclusive steps, how often the forecast of the executed candidate matched the realized outcome.

Selection rule (same triggers, K = 3)	7B	14B	32B
No TTO (greedy draft; Table 1)	27.2 ± 1.7 · 1.00×	46.2 ± 0.9 · 1.00×	57.2 ± 1.1 · 1.00×
Random among the K+1 candidates	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×
Test-first: a test-running candidate whenever there is one, otherwise the draft	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×
ANTE-TTO (belief score, Eq. 23)	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×	xx.x ± x.x · 1.xx×
M1: winner-forecast correctness (%)	xx [ , ]	xx [ , ]	xx [ , ]
M1′: forecast correctness of the replaced greedy drafts (%)	xx [ , ]	xx [ , ]	xx [ , ]

Paired 95% bootstrap intervals of each arm against the greedy draft are given in Table F2b.

Markdown for OpenReview

**Table F2. ANTE-TTO against selection rules that ignore the forecast, on the same candidates (SWE-bench Verified pass@1, %)** Same trigger steps (abstain or predicted FAIL), the same greedy draft plus 3 sampled alternatives, the same temperature; only the choice of the winner changes. Mean ± s.e. over 3 candidate-sampling seeds; B_tok measured as in Figure 4. M1 and M1′ count, on conclusive steps, how often the forecast of the executed candidate matched the realized outcome.

| Selection rule (same triggers, K = 3) | 7B | 14B | 32B |
|---|---|---|---|
| No TTO (greedy draft; Table 1) | 27.2 ± 1.7 · 1.00× | 46.2 ± 0.9 · 1.00× | 57.2 ± 1.1 · 1.00× |
| Random among the K+1 candidates | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] |
| Test-first: a test-running candidate whenever there is one, otherwise the draft | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] |
| ANTE-TTO (belief score, Eq. 23) | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] | [xx.x ± x.x] · [1.xx×] |
| M1: winner-forecast correctness (%) | [xx [ , ]] | [xx [ , ]] | [xx [ , ]] |
| M1′: forecast correctness of the replaced greedy drafts (%) | [xx [ , ]] | [xx [ , ]] | [xx [ , ]] |

Paired 95% bootstrap intervals of each arm against the greedy draft are given in Table F2b.

Table F2b · Per-task outcomes, ANTE-TTO against the greedy draft (500 tasks, seed-pooled)

Discordant pairs are the input to McNemar's test; the paired difference and its 95% bootstrap interval are over tasks.

Outcome pair	7B	14B	32B
Resolved under both	n	n	n
Resolved under ANTE-TTO only	n	n	n
Resolved under the greedy draft only	n	n	n
Resolved under neither	n	n	n
Paired Δ, 95% CI; McNemar p	+x.x [ , ]; p = x.xx	+x.x [ , ]; p = x.xx	+x.x [ , ]; p = x.xx
Among tasks with at least one replaced draft: resolved (TTO) vs resolved (greedy)	xx% vs xx%	xx% vs xx%	xx% vs xx%

The last row is descriptive: triggers fall on uncertain steps, so tasks with a replaced draft are harder on average.

Markdown for OpenReview

**Table F2b. Per-task outcomes, ANTE-TTO against the greedy draft (500 tasks, seed-pooled)** Discordant pairs are the input to McNemar's test; the paired difference and its 95% bootstrap interval are over tasks.

| Outcome pair | 7B | 14B | 32B |
|---|---|---|---|
| Resolved under both | [n] | [n] | [n] |
| Resolved under ANTE-TTO only | [n] | [n] | [n] |
| Resolved under the greedy draft only | [n] | [n] | [n] |
| Resolved under neither | [n] | [n] | [n] |
| Paired Δ, 95% CI; McNemar p | [+x.x [ , ]; p = x.xx] | [+x.x [ , ]; p = x.xx] | [+x.x [ , ]; p = x.xx] |
| Among tasks with at least one replaced draft: resolved (TTO) vs resolved (greedy) | [xx% vs xx%] | [xx% vs xx%] | [xx% vs xx%] |

The last row is descriptive: triggers fall on uncertain steps, so tasks with a replaced draft are harder on average.

Table F2c · Selection quality on replayed trigger steps (M2)

On a subsample of trigger steps, all K+1 candidates are executed from a replay of the trajectory prefix. The best set holds the candidates with the highest realized Eq. 23 score; chance is the expected rate of a uniform pick on the same candidate sets; steps on which all candidates tie are excluded from the rate and reported separately.

Quantity	7B	14B	32B
Trigger steps replayed	100	100	100
Steps with a strict best set (not all tied)	n (xx%)	n (xx%)	n (xx%)
Selected candidate in the best set	xx% vs chance xx%	xx% vs chance xx%	xx% vs chance xx%
Test-first pick in the best set (same steps)	xx%	xx%	xx%
Share of truly failing steps among triggered steps vs untriggered steps (value of the trigger)	xx% vs xx%	xx% vs xx%	xx% vs xx%

Markdown for OpenReview

**Table F2c. Selection quality on replayed trigger steps (M2)** On a subsample of trigger steps, all K+1 candidates are executed from a replay of the trajectory prefix. The best set holds the candidates with the highest realized Eq. 23 score; chance is the expected rate of a uniform pick on the same candidate sets; steps on which all candidates tie are excluded from the rate and reported separately.

| Quantity | 7B | 14B | 32B |
|---|---|---|---|
| Trigger steps replayed | [100] | [100] | [100] |
| Steps with a strict best set (not all tied) | [n (xx%)] | [n (xx%)] | [n (xx%)] |
| Selected candidate in the best set | [xx%] vs chance [xx%] | [xx%] vs chance [xx%] | [xx%] vs chance [xx%] |
| Test-first pick in the best set (same steps) | [xx%] | [xx%] | [xx%] |
| Share of truly failing steps among triggered steps vs untriggered steps (value of the trigger) | [xx% vs xx%] | [xx% vs xx%] | [xx% vs xx%] |

Table F4 · Does the in-context forecast steer the next action? (inference-only intervention, if run)

On logged check steps, the last settled belief in the context is edited and the next step is regenerated greedily; nothing is executed. The next action counts as changed when the tool, the target file or the normalized command differs.

Intervention on the context	Next action changed (%)	Steps
Flip the last settled forecast, PASS ↔ FAIL	xx [ , ]	500
Flip the last decision, TRUST ↔ ABSTAIN	xx [ , ]	500
Control: perturb an unrelated token of the same block	xx [ , ]	500

Markdown for OpenReview

**Table F4. Does the in-context forecast steer the next action? (inference-only intervention, if run)** On logged check steps, the last settled belief in the context is edited and the next step is regenerated greedily; nothing is executed. The next action counts as changed when the tool, the target file or the normalized command differs.

| Intervention on the context | Next action changed (%) | Steps |
|---|---|---|
| Flip the last settled forecast, PASS ↔ FAIL | [xx [ , ]] | [500] |
| Flip the last decision, TRUST ↔ ABSTAIN | [xx [ , ]] | [500] |
| Control: perturb an unrelated token of the same block | [xx [ , ]] | [500] |

Table F5 · Test-time selection on the terminal harness, where every step forecasts the episode outcome (Qwen3.5-9B, Tmax agent; resolved, %; if run)

On the terminal harness the check field forecasts the hidden verifier's outcome of the whole episode at every step, edits included (Appendix J), so ANTE-TTO ranks candidate fixes by their forecast before any of them runs (Figure 14, right). Table 2 reports no TTO row; these are new runs with the Table 2 protocol, 3 candidate-sampling seeds.

Method	TB-Dev	TB-Lite	TB-2.0
ANTE, greedy (Table 2)	42.5 ± 3.3	45 ± 1.3	24.7
+ ANTE-TTO (trigger on the predicted exit field, Appendix J)	xx.x ± x.x	xx ± x	xx.x ± x.x
+ random pick among the K+1 candidates	xx.x ± x.x	xx ± x	xx.x ± x.x

Markdown for OpenReview

**Table F5. Test-time selection on the terminal harness, where every step forecasts the episode outcome (Qwen3.5-9B, Tmax agent; resolved, %; if run)** On the terminal harness the check field forecasts the hidden verifier's outcome of the whole episode at every step, edits included (Appendix J), so ANTE-TTO ranks candidate fixes by their forecast before any of them runs (Figure 14, right). Table 2 reports no TTO row; these are new runs with the Table 2 protocol, 3 candidate-sampling seeds.

| Method | TB-Dev | TB-Lite | TB-2.0 |
|---|---|---|---|
| ANTE, greedy (Table 2) | 42.5 ± 3.3 | 45 ± 1.3 | 24.7 |
| + ANTE-TTO (trigger on the predicted exit field, Appendix J) | [xx.x ± x.x] | [xx ± x] | [xx.x ± x.x] |
| + random pick among the K+1 candidates | [xx.x ± x.x] | [xx ± x] | [xx.x ± x.x] |

Table B2 · SWE-bench Verified gains over both baselines (pass@1 points; submitted Table 1)

The headline of the submission is measured against Reactive-RL; the revision reports both baselines side by side. Reactive-RL is the only variant trained without the format term and the budget warnings (Appendix G, p.31–32); Retrospective-RL carries both.

Comparison	7B	14B	32B
ANTE − Reactive-RL	+2.8	+4.4	+6.2
ANTE − Retrospective-RL	+2.0	+2.8	+4.8
ANTE w/o BAPO − Retrospective-RL (the belief and its prediction loss alone)	+0.2	+0.2	+1.8
ANTE − ANTE w/o BAPO (BAPO)	+1.8	+2.6	+3.0

All entries are from the submitted Table 1; the w/o BAPO row was evaluated once and is being re-evaluated 3× (Table R1).

Markdown for OpenReview

**Table B2. SWE-bench Verified gains over both baselines (pass@1 points; submitted Table 1)** The headline of the submission is measured against Reactive-RL; the revision reports both baselines side by side. Reactive-RL is the only variant trained without the format term and the budget warnings (Appendix G, p.31–32); Retrospective-RL carries both.

| Comparison | 7B | 14B | 32B |
|---|---|---|---|
| ANTE − Reactive-RL | +2.8 | +4.4 | +6.2 |
| ANTE − Retrospective-RL | +2.0 | +2.8 | +4.8 |
| ANTE w/o BAPO − Retrospective-RL (the belief and its prediction loss alone) | +0.2 | +0.2 | +1.8 |
| ANTE − ANTE w/o BAPO (BAPO) | +1.8 | +2.6 | +3.0 |

All entries are from the submitted Table 1; the w/o BAPO row was evaluated once and is being re-evaluated 3× (Table R1).

The isolation and seed table for the Strengths-field concern B4 is Table R1 in the 3Rua tab.

Draft response

回复稿（英文，四条，每条 ≤ 5,000 字符）

Comment 1 以 Kangrui 的三点为骨架改写：把 “We respectfully disagree that ANTE therefore does not anticipate…” 换成“事实全认、后果读法不同”，更难反驳；去掉还没跑的 [+Z] 断言，改成占位。Comment 2 以 Jifan 的草稿为底压缩。所有数字占位 [INSERT …]。

Comment 1 / 4 · the weaknesses: what the forecast is, and what anticipation means here

Thank you for reading the method at the level of its definitions. Every factual statement in the weaknesses about the repository schema is correct, and we answer the conclusions drawn from them with measurements rather than with wording. We take the points in order and cite the submitted PDF by page.

The facts we agree with. On repository tasks the forecast has two fields, the result of the check the current step runs and the exit status (p.2–3); an edit step is forecast NA plus the exit code (p.47); the summary receives no RL gradient (p.4); and Figure 1 depicts a two-fix comparison written before the edit that this schema does not produce. We will redraw Figure 1 from a logged trajectory in which the agent applies a fix, forecasts that the check which follows will fail, and is proven right by execution (the pattern of Appendix O.4, step 21), and we will rewrite the abstract and p.2 so that no sentence places the forecast before the edit.

Where we read the consequence differently. The unit of anticipation in ANTE is the pair edit → check, not the edit alone. Immediately after an edit, and before any consequence of it has been observed, the policy writes a forecast of the outcome of the check that tests that edit. That forecast concerns a result that does not yet exist; execution then confirms or refutes it, and the prediction loss and the decision reward are computed from that verdict (Eq. 3, Eq. 18). The code has already changed at that point, but in a sandboxed repository the change is reversible (the agent in Appendix O.2 stashes its patch and re-runs the tests at step 25), and the irreversible commitment is submission, which the protocol does not allow before a verified re-run (Appendix P, rule 5). Whether the agent foresees the effect of its own edits is therefore an empirical question about exactly those checks, and we now measure it directly: the hit rate on checks run right after a code edit, split out as you ask in Q1 (next comment). On the terminal harness the schema is different: the check field forecasts the hidden verifier's outcome of the whole episode at every step, edits included (Appendix J), and ANTE-TTO there ranks candidate fixes by that forecast before any of them runs (Figure 14, right). The revision states the two schemas side by side in §2.1 instead of describing only the repository one.

On the remaining three sentences. The concern that part of the rise in hit rate comes from easy checks is legitimate and testable; our reply to Q1 reports the requested split together with base-rate controls. The theory is presented as motivation: the paper states its idealization (p.19) and lists the three ways in which the implemented objective differs from the surrogate (p.21); the revision says this in the main text and adds the small-sample estimate of the terms of Proposition 2 requested by Reviewer 3Rua. "Omniscient over both time and environment state" is removed, and "internal world model" is defined in §1 as the policy's belief about the execution outcome of its next action, verified by execution, which is what every accuracy number in the paper measures.

Comment 2 / 4 · Q1: hit rate by type of check

Q1, hit rate by type of check. We agree that the aggregate hit rate of Figure 5(a) cannot answer this on its own: it averages over checks the agent chooses, and BAPO's step credit applies only to conclusive check steps (Eq. 5), so a shift toward predictable checks is possible in principle. We re-analyzed the logged RL rollouts as requested.

Labeling. For each conclusive step we record the check (the normalized command plus a content hash of the script or test file it runs) and the source state (a hash of the diff against the base commit over non-test files, new package files included). A step is a re-run (C) if the same check already ran on the same source state; pre-fix (B) if the source state equals the base commit, including after git stash; and post-edit (A) otherwise, split into A-rerun, the same check ran earlier on a different source state, and A-new. Editing the agent's own reproduction script changes the check, not the source, so the Appendix O.3 example is B. On [INSERT: 30] hand-labeled trajectories the rule agrees with manual labels on [INSERT]% of steps.

Results (32B; 7B and 14B in the new appendix). [INSERT: Table F1: for A-rerun, A-new, B and C, the share of conclusive steps and the hit rate at the start and the end of RL, the best constant forecast, and MCC.] Of the +25-point rise, [INSERT: a] points come from higher accuracy on post-edit checks, [INSERT: b] from pre-fix checks and re-runs, and [INSERT: m] from the change in mix (exact shift-share decomposition). Without BAPO the type shares move by [INSERT], against [INSERT] with BAPO.

Base rate. A post-edit hit rate can also rise because the fixes improve while the forecast stays PASS, so we report MCC and balanced accuracy per type, as Table 12 does, and the recall on post-edit checks that fail, which is the case Figure 1 is about. [INSERT: Type-A MCC start → end; recall on failing post-edit checks start → end.] [INSERT if run: fixed probe set, the same held-out conclusive contexts scored by every RL checkpoint with history and reasoning teacher-forced and the outcome fixed; log-loss and AUROC on post-edit contexts.]

On O.3. You are right that this example is a pre-fix check: the forecast is the same before and after RL, and the hit comes from a better reproduction script. The revision labels it as such, shows the forecast at the post-patch re-run of the same trajectory (step 25), and adds a post-edit example [INSERT: task id].

Revisions. Figure 5(a) gains a per-type panel with MCC; Figure 18 gains the share of each type; the abstract reports the post-edit result as the world-model result.

Comment 3 / 4 · Q2: ANTE-TTO

Q2, ANTE-TTO. The arithmetic you give is right: under Eq. 23 a check forecast PASS outranks every edit, two edits differ only by their predicted exit code, and a check correctly forecast to FAIL with a clean exit scores 1, below an unverified edit at 2. The ordering PASS ≻ NA ≻ FAIL was chosen to rank a predicted resolution first; it does favour optimistic forecasts, and the O.1 step-7 winner is a case in which the forecast was wrong and the execution exposed it. We therefore report the controls you name, on identical candidates.

Controls. The trigger rule (abstain or predicted FAIL, Algorithm 3), K = 3 and the sampling temperature are fixed; only the choice among the K+1 candidates changes: (a) Eq. 23; (b) a uniform random pick; (c) test-first, take a test-running candidate whenever there is one and otherwise keep the draft, which is the ordering your analysis implies; (d) the greedy draft, no resampling. Each arm runs with 3 candidate-sampling seeds. We report mean ± s.e., the measured token budget B_tok as in Figure 4, a paired 95% bootstrap interval against (d) over the 500 tasks, and the discordant pairs (tasks resolved under (a) only, and under (d) only). [INSERT: Table F2 at 7B / 14B / 32B.] [INSERT: K ∈ {3, 5, 7}.]

Winner forecasts. From the same logs, the share of executed winners whose forecast matched the realized outcome on conclusive steps, next to the same share for the greedy drafts they replaced. [INSERT: Table F2, rows M1 and M1′.] On a subsample of [INSERT: 100] trigger steps we also execute all K+1 candidates from a replay of the trajectory prefix and report how often the selected candidate was among the best available by realized score, against the chance rate for the same candidate sets, with ties counted separately. [INSERT.]

What changes in the paper. The Table 1 TTO row is replaced by the measured mean ± s.e.; the sentences on p.8–9 that call its gains consistent, and the paragraph on p.9 that calls them the empirical side of Proposition 2, are rewritten to what the controls support. If (c) matches (a), we will say so and describe ANTE-TTO as belief-triggered resampling whose selection rule adds little beyond preferring a check; the value of the trigger is then reported on its own [INSERT: share of truly failing steps among triggered steps, against the untriggered rate].

Comment 4 / 4 · theory, the name, and the comparison concerns in the Strengths field

Remaining points.

(i) Theory. The bounds are standard and the assumptions idealized, as the paper states (p.19, p.21). The revision moves that statement into §2.3, presents the propositions as motivation for the prediction loss and the selection rule, and adds the small-sample estimate of the terms of Proposition 2 requested by Reviewer 3Rua [INSERT: numbers].

(ii) Name and abstract. "Omniscient over both time and environment state" is removed. "Internal world model" is defined in §1 as the policy's belief about the execution outcome of its next action, verified by execution; "world model accuracy" becomes "forecast hit rate on conclusive steps".

(iii) The comparison concerns in the Strengths field. Retrospective-RL is reported next to Reactive-RL as a second headline baseline in the abstract and §3.2, and is added to the out-of-distribution comparison of Table 3 [INSERT: Table R4]. At 7B and 14B the belief and its prediction loss add +0.2 over Retrospective (Table 1), which we now state as such; the 32B gap is 54.2 against 52.4, and the w/o BAPO row is re-evaluated three times to carry its own ± [INSERT]. The BAPO increment is re-measured with an isolation control that keeps the decision reward and changes only the advantage construction, and [INSERT: with training seeds at 14B] (Table R1, shared with Reviewer 3Rua). [INSERT: the evaluation budget of every Table 1 row, and the stopping rule of the Retrospective run in Figure 6 / Appendix O.2, after the authors' check of the evaluation records.]

We hope these measurements address the main reason for the score. If any point remains, we would value the chance to address it before the discussion closes.

Needs your decision

要你拍板（BwwJ 专属；算力的部分见 rebuttal1）

rollout 在不在：F1 要每个 RL 步的 rollout dump（memory 里只确认 32B v7 的 runs/…/rollout/{5,10}.jsonl 和 7B 的 1.jsonl 在 S3）。7B/14B/32B 最终那三条 run 的全程 rollout 是否齐；不齐的话按 Jifan 的办法在保存的 checkpoint 上固定题集重 roll，并在回复里说明。
容器重放：F1 的 A/B/C 精确标注、M2、3Rua 的 δ_k 都靠“按前缀重放到某步再执行”。R2E-Gym 的镜像还能拉吗、重放脚本谁写。做不了就用工具调用解析的近似标注（漏 sed -i 这类 bash 改动），并说明。
Figure 1 的轨迹：用 O.4 step 21（现成、32B RL、trusted FAIL 应验）还是从 F1 标注里挑一条 A-new 应验的；图里每句引文都要能在轨迹里找到（上一版 Overleaf 的 “force bytes in py2” 就找不到）。
F4 / F5 做不做：F4 便宜（纯生成）但只证明“belief 被用上”；F5 要 tmax-9B 的 TTO 评测（3 个 benchmark × 3 seed），Table 2 没有 TTO 行，不做就把 Kangrui 的 “[+Z] on Terminal-Bench” 从回复里删掉。
O.1 的例子：改写成“重采样让错误 forecast 被执行揭穿”，还是换一个赢家 forecast 正确的例子（要先从日志里找到）。
B1 那段：Comment 4 的占位要填“各行评测预算 + O.2 停在 50 步的原因”，等你查记录后定。
发帖顺序：他是两位 2 分之一、条目最多；skill 的建议是先回低分的。他的 Q1 只靠日志（最快），可以先发 Comment 1+2，TTO 和 Strengths 栏的数到了再发 3+4。

Precedents used

用到的先例（11/06 读的 10 个 OpenReview 线程；今天没新爬，OpenReview 的 API 仍 403）

Move	Source	Used where
Answer a long, detailed review item by item with page pointers, in neutral wording, never characterizing the reviewer	SWE-RL (NeurIPS 2025; the confidence-5 reviewer turned positive)	All of Comment 1; the right-hand column of the dissection
Clear a conceptual disagreement with one minimal example; use “misunderstanding” only in the note to the AC	Tree-GRPO (ICLR 2026)	The edit → check unit; the O.2 step 25 stash example
Run the control the reviewer asks for, report the failure regime, pair it with the fix	GiGPO (NeurIPS 2025)	F1 reading 3 plus the mitigation; “if (c) matches (a)” in F2
Concede the one sentence that overclaims, give the replacement wording, say which evidence stands	VAGEN (NeurIPS 2025)	The Figure 1 caption, the abstract sentence, “consistent gains”
Give “world model” an operational definition and the delta to the nearest work instead of arguing about the label	WMA (ICLR 2025)	End of Comment 1; Comment 4 (ii)
Negative: a control that comes out close while the method is still called necessary; a soft defence of a contested name	Dyna-Think, SPA (ICLR 2026, rejected)	The F2 contingency; “omniscient” simply removed
Negative: deferring experiments to the final version; promising what the uploaded PDF does not contain	Long-context SWE RL (rejected); one instance in WMA	Every [INSERT] becomes a number or is deleted before posting

ICLR 2027 · Submission35229 · ANTE

Rebuttal 作战图

每位审稿人先放原文，原文里的黑色编号（如 B5）点一下就跳到下面对应的分析和 rebuttal 建议。所有核对都以审稿人看到的投稿版 PDF（2026-09-25，50 页）为准，页码 p.N 指投稿版；没有看代码和实验记录，页面上的数字全部出自论文或审稿意见。

Reviewer 3Rua3 weak accept · 信心 3唯一的支持者：要把功劳拆清楚 Reviewer BwwJ2 weak reject · 信心 2读得最细：比较不干净、说的比做的多 Reviewer zcGj2 weak reject · 信心 3主要是写作：delta 和图

最先做的三件事

查评测预算（B1）：baseline 是不是也按 64K context、200 步评的。这决定 BwwJ 的主批评是写作问题还是硬伤。
按检查类型拆 hit rate（B6）：只用已有日志，是最值钱的一个分析。
纯写作就能改的：收窄 claim、删 “omniscient”、重画 Figure 1（B5、B8、A3），headline 改成对 Retrospective（B2），图重画（C2）。

Reviewer 3Rua

Rating 3 · Weak acceptConfidence 3P2 · I2 · S2

TA 要什么：认可机制和对照设计，要把功劳拆清楚：BAPO 里是“按 belief 分组”还是 decision reward 起作用，TTO 里是 ranking 还是多采样起作用；再把 claim 收窄到 execution-outcome prediction。做了 isolation run 和 TTO 对照最可能升分；回复里任何夸大都可能让唯一的支持者掉分。

原文 · Official Review by Reviewer 3Rua21 Oct 2026, 03:39 (modified 06 Nov 2026, 04:29)

Paper SummaryANTE makes each agent step start with a belief state: a summary plus a prediction of the execution outcome (test PASS/FAIL/NA, abnormal exit) and a TRUST/ABSTAIN decision. Predictions are grounded by real execution labels, used for supervised warm-up, then optimized jointly with the task reward; BAPO groups trajectories by prediction × decision to anchor step-level credit, with an additional decision reward. At inference, ANTE-TTO samples K=3 candidates and selects by predicted score, without an external scorer. On SWE-bench Verified the method improves 7B/14B/32B models from 24.4/41.8/51.0 to 27.2/46.2/57.2, and prediction accuracy rises from 44% to 69%. The paper also reports Terminal-Bench results, 11 out-of-domain benchmarks, a cost analysis, and a theoretical appendix.

Presentation: 2: Generally understandable, but requires substantial effort in places.Informativeness: 2: I learned something interesting or useful.Soundness: 2: I have minor or fixable concerns about the paper’s soundness.

Critical Strengths

The mechanism is concrete and falsifiable: predict → verify by execution → convert into step-level credit → select at inference. Each interface is machine-readable, unlike another reflection prompt.
The controlled baselines are well designed for this literature: Reactive and Retrospective variants with paired SFT corpora and a paired RL pool, three model scales, two harnesses.
The theory appendix is not decorative: it explicitly carries residual and conditioning terms (Appendix D.5, Eq. 16–17) instead of presenting a clean bound that ignores them.
Cost transparency: the 1.31× token overhead of TTO is reported, including discarded candidates.

Critical Weaknesses

A1BAPO is not isolated. The "w/o BAPO" condition removes both the decision reward and belief-anchored grouping, so the reported drop supports the combination but not the claimed novelty of grouping by belief. Fixing the prediction loss and decision reward while only changing the advantage construction would settle this.
A2The TTO gain is close to noise and lacks the decisive control. Gains are +1.4/+2.4/+0.6, while the reported error bars come from repeated evaluation of one checkpoint, not training seeds; +0.6 is within noise. More importantly, there is no control at the same trigger positions, candidate count, and budget using random or simple selection, so it is unclear how much of the gain comes from belief ranking rather than extra sampling.
A3"World model" overstates what is modeled. What is predicted is a discrete test outcome and exit status — a result predictor, not a state model of the environment. The out-of-domain +6.2 average (MMLU-Pro +10.4) is presented with a stronger causal reading than a single belief intervention supports.
A4Theory-to-practice gap. Proposition 2 depends on inter-candidate KL, a mutual-information term, and the point-prediction error ζ; none of these are estimated empirically, so validity of the bound does not imply tightness.

Questions For Authors

A1Can you report a BAPO isolation run: same prediction loss, same decision reward, same data and budget, only the step-level advantage construction changed? How much of the +1.8–3.0 remains?
A2For TTO, can you compare against random/simple candidate selection at identical trigger positions, K, and token budget, with variance over training seeds?
A4Can you estimate the extra terms in Proposition 2 (candidate divergence, ζ) on a small sample, to indicate whether the bound is tight enough to be informative?
A3Would you consider narrowing the "world model" framing to execution-result prediction, and qualifying the cross-domain causal claim accordingly?

Critical Strengths注：这一栏里写的其实全是批评
The main comparison does not cleanly show that the forecasts cause the gains, and this is the main reason for my score. B1A larger evaluation budget is stated only for ANTE. In the main case study, a baseline run stops at the training step limit while it is still editing. If the baselines had a smaller budget, part of the gap comes from budget, not from the method. B2The headline gain is also measured against the only baseline trained without the format reward and budget warnings. B3Against the cleaner memory baseline on SWE-bench Verified, the forecast and its prediction loss add almost nothing at the two smaller scales. B4BAPO's extra few points come from a single training run, and the error bars only reflect re-evaluating one checkpoint.

Critical Weaknesses B5The claims go well beyond what the method actually does. The title, the abstract and Figure 1 all present the agent as foreseeing an edit's effect before it makes the edit. In Figure 1, the belief written before the edit compares two possible fixes. It predicts that changing the function's output would pass the target test but break other code that calls the function, while adding a default value keeps all tests passing, so the agent picks the second fix. The method as defined cannot produce such forecasts. The forecast that execution checks has only two discrete fields: the result of the check run in the current step, and whether the command exits cleanly. An edit step is always forecast as "no verdict" plus the exit code, and the summary gets no training signal in RL. So the forecast written before an edit never says whether the edit will make the tests pass or break other tests. The agent only forecasts a test result later, when it runs the tests, and by then the code has already been changed. For the same reason, ANTE-TTO cannot rank candidate edits by their effect on the tests. B6The rising hit rate may also partly come from easy checks on scripts the agent writes itself. B7The theory applies standard bounds under idealized assumptions that do not cover the algorithm as implemented. B8Calling this an internal world model, and the abstract's stated aim of one that is "omniscient over both time and environment state," goes far beyond this evidence.

Questions For Authors

B6The paper's key evidence that the world model gets better is the rising forecast hit rate on conclusive steps, that is, checks that return PASS or FAIL (Figure 5(a)). At 32B it goes from 44% to 69% during RL. These checks are tests that the agent chooses to run, including reproduction scripts it writes itself. The protocol also asks the agent to make each script print its own PASS or FAIL verdict (Appendix P). Some of these forecasts are easy. One example is predicting that a bug-reproduction script fails before any fix. Another is predicting that a script re-run with no code change gives the same result as last time. The paper's own example of the belief sharpening during RL (Appendix O.3) is of the first kind: the improved forecast is that the reproduction script fails on the still-unfixed code. BAPO also gives credit to the action at a step whose trusted forecast comes true, so the policy may even learn to run more such easy checks. Figure 18 shows that the share of scored forecasts stays stable, but not what kinds of checks they are. Could you split the hit rate in Figure 5(a) by type of check: checks run right after a code edit, reproduction runs before any fix, and re-runs with no code change in between? Could you also show how the share of each type changes during RL? Only the first type shows whether the agent can foresee the effect of its own edits.
B9ANTE-TTO picks among candidates with a fixed score (Eq. 23). A forecast of PASS gets 3 points, "no verdict" gets 1, FAIL gets 0, and a predicted clean exit adds 1. Under the protocol, edits and views are always forecast as "no verdict", so they score at most 2, while a check forecast to pass scores 3 or 4. So any check predicted to pass beats every edit, and two different edits can only be told apart by their predicted exit code. In other words, the score favours optimistic forecasts: a useful check that is correctly expected to fail ranks below an unverified edit. The paper's own SWE example shows this (Figure 14 and Appendix O.1). At step 7, the drafted edit scored 2. The winning candidate ran the reproduction script and scored 4 because it forecast PASS, based on the wrong belief that the bug was not there. The script in fact failed. Had the forecast been correct (FAIL with a clean exit), this candidate would have scored only 1. The rule would then have kept the unverified edit that the paper says ANTE-TTO avoided. B10The TTO gains in Table 1 are also small (+1.4, +2.4 and +0.6 points) and have no error bars. At 7B and 32B they are smaller than ANTE's own run-to-run standard error. On the same candidates, how does ANTE-TTO compare with simple rules that ignore the forecast? Examples are picking a test-running candidate whenever there is one, or picking at random. How often is the winning candidate's forecast correct? Please report these with error bars.

Flag For Ethics Review: NoRating: 2: Weak rejectionConfidence: 2: I have broad familiarity with this field but little or no direct research experience on this topic.

分析与 rebuttal 建议

可借力：他自己把 Retrospective 称为 “the cleaner memory baseline”；Soundness 给的是 “minor or fixable”

评测预算只写了 ANTE 的

待核WE

分析

前提成立：p.7 只写 the ANTE entries on SWE-bench Verified run with a 64K context and a 200-step cap，baseline 的评测预算全文没写。Figure 6 和 O.2 里的 Retrospective 正好在第 50 步结束（p.9、p.41），50 正是 RL 训练的步数上限（Table 9, p.31）。差距里有多少来自预算，论文本身答不了。

建议

先查记录，查清前不要写 “all rows share one budget”。各行都是 64K/200：改 p.7 那句，并解释 case study 为什么停在 50 步。baseline 预算更小：直接承认，用 64K/200 重评 Reactive-RL 和 Retrospective-RL（各 3 次，不用训练），如实报新差距。两种情况都附上各行的终止原因统计（submit / 步数上限 / context 上限）。

需要你

Table 1 每一行实际的 context 和 step cap；O.2 那条 Retrospective 为什么停在第 50 步；Figure 19 那条 Retrospective 为什么停在第 37 步（p.39）。

headline 对的是唯一没有 format 项的 Reactive

成立WE

分析

论文自己写了：Reactive has no block and its reward is exactly binary（p.31），训练时也收不到 format reminder 和 budget warning（p.32）。还有一个他没点到的不对称：Retrospective 和 ANTE 的 addendum 都有 verify 规则和 VERDICT 要求，Reactive 只有 base prompt（p.44、p.47–48）。所以 p.7 every matched pair isolates the belief 对 Reactive 不成立。

建议

直接承认。headline 改成对 Retrospective：它和 ANTE 共享 format 项、训练期提示和 verify 规则，ANTE 领先 2.0/2.8/4.8（SWE）和 2.0–5.5（terminal，p.7）。摘要的 “6.2% above a matched GRPO baseline”、贡献 4 和 p.7 改成两个 baseline 并列报，主动列出 Reactive 和 ANTE 的全部差异。OOD 表只有 Reactive：补评 Retrospective-RL-32B，补不了就把结论写成 “vs Reactive”。

小尺度上 forecast 单独几乎没用

成立WE

分析

7B、14B 上 w/o BAPO 对 Retrospective-RL 只多 0.2（25.4 vs 25.2，43.6 vs 43.4），而且 w/o BAPO 只评了一次（p.7）。看 ↑ 更弱：这两个尺度上 w/o BAPO 的 RL 增益（↑0.6、↑2.2）小于 Retrospective（↑2.8、↑3.2）。真正的支持在 32B：w/o BAPO 的 SFT 起点低 1.8（46.4 vs 48.2），RL 后反超 1.8（54.2 vs 52.4）。

建议

承认小尺度上 forecast + prediction loss 单独和 memory 打平，把 p.7 “matches or exceeds Retrospective” 改成按尺度如实描述。补一个结构性论点：forecast 能被执行检验，所以能当 BAPO 的分组 key 和 decision reward 的对象，memory 做不到；它的价值主要通过 BAPO 体现，32B 从更低起点反超是论文里最强的证据。给 w/o BAPO 三个 checkpoint 各补 2 次评测拿 ±。

BAPO 增益来自单次训练

成立ET

分析

± 是 s.e. over 3 evaluation runs of one checkpoint（p.7），全文没有训练 seed。7B 的 +1.8 约等于 ANTE 的 s.e. 1.7，terminal TB-Dev 的 +1.4 也在 ±3.3 以内。

建议

承认，回复里不要把评测 ± 说成训练方差。可以如实说：四组独立训练的对比（SWE 7B/14B/32B + terminal 9B）BAPO 增量方向都为正，但这代替不了 seed。先给 w/o BAPO 补评测 ±；算力允许再补 seed，优先 14B（和 7B 同算力、增量更大）。

需要你

补不补 seed、哪个尺度、讨论期内能否跑完。

说的比做的多：改代码之前的 foresight

成立W

分析

在 repository harness 上完全成立：App P 规定 dev 步（看、搜、改、装、提交）的 target_test 恒为 NA（p.47），summary 在 RL 中没有梯度（p.4）。Figure 1 在改代码之前比较 Forecast A/B 对 242 个测试的影响（p.2），既不是 schema 能产生的 forecast，也不会被验证。论文自己在 App N 的说法才是准的：回归要到 the next check 才暴露（p.39）。

step t · 改代码

target_test: "NA"

按定义不打分。Figure 1 却把“预见副作用”画在这里。

step t+1 · 跑测试之前

"PASS" | "FAIL"

foresight 在这里：代码已改、结果未见，先押注自己改动的效果。

step t+1 · 执行之后

VERDICT: PASS / FAIL

执行给出对错，进 prediction loss 和 decision reward。

建议

承认 Figure 1、摘要和引言说过头：方法验证的是“改完之后、跑检查之前对检查结果的 forecast”，不是改之前比较两种改法。用真实轨迹重画 Figure 1（例如 O.1 step 8–9：改完后预测 PASS，执行确认），删掉 p.2 anticipates that the proposed edit would break other tests，明说 repository 上 TTO 不能按测试影响给 edit 排序。不要拿 terminal 的 expect_success 辩护，它不进 prediction loss（p.35）。

需要你

选哪条真实轨迹重画 Figure 1。注意：现在 Overleaf 的 p.9 新写了 ANTE's belief predicts what the next edit will change，正好是被批评的说法，修订版也要改。

hit rate 可能来自送分题

前提成立LW

分析

44% → 69% 是 agent 自选检查上的 hit rate，包括自写的 repro 脚本，App P 还要求脚本自己打印 VERDICT（p.47）。O.3 比他说的更弱：RL 前后的 forecast 完全一样（FAIL、exit false，p.42），变的是 RL 后那次脚本真的失败了，更像“动作变了”而不是“预测变准了”。另外论文说 settled share “holds”（p.37），但 Figure 18(a) 上 32B 从约 55% 升到约 65%，7B 从约 65% 降到约 50%（p.38）。

建议

照做，只用已有日志：把 conclusive 步分成 ① 改完代码后的检查 ② 修复前的 repro ③ 代码没变的重跑，报各类 hit rate 和占比随训练步的变化，写明 hit 的定义。撤下 O.3 这个例子，把 Figure 18(a) 的 “holds” 改成如实描述。论文里现成的“改完后检查命中”的例子（O.1 step 9、O.4）只当轶事，结论以拆分结果为准。

需要你

分类规则怎么定；如果 ① 这一类不升，world model 的定位要提前想好怎么写。

理论假设太理想

部分成立W

分析

大体成立，论文自己也写了：Lemma 1 / Prop. 4 针对 an idealized form of the policy term（p.17、p.21）。但不是全不覆盖实现：Prop. 3 精确分解了 BAPO 实际用的两个组 baseline，Corollary 3 套用到实际的 TTO 打分（U = 4，p.24），只是 δ_k、ζ 没有实测。

建议

部分承认：理论定位为设计依据（为什么按 belief 分组、为什么监督 forecast），不是性能保证；回复里分清哪些直接对应实现（Prop. 3、Corollary 3），哪些是理想化（Lemma 1、Prop. 4）。措辞降级和 δ_k、ζ 的估计跟 A4 合做。

“omniscient” 与 internal world model

成立W

分析

投稿版摘要原文就是 we aim to build a code world model that is omniscient over both time and environment state（p.1），引言和结论还说 belief 包含 what the next command will change 和 “past, present, and future”（p.2、p.9）。被执行验证的只有两字段的点预测，Figure 7 题注自己称之为 the one coordinate of the next observation that the verifier ψ reads（p.18）。

建议

直接承认：这句投稿版里有，修订版已删（不能暗示从没写过）。p.2/p.9 那两处一起改。world model 精确定义为“对下一观测中 verifier 可读部分的预测”。标题由你定，回复里只承诺正文措辞，和 A3 口径一致。

TTO 打分偏向乐观预测

成立WL

分析

算术完全对。按 Eq. 23（p.32），repository 上 edit 恒为 NA：

检查，预测 PASS + 正常退出

任何 edit（NA）+ 正常退出

检查，正确地预测 FAIL + 正常退出

O.1 / Figure 14 step 7 的赢家正是靠错误的 PASS 得 4 分（p.33、p.40）；如果它预测对了，草拟的 edit 会赢。只有 terminal 上 check 字段是整局的 expect_success，PASS 高于 FAIL 才等于按预测的任务成败排序（Figure 14 右）。

建议

在 repository 上全面承认。改写 O.1：那次检查是因为错误的 PASS 才被选中，不能当 ranking 正确的例子；删掉 Figure 14 里 “the drafted edit would have shipped unverified”（App P rule 5 本来就要求最后一次 edit 后必须验证，p.47）。TTO 拆成 trigger 和 ranking 分别讨论，ranking 有没有用交给 B10 的对照，不好就如实定位为“belief 触发的重采样”。

需要你

核 O.1 原始轨迹：正文说 step 7 和 9 都 abstain，按 Algorithm 3 两步都该触发，Figure 14 却写 “1 of 10 Steps paid for extra samples”（p.33）。

B10

TTO 没有误差棒，缺简单规则对照

成立LE

分析

TTO 行 28.6/48.6/57.8 没有 ±，而且触发步按 temperature 1.0 采样（p.7、p.32）；7B、32B 的增益小于 ANTE 自己的 s.e.。同触发点下忽略 forecast 的规则、winner 的 forecast 正确率，论文都没有。

建议

承认 7B、32B 的增益在噪声内，先改措辞。winner 的 forecast 正确率和每题触发次数可以直接从现有 TTO 轨迹数出来（L）。算力允许：TTO 再跑 2 次给 ±，并在同触发点、K = 3 下比较随机 / 有测试就选测试 / Eq. 23（E），和 A2 一次答两位。结果不论好坏都照实报。

需要你

跑不跑；现有 TTO 日志有没有存每个候选的 belief 和分数。

Reviewer zcGj

Rating 2 · Weak rejectConfidence 3P2 · I2 · S2

TA 要什么：不质疑方法本身（觉得 sensible，Soundness 只是 minor or fixable）。要的是：和最近的方法差在哪、为什么重要、实验里哪里证明了，以及能读清的图和数学。最可能加分：正文加一张诚实标注的 delta 表，讨论期直接上传重画后的图，两个问题照记录答清。

原文 · Official Review by Reviewer zcGj20 Oct 2026, 03:47 (modified 06 Nov 2026, 04:29)

Paper SummaryThe paper presents a reinforcement learning framework for learning internal world models for coding agents. The main innovation is that before each action, the agent outputs a “belief” representing the agent’s thinking of the state of the world, a prediction about what will happen next, and a decision whether to trust in the prediction or abstain. The predictions are benchmarked against the actual outcomes and used to assign credit to different steps, and at inference time, to select that best-forecast action or resample if the decision is to abstain. In the experiments, the method is compared with agents that do not explicitly output their forecasts before action.

Critical StrengthsThe method seems sensible and a good number of out-of-domain datasets is analysed in the experiments, with promising results.

Critical Weaknesses

C1The difference to the closest methods has not been explained clearly, nor demonstrated in the experiments. Appendix A lists related work, and a reader is left with an impression that there are several previous works that do something similar even if not in exactly the same way. What is the delta to the closest methods, why is it significant, and is it possible to demonstrate the benefit in the experiments?

p.5 写 excluding the trusted crash forecasts that score zero above，但正文从没给出这个 0 分的情形（只在附录 Eq. 18）；正文引用的 Eq. 18、Prop. 3、Corollary 3 都只在附录；两页内 y/ŷ/ỹ 和六种 A 符号并存（p.5–6）。

建议

σ 的所有情形在正文写全；正文用到的附录结论就地一句话讲清；减少同时出现的符号。Prop. 1/2 在正文各缩成一句“它支撑哪个设计”（为什么 SFT 之后还要 RL；为什么用执行结果监督 forecast），完整陈述和残差项留附录（3Rua 夸过这部分）。腾出的篇幅给 C1 的 delta 表。

超参怎么定的

待核W

分析

论文答了一部分（p.30–31）：γ 和 q° 由 break-even 公式从 (α, q*, κ) 推出；A sweep selects the prediction weight λ = 10；N = 8 和 clip follow common practice；ω = 1/2 为了稳定（p.6）。没交代的：α = 0.05、q* = 5/9、κ = 0.4α 自己怎么来的，λ sweep 的取值和选择数据，学习率、采样参数、TTO 的 K = 3 和 3/1/0/+1 打分的来历，baseline 是否同等调参，报告的是哪个 checkpoint。p.27 还把 TB-Dev 叫 “development set”，却在 Table 2 当 benchmark 报。

建议

一张表把每个超参归类，各给一句理由：公式推出（γ、q°）/ 沿用前人（N、clip、目标形式）/ 设计约束（std floor 取 α+γ）/ 手设或调出（α、q*、κ、λ、K、TTO 分值）。写明有没有在 SWE-bench Verified 或 terminal 测试集上选过超参或 checkpoint。只能照记录写，没调过就直说“手设，未调”。

需要你

以上各超参的真实来历；λ sweep 试过哪些值、用什么数据选；TB-Dev 和 Val-100 实际做什么用。

为什么跨 rollout、不管位置地按 forecast 分组

正文只一句W

分析

问题合理：正文只说已有做法 not readily applicable（p.5），真正的理由在 Appendix A（p.16：key needs neither recurring states nor aligned positions nor a critic），例子在 Appendix I（p.34）。而且位置没有完全消失：W 是从该步起的 decision reward 后缀和，天然随位置变（p.34），论文用 std floor 压它（p.31）。

建议

正面解释：per-step credit 需要一个 baseline 表示“处在这种情况下的步通常能拿多少”。仓库状态几乎不重复，第 t 步在不同 rollout 里也不是同一个阶段；但“我预测这个 check 会 PASS / FAIL”会反复出现，所以 V̂(ỹ)、Q̂(ỹ, d) 就是 belief 上的 value 和 action value（p.5–6）。位置体现在“比什么”（W 是后缀和），不体现在“跟谁比”。把这段直觉和 Appendix I 的例子搬进正文，如实说没有做按位置分组的 ablation；A1 的 isolation run 可以顺便加一个按位置分组的变体。

Shared work

一次做，答多位

做什么	回答	类型
收窄 claim、删 “omniscient”、重画 Figure 1（连同字号和配色）	A3 B5 B8 C2	W
讲清评测预算；headline 改成对 Retrospective，两个 baseline 并列报	B1 B2 B3	W（预算不一致则 E）
按三类检查拆 hit rate 和占比	B6 A3	L
TTO：同触发点比较 Eq. 23 / 随机 / 有测试就选测试，补 ±，数 winner 正确率	A2 B9 B10	L + E
BAPO isolation run（14B 优先，可加按位置分组的变体）+ w/o BAPO 补 ±	A1 B3 B4 C5	E + T
理论降级为 motivation；小样本估 δ_k 和 ζ	A4 B7 C3	W + L
正文 delta 表 + 超参来历表	C1 C4	W

W = 只改写作 · L = 分析已有日志 · E = 新评测（不训练）· T = 新训练

Version check

投稿版和现在的 Overleaf：rebuttal 要知道的差别

“omniscient”：投稿版摘要有，Overleaf 已删。回复说“修订版已删”，不能说没写过。
Reactive 没有 format 项和 budget warning 的说明（投稿版 p.31–32）：Overleaf 把这两段删了。修订版应该恢复并把各 baseline 的差异列清楚，删掉反而像在掩盖。
Overleaf p.9 的 case study 新写了 ANTE's belief predicts what the next edit will change 和 “force bytes in py2”：正是 B5 批评的说法，而且这句引语在轨迹里找不到，要改。
判分表数字两版不同：投稿版 Table 12 是 Coverage 54.8% / Acc 78.5 / MCC +0.512；Overleaf 是 Covered Acc 73.2 / MCC +0.482。回复只能用一套，要先确认哪套对。
λ sweep：投稿版 p.30 有 “A sweep selects λ = 10”，Overleaf 删了这句，但 D.4 还在引用 sweep，成了悬空引用。
编号：审稿人引用的都是投稿版编号（Eq. 23、Figure 14、Table 12 等）。Overleaf 里 Eq. 23 现在是 Eq. 22，投稿版 Table 7 删了以后后面的表号都减 1。回复里一律用投稿版编号。

Needs your input

需要你确认（不确定的不写进回复）

评测预算：Table 1 每一行的 context 和 step cap；O.2 的 Retrospective 为什么停在第 50 步，Figure 19 的为什么停在第 37 步。B1
误差棒的来源：temperature 0 的 3 次评测之间是什么在变；TTO 行和 w/o BAPO 行各评了几次。A2 B4
TTO 日志：有没有存每个候选的 belief、分数、logprob；O.1 step 9 为什么没算作付费步。B9 B10 A4
判分表：54.8 / 78.5 / +0.512 和 73.2 / +0.482 哪套是对的。
超参：λ sweep 的取值和选择数据；α、q*、κ、K、TTO 分值是手设还是调过；报告哪个 checkpoint；TB-Dev、Val-100 的用途。C4
标题：“Learning Internal World Models” 保留并在正文精确定义，还是改。A3 B8
算力和实验：BAPO isolation（14B）、seed、TTO 对照、w/o BAPO 补评、Retrospective-32B 的 OOD、ECHO 补评，各做哪些。