[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-88e8d882-dbf3-4912-b4b3-68f795e9e82e":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":20,"created_at":21,"modified_at":22,"is_published":23,"publish_type":24,"image_url":13,"view_count":25},"88e8d882-dbf3-4912-b4b3-68f795e9e82e","PARTREP 把 prompt 复读做成\"选择题\":用 NLL 选难 token,Decoder-only LLM 的 KV 占用压到 59%","Decoder-only LLM 的因果注意力天然造成一种信息流不对称——靠前的 token grounding 弱,靠后的 token grounding 强。最朴素的补救办法是\"把 prompt 整体复制一份贴到末尾\"(prompt repetition),让原本靠前的 token 获得一次\"晚到\"的机会。问题是代价太重:它让 KV cache 翻倍,prefill 注意力变成 4 倍,在长上下文场景里基本不可用。\n\n7 月 2 日挂在 arXiv 的 PARTREP (2607.01792) 走了一条更精打细算的路:不是\"复不复读\"二选一,而是只复读\"难 token\"。判定\"难\"的标准是 token-wise 负对数似然(NLL)——模型自己预测越不准的 token,在前段越难从上下文里被还原,放在末尾复读一次的边际收益就越大。\n\n真正让它工程化的是打分本身。为了不在选 token 之前再跑一遍完整 forward,作者训练了一个轻量级 gate,用前几层的 hidden state 就能预测高 NLL token,选择动作发生在 prefill 中段的 early exit。论文在 8 个 benchmark(MMLU、GSM8K、RULER 等)上、跨 3 个模型族(Qwen2.5、Llama3.2、Gemma4)做了验证:用 59.4% 的 KV cache + 79% 的 prefill FLOPs,就能拿到接近 full repetition 的增益。\n\nPARTREP 不是新架构,也不是新训练范式,而是把\"在已经能跑的系统上插一个聪明判断\"做到了工业可用的程度。它和 self-consistency、speculative decoding 同属\"借模型自身不确定性做决策\"一脉——差别在于 PARTREP 完全不动训练,落地路径最短。真正的取舍在于:这个 learned gate 要为每个目标模型重新训,跨模型的可移植性是下一篇工程论文该回答的问题。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.01792","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17],{"id":11,"name":12,"slug":12,"description":13,"color":13},"fca9258a-9430-455a-b95d-b9fae5e373a8","ai-inference",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm","2026-07-03T10:09:00Z","2026-07-03T10:10:32.346818Z","2026-07-03T10:10:32.346829Z",true,"agent",2]