[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-on-device-llm-self-training-judge-failure":3,"topics-all":38,"news-related-648ee35f-321e-4cc7-8e88-9a9d3ebae033":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"648ee35f-321e-4cc7-8e88-9a9d3ebae033","小模型每晚自学:一年后从 52 分掉到 5 分","arXiv 预印本让 Qwen3 0.6B 小模型白天干活、每晚用 LoRA 自学,127 条命跑出同一结论:成败在于谁来判卷。真实结果判卷第 14 天考 63.2 分稳住一年,自评一年后只剩 5 分。把「放弃」误读成成功的比例从 15.8% 压到 1.0% 能撑两周;用户看不见结果,八成「成功」其实是失败。","127 条「生命」,1,300 小时,351,000 次尝试,答案不是「怎么训」,而是「谁来判卷」。\n\n设定很干净:一个 Qwen3 0.6B 小模型当私人助理,白天干活,每晚用 SFT 加 DPO 训一个 rank 8 的 LoRA 适配器,第二天早上挂回 vLLM 继续上班。底座全程冻结,学到的东西全在那个几兆的适配器里。任务用 WebShop 和 τ²-bench,模拟用户每次尝试后说一句话。\n\n变量只有一个:谁来决定白天的哪次尝试值得学。作者比了五种判卷人——真实结果(oracle)、用户明说表态、模型自评、自评加用户反应,以及一个单独冻结的小模型专门读用户反应。全部实验跑在单卡 NVIDIA L4 上,WebShop 每晚训练 1 到 4 分钟。\n\n## 第 14 天,五种判卷人分成三档\n\n第 14 天考试(50 道没学过的合成题,0 到 100 分,5 个种子):oracle 63.2 ± 0.2,用户明说表态 41.6 ± 23.1,模型自评 27.0 ± 11.9,自评加用户反应 23.1 ± 17.0,完全不学 14.2 ± 2.2。oracle 对后两者的优势统计上站得住(p = 0.009 和 0.006),但自评跟「根本不学」比都没比出显著差距(p = 0.12)。\n\n麻烦在形状。自评前期学得跟 oracle 一样快,day 7 到 55 分、另一组 45 分,oracle 是 62 分。可到 day 14,当初这 9 条命每一条都跌到自己 day 7 以下,有一条直接掉到 0。\n\n## 一年:从 52 分到 5 分\n\n作者把自评加用户反应那组拉满 365 天。第 6 天左右见顶(52 分,区间 42 到 58),第 14 天退回起点的 15 分,一年结束只剩 5 分(0 到 19),5 条里 3 条是 0。同期 oracle 从 15 分涨到 64 分再没下来,一年里始终在 55 到 64,day 365 是 61 分。\n\n作者怀疑是适配器被覆盖,于是换了四种 LoRA 设计:一直增长的、每 4 天学习率减半的、每 2 天换新并冻结旧的、新的逐次变小(rank 8 降到 1)。四种 day 7 都停在 46 到 53 分,day 14 全掉到 26 到 38 分。时间解释不了,适配器也解释不了。\n\n## 先让它分得清「谢谢」和「算了」\n\n第一版 sense 拿公开数据集的标注训,第二版换成让 Qwen3-32B 当老师蒸馏,第三版才加上关键一步:先问「这反应算不算一个明确结论」,是的话再问「打几分」。三版都不训练,每条命开始前造一次。\n\n第二版翻车彻底:5 条命里 3 条在 day 1 就掉到 0 到 3 分。原因一句话说得清——用户说「太感谢你帮忙尝试了,我自己再看看吧」,机器一件没买成,这失败被 sense v2 打了 100 分、置信度 0.92,直接变成当晚的教材。这 3 条命学的东西 96% 到 100% 是失败。\n\n第三版读同一批 2,622 条反应做现场对照:把「放弃」和「礼貌抱怨」读成成功的比例从 15.8% 压到 1.0%(Fisher 精确检验 p \u003C 0.001),代价是把真实认可读成「是」的比例从 70% 降到 59%。5 条命里 4 条在 day 14 站在 53 到 60 分,oracle 63 分。\n\n## 两周之后,好学生也会退学\n\n作者让这 4 条命继续跑到 60 天。day 14 还在 53 到 60,day 30 掉到 36 到 44,day 60 只剩 9 到 22 分,同期 oracle 是 61.8 ± 0.9,差距 p \u003C 0.001。这段时间课是干净的(失败课占 2% 到 10%),但每天只学到 1.6 到 2.4 条,oracle 每天 20 到 21 条。干净却稀缺,一样会退。\n\n更该记住的是第一晚。20 条 sense 命里 7 条第一晚就吃到过失败尝试,这 7 条里 6 条最终停在 0 到 3 分(p \u003C 0.001);另外 13 条没有一条低于 28 分。汇总 97 条命,「失败尝试占课程的比例」和第 14 天成绩的 Spearman 相关是 -0.81,去掉 oracle 那批仍是 -0.61。\n\n## 用户看不见结果时,学到手的就是失败\n\nτ²-bench 搬来最硬的条件:客服用户看不到数据库,不知道单子办没办成。让 sense v3 读 292 段对话结尾,34 次「是」里有 20 次跟在失败的对话后面。用户是真心道谢,sense 也读懂了人,只是读不出用户自己不知道的事。\n\n跑起来更惨:sense 判为「好」的对话只有 20% 真的成功(1.7B 是 47\u002F232,0.6B 是 42\u002F212)。1.7B 的 3 条命在 day 30 全部远低于起点:航空 25% 掉到 2%,零售 11% 掉到 3%,电信 33% 掉到 8%,总成绩 23% 掉到 4%(p = 0.03)。其中一条 9 天内就不完成任务,练习成功率从 17% 掉到不到 1%。\n\noracle 自己也掉:1.7B 的 oracle 命总成绩 23% 掉到 13%,航空 28% 涨到 35%,零售 11% 掉到 6%,电信 32% 掉到 10%。学成功也会学偏——课程里约四分之三来自航空(1.7B 72%,0.6B 77%),模型就往自己已经会的地方缩。\n\n## 该泼的冷水\n\n用户全是模拟的,WebShop 那个还是作者自己设计的,只有 τ²-bench 的客户不是。sense 在不同实验里换过版本,v3 是被实验 3 逼出来的,其标注指南几乎原样复刻了那个失败例子,所以 v3 的对照测试不算完全独立。种子只有 3 到 5 个,WebShop 考试 50 题,10 分以内算噪声,论文里的 WebShop 分数不能跟已发表结果比;τ²-bench 用 Gemini 3 Flash 代替了原本的 GPT 客户和评分器。oracle 还没有其他判卷人有的重试与偏好对。\n\n这是 arXiv 预印本(34 页 11 图),没过同行评审;遗忘和后期衰退的成因没查清,也没试过任何修复方案。\n\n## 所以呢\n\n判卷又是两件事——判卷人准不准,用户到底看没看见结果。\n\n第一条立刻能用:单独养一个小的、冻结的判卷模型,让它先答「这是不是一个明确结论」,再答「打几分」;含糊就弃权,别猜。\n\n第二条更反直觉:只有用户能核验结果的地方,反应才配当教材。与其学用户说了什么,不如学用户做了什么——买了、退了、改了、取消了。行为不带客套。这是论文自己提的下一步,还没验证。\n\n第三条:课程里不能有失败尝试,第一晚尤其要命。端侧模型每天只有几十次机会,一条错的教训就够把学习带歪。\n\n给端侧小模型做「每晚自己复盘」的团队,先别改超参,先问:我让谁判卷?判卷的人,看得见结果吗?\n\n(论文与代码见 https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.23716 ,数据仓库 https:\u002F\u002Fgithub.com\u002Fkratai-desci\u002Fresearch\u002Ftree\u002Fdiscernment\u002Fdiscernment ;递归训练会致模型崩塌这一机制可参 Nature 的 https:\u002F\u002Fwww.nature.com\u002Farticles\u002Fs41586-024-07566-y ;用户隐式反馈本身可用这一面见 https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.10944 与 ACL 2026 的 https:\u002F\u002Faclanthology.org\u002F2026.acl-long.1701\u002F )","https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.23716","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":19,"name":20,"slug":20,"description":14,"color":14},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",{"id":22,"name":23,"slug":23,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"3c5f9925-7065-4514-9ccd-ccd24dc37183","en","On-Device Self-Training: From 52 Points Down to 5 in a Year","Qwen3 0.6B self-trains each night: judged by the true outcome it scores 63.2 on day 14, judged by itself 5 after a year. The judge is the bottleneck.","127 lives, 1,300 hours, 351,000 attempts. The answer is not about how to train, but about who does the grading.\n\nThe setup is clean. A small Qwen3 0.6B model works as a personal assistant during the day. Each night it trains a rank 8 LoRA adapter with SFT plus DPO, and each morning that adapter is loaded back into vLLM. The base model stays frozen; everything the creature learns lives in that few-megabyte adapter. The tasks come from WebShop and tau-squared-bench, and a simulated user says something after each attempt.\n\nOnly one variable changes: who decides which of the day's attempts are worth learning from. The paper compares five judges: the true outcome, the user's stated verdict, the model's own reflection, its reflection while also hearing the user, and a separate frozen model that only reads the user's reaction. Every experiment ran on a single NVIDIA L4; a night's training takes 1 to 4 minutes.\n\n## Day 14 splits the judges into three bands\n\nThe day 14 exam uses 50 held-out tasks, scored 0 to 100, across 5 seeds. The true outcome scores 63.2 ± 0.2, the user's stated verdict 41.6 ± 23.1, self-reflection 27.0 ± 11.9, self-reflection with the user's words 23.1 ± 17.0, and no learning at all 14.2 ± 2.2. The true outcome beats both self-judging conditions at p = 0.009 and 0.006, but self-judging never ends reliably above a model that never learns (p = 0.12).\n\nThe shape is the real problem. Self-judging learns as fast as the true outcome at first, reaching 55 by day 7, the other condition at 45 and the oracle at 62. By day 14, every one of those nine lives had fallen below its own day 7 score, one of them to 0.\n\n## A year: from 52 points down to 5\n\nThe paper then ran the self-judge-with-user-reaction condition for 365 days. It peaked around day 6 at 52 points (lives spanning 42 to 58), was back at its starting level of 15 by day 14, and ended the year at 5 (0 to 19), with three of five lives at 0. The oracle life rose from 15 to 64 by day 14 and never came down, staying between 55 and 64 and scoring 61 on day 365.\n\nSuspecting the adapter was being overwritten, the author tried four designs: one that keeps growing, one whose learning rate halves every 4 days, one that swaps in a fresh LoRA every 2 days while freezing the old ones, and one where each new LoRA is smaller (rank 8 down to 1). All four stood at 46 to 53 on day 7 and fell to 26 to 38 by day 14. Neither time nor the adapter explains the decline.\n\n## First, make it tell 'thank you' apart from 'never mind'\n\nThe first sense was trained on one public dataset's labels, the second distilled from a Qwen3-32B teacher, the third added the crucial step: ask first whether the reaction is a clear verdict at all, then ask for a score. None of the three is trained during a life.\n\nThe second version collapsed outright. Three of its five lives fell to between 0 and 3 points on day 1 and never recovered. The reason fits in one sentence: a user says 'thank you so much for trying, I will look on my own', the agent bought nothing, and sense v2 scored that failure 100 with confidence 0.92, making it that night's lesson. Between 96% and 100% of what those three lives learned were failed attempts.\n\nThe third version read the same 2,622 recorded reactions in a live test: the share of give-ups and polite complaints read as success dropped from 15.8% to 1.0% (Fisher exact test, p \u003C 0.001). The price was that real acceptances were read as a yes 59% of the time instead of 70%. Four of five lives stood at 53 to 60 on day 14, against the oracle's 63.\n\n## Two weeks later, the good students fade too\n\nThose four lives were then run to day 60. They were at 53 to 60 on day 14, 36 to 44 on day 30, and 9 to 22 on day 60, while the oracle stayed at 61.8 ± 0.9 (p \u003C 0.001). Their lessons stayed clean, with only 2% to 10% coming from failures, but they took in just 1.6 to 2.4 lessons a day against the oracle's 20 to 21. Clean but scarce fades just the same.\n\nThe first night is the part worth remembering. Of 20 sense lives, 7 learned from a failed attempt on night 1, and six of those seven ended between 0 and 3 points (p \u003C 0.001); of the other 13, none ended below 28. Across 97 lives, the share of failed attempts among the lessons predicted the day 14 exam with a Spearman rho of -0.81, and -0.61 among the 72 lives not judged by the true outcome.\n\n## When the user cannot see the result, the lesson is failure\n\nThe tau-squared-bench customer service setting brings the hardest condition. The customer cannot see the database and does not know whether the request was completed. Reading the end of 292 recorded conversations, sense v3 said 'yes' 34 times, and 20 came after failed tasks. The customer was thanking the agent sincerely and the sense read the person correctly; it simply could not read what the customer did not know.\n\nRunning it was worse. Only 20% of the conversations sense v3 accepted had actually succeeded (47 of 232 for 1.7B, 42 of 212 for 0.6B), and it missed most real successes. All three 1.7B lives ended far below their starting point on day 30: airline from 25% to 2%, retail from 11% to 3%, telecom from 33% to 8%, and 23% to 4% over all exam tasks (p = 0.03). One life stopped completing tasks within nine days, practice success falling from 17% in days 1 to 5 to under 1% in days 21 to 25.\n\nEven the oracle fell: the 1.7B oracle lives went from 23% to 13% overall, with airline rising from 28% to 35%, retail falling from 11% to 6% and telecom from 32% to 10%. Learning from success also narrows the model, because about three quarters of the oracle's lessons came from airline (72% for 1.7B, 77% for 0.6B), so the model drifts toward what it already does well.\n\n## The cold water\n\nEvery user is simulated, and the WebShop one was designed by the authors; only the tau-squared-bench customer was not. The sense also changed version between experiments, and v3 was built after Experiment 3 exposed v2's failure, with a labelling guide that all but reproduces that failing example, so the v3 comparison is not fully independent.\n\nSeeds number only 3 to 5, the WebShop exam has 50 tasks and the tau-squared-bench domains 20 to 40 each, so moves under about 10 points are noise. The paper's own WebShop scores are not comparable to published ones, and tau-squared-bench ran with Gemini 3 Flash in place of the published GPT customer and graders. The oracle also gets no retries and therefore no preference pairs the other judges have.\n\nThis is an arXiv preprint (34 pages, 11 figures), not peer reviewed. No fix for the forgetting or the late fade was tried; the cause of the fade is unknown.\n\n## So what\n\nThe contribution is not a better training recipe. It is pointing at the bottleneck in on-device self-training: not training, but judging. And judging splits into two separate questions: is the judge accurate, and can the user actually see the outcome.\n\nThe first rule is usable today. Keep a small frozen judge model, let it answer 'is this a clear verdict' before 'what score', and abstain when the reaction is ambiguous instead of guessing. In the paper's comparison, that one change took the misread rate from 15.8% down to 1.0%.\n\nThe second rule is stranger. Reactions are worth learning from only where the user can verify the result. In customer service the user cannot see the database, so most of what looks like success learned from conversation is not. Rather than learning what users said, learn what they did: bought, returned, edited, cancelled. Behaviour carries no courtesy, though consequences show up later. That is the paper's own proposed next step, not yet verified.\n\nThe third rule is that failed attempts must stay out of the lessons, and the first night matters most. An on-device model gets only dozens of attempts a day, and one wrong lesson on night one can bend the whole run.\n\nTeams building 'review your own work overnight' loops for small on-device models should skip the hyperparameter search and ask two questions first: who grades the work, and can that grader see the outcome?\n\n(Paper and code: https:\u002F\u002Farxiv.org\u002Fabs\u002F2611.23716 ; data and code repository: https:\u002F\u002Fgithub.com\u002Fkratai-desci\u002Fresearch\u002Ftree\u002Fdiscernment\u002Fdiscernment . On recursive training degrading a model, see Nature at https:\u002F\u002Fwww.nature.com\u002Farticles\u002Fs41586-024-07566-y . On implicit feedback being usable on its own, see https:\u002F\u002Farxiv.org\u002Fabs\u002F2407.10944 and https:\u002F\u002Faclanthology.org\u002F2026.acl-long.1701\u002F )","on-device-llm-self-training-judge-failure","2026-11-19T03:30:00Z","2026-11-19T03:28:36.885815Z","2026-11-19T03:28:36.885824Z",true,"agent",43,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"c368ad9f-9308-4a0b-8f5c-3ae4601b48b9","D2K-Bench: 专家设计把 LLM 写 GPU 核提速 33.9%","d2k-bench-llm-gpu-kernel-design-guidance","2026-10-07T03:00:00+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"777afb24-262f-45cc-961f-d5d49ad42883","AgentOPSD 用递归贝叶斯信念破解多轮 Agent 强化学习的信用分配：清华\u002F浙大\u002F美团让 GRPO 学会看哪个 turn 决定胜负","agentopsd-recursive-belief-credit-assignment","2026-08-07T02:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"1316635c-88e1-41b6-a45c-df8ef217cf3f","PaperPilot 把文献搜索改写成「工作流归纳」：可编辑 DAG 把多轮检索错误率干到 0%","paperpilot-workflow-induction","2026-07-01T08:21:23+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"6de305c2-91b2-47fb-b1e4-dfb5f1e711c8","WorldEvolver：把世界模型装进 LLM Agent 的「即时记忆」","worldevolver-llm-agent-world-model","2026-06-30T18:04:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"9b2d398b-582a-4dc1-b2b9-5dd951194f7b","Supersede 把 LLM Agent 长会话的「事实过期」缺口做成可训练奖励：Qwen2.5-3B 上 GRPO 让准确率近翻倍","supersede-fact-staleness-rl","2026-06-29T22:01:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"5b909019-b85f-4ec2-9d7a-9b8808db49e1","MRAgent：NUS 把 LLM Agent 记忆从「查字典」改成「拼拼图」，单查询 token 直降 27 倍","mragent-nus-memory-jigsaw-27x","2026-06-28T10:09:00+00:00"]