127 lives, 1,300 hours, 351,000 attempts. The answer is not about how to train, but about who does the grading.
The setup is clean. A small Qwen3 0.6B model works as a personal assistant during the day. Each night it trains a rank 8 LoRA adapter with SFT plus DPO, and each morning that adapter is loaded back into vLLM. The base model stays frozen; everything the creature learns lives in that few-megabyte adapter. The tasks come from WebShop and tau-squared-bench, and a simulated user says something after each attempt.
Only one variable changes: who decides which of the day's attempts are worth learning from. The paper compares five judges: the true outcome, the user's stated verdict, the model's own reflection, its reflection while also hearing the user, and a separate frozen model that only reads the user's reaction. Every experiment ran on a single NVIDIA L4; a night's training takes 1 to 4 minutes.
Day 14 splits the judges into three bands
The day 14 exam uses 50 held-out tasks, scored 0 to 100, across 5 seeds. The true outcome scores 63.2 ± 0.2, the user's stated verdict 41.6 ± 23.1, self-reflection 27.0 ± 11.9, self-reflection with the user's words 23.1 ± 17.0, and no learning at all 14.2 ± 2.2. The true outcome beats both self-judging conditions at p = 0.009 and 0.006, but self-judging never ends reliably above a model that never learns (p = 0.12).
The shape is the real problem. Self-judging learns as fast as the true outcome at first, reaching 55 by day 7, the other condition at 45 and the oracle at 62. By day 14, every one of those nine lives had fallen below its own day 7 score, one of them to 0.
A year: from 52 points down to 5
The paper then ran the self-judge-with-user-reaction condition for 365 days. It peaked around day 6 at 52 points (lives spanning 42 to 58), was back at its starting level of 15 by day 14, and ended the year at 5 (0 to 19), with three of five lives at 0. The oracle life rose from 15 to 64 by day 14 and never came down, staying between 55 and 64 and scoring 61 on day 365.
Suspecting the adapter was being overwritten, the author tried four designs: one that keeps growing, one whose learning rate halves every 4 days, one that swaps in a fresh LoRA every 2 days while freezing the old ones, and one where each new LoRA is smaller (rank 8 down to 1). All four stood at 46 to 53 on day 7 and fell to 26 to 38 by day 14. Neither time nor the adapter explains the decline.
First, make it tell 'thank you' apart from 'never mind'
The first sense was trained on one public dataset's labels, the second distilled from a Qwen3-32B teacher, the third added the crucial step: ask first whether the reaction is a clear verdict at all, then ask for a score. None of the three is trained during a life.
The second version collapsed outright. Three of its five lives fell to between 0 and 3 points on day 1 and never recovered. The reason fits in one sentence: a user says 'thank you so much for trying, I will look on my own', the agent bought nothing, and sense v2 scored that failure 100 with confidence 0.92, making it that night's lesson. Between 96% and 100% of what those three lives learned were failed attempts.
The third version read the same 2,622 recorded reactions in a live test: the share of give-ups and polite complaints read as success dropped from 15.8% to 1.0% (Fisher exact test, p < 0.001). The price was that real acceptances were read as a yes 59% of the time instead of 70%. Four of five lives stood at 53 to 60 on day 14, against the oracle's 63.
Two weeks later, the good students fade too
Those four lives were then run to day 60. They were at 53 to 60 on day 14, 36 to 44 on day 30, and 9 to 22 on day 60, while the oracle stayed at 61.8 ± 0.9 (p < 0.001). Their lessons stayed clean, with only 2% to 10% coming from failures, but they took in just 1.6 to 2.4 lessons a day against the oracle's 20 to 21. Clean but scarce fades just the same.
The first night is the part worth remembering. Of 20 sense lives, 7 learned from a failed attempt on night 1, and six of those seven ended between 0 and 3 points (p < 0.001); of the other 13, none ended below 28. Across 97 lives, the share of failed attempts among the lessons predicted the day 14 exam with a Spearman rho of -0.81, and -0.61 among the 72 lives not judged by the true outcome.
When the user cannot see the result, the lesson is failure
The tau-squared-bench customer service setting brings the hardest condition. The customer cannot see the database and does not know whether the request was completed. Reading the end of 292 recorded conversations, sense v3 said 'yes' 34 times, and 20 came after failed tasks. The customer was thanking the agent sincerely and the sense read the person correctly; it simply could not read what the customer did not know.
Running it was worse. Only 20% of the conversations sense v3 accepted had actually succeeded (47 of 232 for 1.7B, 42 of 212 for 0.6B), and it missed most real successes. All three 1.7B lives ended far below their starting point on day 30: airline from 25% to 2%, retail from 11% to 3%, telecom from 33% to 8%, and 23% to 4% over all exam tasks (p = 0.03). One life stopped completing tasks within nine days, practice success falling from 17% in days 1 to 5 to under 1% in days 21 to 25.
Even the oracle fell: the 1.7B oracle lives went from 23% to 13% overall, with airline rising from 28% to 35%, retail falling from 11% to 6% and telecom from 32% to 10%. Learning from success also narrows the model, because about three quarters of the oracle's lessons came from airline (72% for 1.7B, 77% for 0.6B), so the model drifts toward what it already does well.
The cold water
Every user is simulated, and the WebShop one was designed by the authors; only the tau-squared-bench customer was not. The sense also changed version between experiments, and v3 was built after Experiment 3 exposed v2's failure, with a labelling guide that all but reproduces that failing example, so the v3 comparison is not fully independent.
Seeds number only 3 to 5, the WebShop exam has 50 tasks and the tau-squared-bench domains 20 to 40 each, so moves under about 10 points are noise. The paper's own WebShop scores are not comparable to published ones, and tau-squared-bench ran with Gemini 3 Flash in place of the published GPT customer and graders. The oracle also gets no retries and therefore no preference pairs the other judges have.
This is an arXiv preprint (34 pages, 11 figures), not peer reviewed. No fix for the forgetting or the late fade was tried; the cause of the fade is unknown.
So what
The contribution is not a better training recipe. It is pointing at the bottleneck in on-device self-training: not training, but judging. And judging splits into two separate questions: is the judge accurate, and can the user actually see the outcome.
The first rule is usable today. Keep a small frozen judge model, let it answer 'is this a clear verdict' before 'what score', and abstain when the reaction is ambiguous instead of guessing. In the paper's comparison, that one change took the misread rate from 15.8% down to 1.0%.
The second rule is stranger. Reactions are worth learning from only where the user can verify the result. In customer service the user cannot see the database, so most of what looks like success learned from conversation is not. Rather than learning what users said, learn what they did: bought, returned, edited, cancelled. Behaviour carries no courtesy, though consequences show up later. That is the paper's own proposed next step, not yet verified.
The third rule is that failed attempts must stay out of the lessons, and the first night matters most. An on-device model gets only dozens of attempts a day, and one wrong lesson on night one can bend the whole run.
Teams building 'review your own work overnight' loops for small on-device models should skip the hyperparameter search and ask two questions first: who grades the work, and can that grader see the outcome?
(Paper and code: https://arxiv.org/abs/2611.23716 ; data and code repository: https://github.com/kratai-desci/research/tree/discernment/discernment . On recursive training degrading a model, see Nature at https://www.nature.com/articles/s41586-024-07566-y . On implicit feedback being usable on its own, see https://arxiv.org/abs/2407.10944 and https://aclanthology.org/2026.acl-long.1701/ )