[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-siren-rope-learnable-rotation-positional-encoding":3,"news-related-f092a4b2-4e8b-45c7-9045-570047b8f92c":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"f092a4b2-4e8b-45c7-9045-570047b8f92c","SIREN-RoPE：让位置编码学会「旋转」","当行业在追逐更大的参数规模时，4月27日 arXiv 上的一篇新论文将 RoPE 的旋转流形变成了主角。\n\n**被忽视的维度**\n\nRoPE 自提出以来，其旋转流形一直被视为固定手工结构，填充的只是离散序号索引。token embedding 编码词元「是什么」，而词元之间的时间、位置、上下文关系却从未被系统挖掘。\n\n这篇论文的核心洞察是：类比复数引入虚数轴的正交维度，将旋转流形视为可学习、信号条件化的空间，可在注意力机制中开辟一个正交的全新表达维度。\n\n**SIREN-RoPE：双分支旋转注入**\n\n论文提出 SIREN-RoPE，通过双分支正弦表示网络，将连续时间戳、周期模式、分类元数据注入旋转维度。在某主流社交网络推荐系统上的生产评估显示，激活这一隐藏维度后校准和排序指标均获一致提升，计算开销几乎为零。\n\n**启示**\n\nRoPE 旋转空间一直是 Transformer 中「已有定论」的细节。论文证明它实际上是一座未被开采的金矿——不仅有理论价值，更已在真实产品场景被验证。这为大模型研究者指明新方向：除了堆叠层数，还可通过旋转空间信号注入来增强模型的关系推理能力。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.24717","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"40269b40-7942-4650-9672-ed2e6524d37a","ai-technology",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":13},"0d095a03-5142-4dce-bceb-a5833b9f10c9","en","SIREN-RoPE: Let Positional Encoding Learn to \"Rotate\"","While the industry chases larger parameter scales, a new paper on arXiv on April 27 puts RoPE's rotation manifold in the spotlight.\n\n**An overlooked dimension**\n\nSince RoPE was proposed, its rotation manifold has been treated as a fixed hand-crafted structure, filled only with discrete serial number indices. Token embedding encodes \"what\" a token is, but the temporal, positional, and contextual relationships between tokens have never been systematically mined.\n\nThe core insight of this paper: analogizing complex numbers' introduction of the imaginary axis's orthogonal dimension, treating the rotation manifold as a learnable, signal-conditioned space, opening up an orthogonal new expression dimension in the attention mechanism.\n\n**SIREN-RoPE: dual-branch rotation injection**\n\nThe paper proposes SIREN-RoPE, injecting continuous timestamps, periodic patterns, and categorical metadata into the rotation dimension via dual-branch sinusoidal representation networks. Production evaluation on a major social-network recommendation system shows activating this hidden dimension consistently improves both calibration and ranking metrics, with near-zero compute overhead.\n\n**Implications**\n\nThe RoPE rotation space has long been treated as a \"settled\" detail in Transformers. The paper proves it's actually an untapped gold mine — not only with theoretical value, but already verified in real product scenarios. This points a new direction for LLM researchers: beyond stacking layers, you can also enhance models' relational reasoning capability through rotation-space signal injection.","siren-rope-learnable-rotation-positional-encoding","2026-04-29T01:10:00Z","2026-04-29T01:11:23.686006Z","2026-08-19T02:08:40.142862Z",true,"agent",119,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"217f417d-1b9c-475b-99f4-e21e7c909711","MHAR 把 Transformer 残差流从「单车道」拆成 H 条独立路由:子空间第一次有权自己挑历史层","multi-head-attention-residuals-mhar","2026-08-01T07:30:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"4d92e0b1-04a3-4524-9ae3-b8456aa74f2a","NAVER 提出 On-Policy Delta Distillation:用「差分信号」重新定义推理蒸馏","naver-on-policy-delta-distillation","2026-07-18T16:07:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"a457f7b9-dde3-4d00-bbc0-cdf9ef2dde14","xHC：Transformer 残差流扩成 16 车道，突破 N=4","xhc-expanded-hyper-connections","2026-07-18T00:15:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"0a3f5044-ed20-4c2a-b710-bd26cd276d3e","ALiBi 的隐藏数值故障：长上下文越长，部分注意力头越可能“失明”","alibi-attention-underflow-long-context","2026-08-06T10:30:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"e3f049f5-2f0d-48d2-8e88-246ef006fa16","LoopMTP 给循环 Transformer 装上前瞻路标：固定参数下让每一轮都做不同的事","loopmtp-latent-multi-token-loop-guidance","2026-08-04T13:13:09+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"cb7fb8b3-5862-4cba-adab-c4794e989966","图灵奖得主 Pearl 长访谈：LLM 能讲因果只是因为人类替它爬过了因果阶梯","judah-pearl-llm-causal-ladder-agi","2026-07-31T07:00:00+00:00"]