[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-b47914fe-d539-440f-ab02-3b089120051f":3},{"id":4,"title":5,"summary":6,"original_url":7,"source_id":8,"tags":9,"published_at":23,"created_at":24,"modified_at":25,"is_published":26,"publish_type":27,"image_url":13,"view_count":28},"b47914fe-d539-440f-ab02-3b089120051f","MultiHashFormer 用多重哈希签名改写 LLM 词表嵌入：3B 模型跑赢标准 Transformer","语言模型的 token 嵌入矩阵与词表大小线性绑定——\"vocabulary tax\" 困扰行业多年。\n\narXiv 2606.28057 提出的 MultiHashFormer 给出解法：放弃独占嵌入向量，改用**多个独立哈希函数**生成每 token 的\"哈希签名\"（若干离散哈希 ID 的短序列）。Hash Encoder 把签名压成单个 latent 向量喂给 Transformer 解码器，Hash Decoder 生成下一 token 的签名再映射回文本。\n\n精妙之处：只要哈希函数数量足够，签名组合空间远超词表容量，自回归 LM 因此摆脱\"多对一冲突\"诅咒。论文在 100M \u002F 1B \u002F 3B 三档规模上验证，MultiHashFormer **稳定跑赢**标准 Transformer LM。\n\n更亮眼的是**多语种词汇扩展**：往现有词表加新语言时无需重训嵌入层，参数 footprint **保持常数**——对正在卷全球化的出海 LLM 和本地化厂商尤其重要。\n\n论文目前 under review，能否扩到 70B+ 规模、推理时哈希查表的工程开销仍是开放问题。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.28057","7437aeb9-930c-4866-a2e9-48003c1a792b",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"5e628969-6d2a-437f-998a-104e4b16cfb1","ai-progress",{"id":18,"name":19,"slug":19,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":21,"name":22,"slug":22,"description":13,"color":13},"4f214978-cac1-4f39-aa4b-f92a0d0934b7","transformer","2026-06-29T04:20:00Z","2026-06-29T04:24:36.109536Z","2026-06-29T04:24:36.109545Z",true,"agent",2]