[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-bitnet-1-58bit-embedding":3,"news-related-c197245e-0a8a-4028-9ad2-6547bdc01be5":36},{"id":4,"title":5,"summary":6,"content":6,"original_url":7,"source_id":8,"tags":9,"translations":23,"news_slug":29,"published_at":30,"created_at":31,"modified_at":32,"is_published":33,"publish_type":34,"image_url":13,"view_count":35},"c197245e-0a8a-4028-9ad2-6547bdc01be5","BitNet 团队把 1.58-bit 量化推进到 embedding:检索向量也可以\"训练时就压\"","Microsoft BitNet 团队本周在 Hugging Face 开源 BitNet Embeddings 系列，主打\"BITEMBED\"框架——这是首次把训练阶段 1.58-bit 量化从 LLM 主干延伸到文本 embedding 任务，覆盖 Qwen3-0.6B 和 Gemma3-270M 两个 backbone，权重三元化 {-1, 0, +1}、激活 8-bit，配合 SubLN 与 RoPE，配套上下文 32K、MIT 许可。\n\n技术路径分两步：**第一步**先把 pretrained LLM 直接转成 BitNet-style embedding encoder，保留 absmean 量化、abs-max-per-token 激活量化；**第二步**在 representation learning 上重新对齐——先在 1B 文本对上做持续对比预训练（InfoNCE），再用监督对比微调从 FP16 教师蒸馏，包含 similarity-distribution 和 attention-relation 两路蒸馏信号，让三元权重下的表征能力往 FP16 教师看齐。\n\n数字很硬：在 MMTEB（eng, v2）上 BitNet-Embedding-0.6B 拿到 **67.60** 平均分，仅比 FP16 教师（67.95）低 **0.35**，分类、检索、摘要等 8 项子任务几乎打平；推理端 CPU 8 线程 **pp128 输入 870.90 t\u002Fs**，相对 FP16 **2.28× 加速**，长输入仍能维持 1.4–1.8× 的领先；输出 embedding 还能再量化到 1\u002F2\u002F4\u002F8 bit，存储随场景弹性伸缩。\n\n这一点对 RAG 和企业知识库意义不小：embedding 阶段是 retrieval 系统的\"CPU 端常驻瓶颈\"，把 backbone 训练时就压到 1.58-bit，意味着整条 retrieval pipeline 第一次可以在端到端 1-bit 链路跑通，bitnet.cpp 推理框架继续复用。过去 BitNet 的故事只写到 LLM 本身，这次补上了检索向量的最后一环——开源工程构件就位，剩下的就是把它接进 LangChain \u002F LlamaIndex 等主流 orchestration 层，社区生态可能很快把这套配置做成默认 RAG 选项。\n\n更值得注意的是 BitNet 这条线的\"渐进扩张\"策略：从 1-bit LLM 推理框架，到今天 1.58-bit embedding，每一步都没有急着从 FP16 切整数精度，而是 **训练时就让模型学会在 ternary 表达里收敛**——这条路线对端侧 AI 的可复制性可能比想象中更大。","https:\u002F\u002Fhuggingface.co\u002Fmicrosoft\u002Fbitnet-embedding-0.6b","24d5c6c5-6573-4180-a1fd-f1459842d1af",[10,14,17,20],{"id":11,"name":12,"slug":12,"description":13,"color":13},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":15,"name":16,"slug":16,"description":13,"color":13},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":18,"name":19,"slug":19,"description":13,"color":13},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":21,"name":22,"slug":22,"description":13,"color":13},"b49648f9-963e-4082-8684-3d085b7358fe","quantization",[24],{"id":25,"lang":26,"title":27,"summary":28,"content":28},"2f07cd7a-d21b-436e-868b-86d3767bb8cd","en","BitNet brings 1.58-bit quantization to embeddings","Microsoft's BitNet team open-sourced the BitNet Embeddings series on Hugging Face this week, focused on the \"BITEMBED\" framework — for the first time extending training-time 1.58-bit quantization from the LLM backbone to the text-embedding task, covering Qwen3-0.6B and Gemma3-270M backbones, with weights ternary-quantized to {-1, 0, +1}, activations 8-bit, paired with SubLN and RoPE, 32K context, MIT license. The technical path is two steps: **step 1** first converts a pretrained LLM directly into a BitNet-style embedding encoder, preserving absmean quantization and abs-max-per-token activation quantization; **step 2** realigns on representation learning — first does continued contrastive pretraining (InfoNCE) on 1B text pairs, then uses supervised contrastive fine-tuning to distill from an FP16 teacher, with two distillation signals: similarity-distribution and attention-relation, letting the representation capability under ternary weights converge toward the FP16 teacher. Numbers are tough: on MMTEB (eng, v2) BitNet-Embedding-0.6B gets an **67.60** average score, only **0.35** below the FP16 teacher (67.95), with 8 sub-tasks (classification, retrieval, summarization, etc.) almost tied; on the inference side, CPU 8 threads **pp128 input 870.90 t\u002Fs**, a **2.28× speedup** over FP16, long input still maintains a 1.4–1.8× lead; the output embedding can also be re-quantized to 1\u002F2\u002F4\u002F8 bit, with storage scaling elastically by scenario. This is no small matter for RAG and enterprise knowledge bases: the embedding stage is the \"CPU-resident bottleneck\" of retrieval systems, and pushing the backbone to 1.58-bit during training means the entire retrieval pipeline can, for the first time, run end-to-end on a 1-bit link, and the bitnet.cpp inference framework continues to be reused. The past BitNet story only ran to the LLM itself, this time it completes the last mile of retrieval vectors — open-source engineering components are in place, what remains is plugging it into mainstream orchestration layers like LangChain \u002F LlamaIndex, and the community ecosystem may quickly make this configuration the default RAG option. Worth noting even more is BitNet's \"gradual expansion\" strategy: from 1-bit LLM inference framework, to today's 1.58-bit embedding, each step hasn't been eager to switch from FP16 to integer precision, but rather **making the model learn to converge in ternary representations during training** — this path may be more reproducible on edge AI than imagined.","bitnet-1-58bit-embedding","2026-07-18T03:00:00Z","2026-07-18T04:04:20.158946Z","2026-08-19T02:08:40.142862Z",true,"agent",95,{"items":37},[38,43,48,53,58,63],{"id":39,"title":40,"news_slug":41,"published_at":42},"48e1c261-a40a-4c71-9cba-450a459e6ad3","4-bit 模型反超全精度:QAH 把量化从性能税变成第二次蒸馏","quantization-aware-healing-hypernova-60b","2026-08-25T17:20:00+00:00",{"id":44,"title":45,"news_slug":46,"published_at":47},"b2d34fba-93ee-469b-9f64-6a7568f89745","Liquid AI 用量化感知蒸馏,把 LFM2.5 4-bit 精度拉回 97%","lfm25-qad-quantization-aware-distillation-edge","2026-08-20T11:00:00+00:00",{"id":49,"title":50,"news_slug":51,"published_at":52},"93981299-c75e-4140-ad2c-ae68fb5d27ce","CAT-Q：512 样本把 235B LLM 压到 1.58-bit，成本降 10 万倍","cat-q-1-58-bit-512-samples-icml-oral","2026-06-26T20:25:00+00:00",{"id":54,"title":55,"news_slug":56,"published_at":57},"cac485ab-a429-4ebc-88c6-a1f924f978ff","AWS 开源 KeysAndValues:微调时就让模型学会“遗忘”,单张 A100 撑住 128K","aws-keysvalues-sparse-attention-finetuning","2026-08-26T05:20:00+00:00",{"id":59,"title":60,"news_slug":61,"published_at":62},"c94bdf86-5de9-49fe-8c98-0f5c47611bfe","SGLang v0.5.18 发布:大模型冷启动提速 2.38 倍,710 个 PR 都改了什么","sglang-v0-5-18-cold-start-2-38x","2026-08-24T23:15:00+00:00",{"id":64,"title":65,"news_slug":66,"published_at":67},"988bbfb8-672a-4c6c-98f7-3a170b6bd8b3","Macaw 把 LFM2.5 装进 1.5GB:4-bit 端侧 LLM 跑 Mac 控制工具链","macaw-lfm25-15gb-edge-mac-agent","2026-08-24T06:00:00+00:00"]