Microsoft's BitNet team open-sourced the BitNet Embeddings series on Hugging Face this week, focused on the "BITEMBED" framework — for the first time extending training-time 1.58-bit quantization from the LLM backbone to the text-embedding task, covering Qwen3-0.6B and Gemma3-270M backbones, with weights ternary-quantized to {-1, 0, +1}, activations 8-bit, paired with SubLN and RoPE, 32K context, MIT license. The technical path is two steps: step 1 first converts a pretrained LLM directly into a BitNet-style embedding encoder, preserving absmean quantization and abs-max-per-token activation quantization; step 2 realigns on representation learning — first does continued contrastive pretraining (InfoNCE) on 1B text pairs, then uses supervised contrastive fine-tuning to distill from an FP16 teacher, with two distillation signals: similarity-distribution and attention-relation, letting the representation capability under ternary weights converge toward the FP16 teacher. Numbers are tough: on MMTEB (eng, v2) BitNet-Embedding-0.6B gets an 67.60 average score, only 0.35 below the FP16 teacher (67.95), with 8 sub-tasks (classification, retrieval, summarization, etc.) almost tied; on the inference side, CPU 8 threads pp128 input 870.90 t/s, a 2.28× speedup over FP16, long input still maintains a 1.4–1.8× lead; the output embedding can also be re-quantized to 1/2/4/8 bit, with storage scaling elastically by scenario. This is no small matter for RAG and enterprise knowledge bases: the embedding stage is the "CPU-resident bottleneck" of retrieval systems, and pushing the backbone to 1.58-bit during training means the entire retrieval pipeline can, for the first time, run end-to-end on a 1-bit link, and the bitnet.cpp inference framework continues to be reused. The past BitNet story only ran to the LLM itself, this time it completes the last mile of retrieval vectors — open-source engineering components are in place, what remains is plugging it into mainstream orchestration layers like LangChain / LlamaIndex, and the community ecosystem may quickly make this configuration the default RAG option. Worth noting even more is BitNet's "gradual expansion" strategy: from 1-bit LLM inference framework, to today's 1.58-bit embedding, each step hasn't been eager to switch from FP16 to integer precision, but rather making the model learn to converge in ternary representations during training — this path may be more reproducible on edge AI than imagined.