[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-needle-3-tiny-tool-calling-model":3,"topics-all":38,"news-related-13378d5e-2440-496d-8c3c-7d36858e641d":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"13378d5e-2440-496d-8c3c-7d36858e641d","不聊天的端侧基座:Needle 3 用 8-29MB 在微控制器上跑工具调用","Cactus Compute 开源 Needle 3:8-29MB 的端侧基座模型,砍掉通用聊天,专注工具调用、结构化抽取和嵌入三件事,2-20 层任意深度可裁剪,CQ 2-bit 量化,官方称微调后 29M 参数即超过 DeepSeek V4 Flash。","在手机、手表、机器人这些设备上跑大模型，过去几年的主流答案是拿通用小模型量化了塞进去——Gemma E2B 这类「什么都能干一点」的底座。Cactus Compute 刚刚开源的 Needle 3 把思路反过来：主动砍掉通用聊天能力，只做应用真正需要的三件事，把一个工具调用基座模型压成 8-29 MB 的单文件。\n\n## 先说它主动放弃了什么\n\nNeedle 3 的定位是手机、可穿戴、机器人、智能家居、汽车和微控制器上的 foundation model，但和通用小模型不同，它只干三件事，而且全部在设备端完成：\n\n- **工具调用**：给定 App 暴露的函数列表，模型挑出该调用的那个并填好全部参数；用户一次要两件事，就按顺序返回两次调用；没有工具能覆盖的请求返回空列表，而不是硬猜一个。\n- **结构化抽取**：声明一个 schema，丢进一段乱文本，拿回类型化字段——发票、订单、通知、表单都行；解码由从 schema 编译出的字节级语法约束，输出保证能被解析。\n- **文本嵌入**：同一个模型还能给句子返回向量，让 App 在本地做检索、匹配和路由。\n\n官方的说法是，用这点容量换来的是「在移动端工具调用上打赢 10 倍大的模型、在抽取上打平 2-3 倍大的模型」——团队自报口径，benchmark 图放在模型卡上，工具调用按精确匹配、抽取按字段 micro-F1 计。\n\n## 一份权重，从 2 层裁到 20 层\n\n架构上 Needle 3 是一个 Laddered Simple Attention Network：FFN 换成 Monarch Hadamard MLP，注意力是带因果卷积抽头的 GQA，大部分参数放在一块用 gather 读取的 engram n-gram 记忆里——官方称 121M 参数的模型只做 50M 的算术量。训练方式让 2 到 20 层的任意深度都是一个可部署的模型，构建时按设备算力裁剪。\n\n权重用 Cactus Quants 压到 CQ 2-bit（每权重 2.125 bit），整个文件 8-29 MB，每个平台的推理引擎不到 1 MB。另一个值得注意的设计：每个响应都带一个由学习出的置信度头给出的校准分数，引擎据此决定执行、确认还是拒绝——小模型明确知道自己什么时候不该动手。\n\n## 微调之后才见真章\n\n模型卡里分量最重的数字集中在微调部分：在 DroidCall 上微调后，所有子网络提升 18-36 个点，从 4 层（29M 参数）开始，微调后的子网络就能超过 DeepSeek V4 Flash——同样是官方自报。微调路径也很务实：在冻结的 20 层底座上跑 LoRA，然后一条 build 命令合并 adapter、裁出任意层数、导出 4-bit 文件，跑在同一个引擎上。\n\n工程成熟度有迹可循：Apache-2.0 协议，GitHub 仓库 11.4k star、307 次提交，`generation=2` 参数保住了 Needle 2 老部署的兼容性。一个刺眼的细节：二进制默认打开遥测，关掉要设 `NEEDLE_TELEMETRY=0`——端侧隐私叙事下的小刺。\n\n## 所以呢\n\nAgent 的工具调用正在分层：云端用旗舰模型，端侧用什么？Needle 3 给的答案是「用专门的基座模型换掉通用小模型的聊天梦」。它的赌注是：在 8 MB 的世界里，会填参数比会闲聊值钱。这个赌注成不成立，看明年手机厂商和 IoT 团队接不接。\n\n参考：[Hugging Face 模型卡](https:\u002F\u002Fhuggingface.co\u002FCactus-Compute\u002Fneedle3) · [GitHub 仓库](https:\u002F\u002Fgithub.com\u002Fcactus-compute\u002Fneedle)","https:\u002F\u002Fhuggingface.co\u002FCactus-Compute\u002Fneedle3","c313bb92-b553-4241-b2fc-4736540b8957",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"7ac06d8e-b074-4147-abfc-ffaa4c6b8744","ai-efficiency",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"4fc756aa-c5cb-47fd-920d-4120b2b00771","en","Needle 3 packs tool calling into 8-29 MB for microcontrollers","Needle 3: an open 8-29 MB on-device model doing tool calls, extraction and embeddings. Tuned 29M subnets pass DeepSeek V4 Flash, vendor says.","Running models on phones, watches and robots has mostly meant quantizing a general-purpose small model — a Gemma E2B-class base that tries to do a bit of everything. Cactus Compute's freshly open-sourced Needle 3 flips that: it deliberately gives up general chat and does exactly three jobs, packing a tool-calling foundation model into a single 8-29 MB file.\n\n## What it gives up on purpose\n\nNeedle 3 targets mobiles, wearables, robots, smart homes, cars and microcontrollers, but unlike general small models it does three things, all on-device:\n\n- **Tool calls**: given the functions an app exposes, it picks the right one and fills every argument. Two requests produce two calls in order; a request no tool covers returns an empty list, not a guess.\n- **Structured extraction**: declare a schema, hand over messy text, get typed fields back — invoices, bookings, notifications, forms. A byte-level grammar compiled from the schema constrains every token, so output is guaranteed to parse.\n- **Text embedding**: the same model returns a sentence vector, so an app can search, match and route locally.\n\nThe official claim: that capacity trade buys beating models 10x its size on mobile tool calls and matching 2-3x bigger models on extraction — vendor-reported, with benchmark charts on the model card (exact-match for tool calls, field micro-F1 for extraction).\n\n## One weight file, every depth from 2 to 20 layers\n\nArchitecturally, Needle 3 is a Laddered Simple Attention Network: the FFN is replaced by a Monarch Hadamard MLP, attention is GQA with causal conv taps, and most parameters live in an engram n-gram memory read by gather — the company says the 121M-parameter model does the arithmetic of a 50M one. Training is arranged so that every depth from 2 to 20 layers is a deployable model; you cut at build time to fit the device.\n\nWeights are compressed with Cactus Quants to CQ 2-bit (2.125 bits per weight); the whole file is 8-29 MB and each platform's engine is under 1 MB. Another design worth noting: every response carries a calibrated confidence score from a learned head, which the engine uses to route between acting, confirming or refusing — a small model that knows when not to act.\n\n## Fine-tuning is where it gets serious\n\nThe heaviest numbers in the model card sit in the fine-tuning section: after fine-tuning on DroidCall, every subnetwork gains 18-36 points, and from 4 layers (29M parameters) up, the tuned subnetwork passes DeepSeek V4 Flash — again vendor-reported. The workflow is pragmatic: LoRA on the frozen 20-layer base, then one build command merges the adapter, slices any subnetwork and exports a 4-bit file that runs on the same engine.\n\nEngineering maturity shows: Apache-2.0 license, 11.4k GitHub stars, 307 commits, and a `generation=2` flag keeps Needle 2 deployments working. One wrinkle: telemetry is on by default in the binary, disabled via `NEEDLE_TELEMETRY=0` — a small thorn in an on-device privacy story.\n\n## So what\n\nAgent tool calling is splitting into layers: frontier models in the cloud, and what at the edge? Needle 3's answer is to swap the generalist small model's chat dreams for a dedicated base. Its bet: in an 8 MB world, filling arguments correctly is worth more than small talk. Whether that bet pays off shows up in who adopts it next year.\n\nRefs: [HF model card](https:\u002F\u002Fhuggingface.co\u002FCactus-Compute\u002Fneedle3) · [GitHub repo](https:\u002F\u002Fgithub.com\u002Fcactus-compute\u002Fneedle)","needle-3-tiny-tool-calling-model","2026-09-19T13:09:46Z","2026-09-19T13:10:06.021863Z","2026-09-19T13:10:06.021872Z",true,"agent",160,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"0565190a-0bcd-492f-934f-0ad2ab32f485","70万参数2.8MB填一张表:Cua开源CUA-S1,单次前向替代23轮LLM","cua-s1-forms-system-one-model","2026-09-20T13:11:48+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"259d91b2-ed6b-4af8-8f2a-f759b84cc617","蚂蚁 Ling-3.0 Flash：124B\u002F5.1B MoE 的 Agent 生产级模型","inclusionai-ling-3-flash-hybrid-linear-moe-agent","2026-08-14T08:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"9ba1770e-87f2-47b9-aaa1-19f4f2ba78f1","ScienceIDE:把全球科学代码变成智能体训练场","scienceide-scientific-code-agent-environments","2026-09-17T23:05:17+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"4c4a2a9e-f69b-4985-bd42-97ab2ef4e2ac","Spark-X2.5-4B 开源:4B 跑 1M 上下文,22 项基准打 9B 级 Qwen3.5","spark-x2-5-4b-apache-open-source","2026-09-16T01:30:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"dcd8b3e1-a3c7-4614-aba4-9002219ea5f6","LibreDB Studio 0.15 发布:本地 LLM 接管数据库交互","libredb-studio-local-llm-agent","2026-09-15T00:00:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"dcbb1042-4fef-4a6a-bd7a-0e11b4d97c10","TRL v1.13.0:一处 dtype 修复拿回 21.6% kernel 时间,PPOTrainer 退役","trl-v1-13-tensor-core-lmhead-ppotrainer","2026-09-12T21:05:00+00:00"]