[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-strata-125b-12gb-gaming-gpu":3,"topics-all":38,"news-related-4afaf05e-8b64-4533-8868-c4985e292a35":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"4afaf05e-8b64-4533-8868-c4985e292a35","Strata 125B 开源引擎跑进 12GB 游戏卡","Strata v0.1.39 把 1250 亿参数的 Qwen3.8-Flash-Next 塞进 12GB 游戏卡：Q2_0 实测 94 token\u002Fs 写出、读 32K prompt 2650 token\u002Fs。解码快 6% 且输出逐字节不变，双 4090 提速 63%，两周攒 10.9k star。","一台 12GB 显存的 RTX 5070 加 64GB 内存，就能以每秒 94 个 token 跑一个 1250 亿参数的开源模型——这是开源项目 [Strata](https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata\u002Freleases\u002Ftag\u002Fv0.1.39) README 写明的自测数字。9 月 24 日第一行代码，两周 845 次提交攒下 10.9k star，专做一件事：把 Qwen3.8-Flash-Next 搬进游戏 PC，推理全程数据不出本机。\n\n## 门槛和速度：一张游戏机的配置单\n\n按 README，显卡只需 RTX 20\u002F30\u002F40\u002F50 系或 AMD RX 7900\u002F9070 等主流游戏卡，**12GB 显存起步**，内存 32GB 起、64GB 跑全部档位；启动载入 35-55GB，首启 1-3 分钟整机可能卡顿。\n\n作者两台游戏 PC 实测：RTX 5070 上最激进的 Q2_0 档写出 94 token\u002Fs、读 32K 文档 2650 token\u002Fs；质量更高的 IQ3_S 为 53 token\u002Fs；AMD RX 9070 XT 上 Q2_0 为 60 token\u002Fs；RTX 3090 预计 100-140 token\u002Fs（推算）。上手：AI 助手照 docs\u002FAI_SETUP.md 代装，或走 127.0.0.1:8080 的 OpenAI\u002FAnthropic 兼容接口接入现有应用。\n\n## v0.1.39：解码快 6%，输出一个字节都不变\n\n10 月 4 日的新版里，解码重构（sub-warp 专家打包、PLE\u002FMTP 合并批处理）让 RTX 5070 上 Q2_0 从 68.4 提到 72.7 token\u002Fs（约 +6%），IQ3_XXS 提升 2.5%-6%，4K\u002F32K 长 prompt 后的解码再快 5%-7%——而四个量化档各 10 组对照下，**输出与 0.1.38 逐字节相同**。量化\"差不多就行\"的本地推理圈里，这种纪律相当少见。\n\n长 prompt 收益更大：32K 下 IQ3_XXS 提速最高 18.5%（1500 槽专家缓存），243K token 输入在 RTX 3060 上快 26%。多卡上，双 RTX 4090 混合负载 +63%、代码 +132%（贡献者自测）；四张 16GB 卡跑 8 路并发总吞吐 360 token\u002Fs 对单路 120。Responses API 已实现，Codex CLI 0.160.0 实测可用，后续轮次约 96% prompt 命中缓存。\n\n## 显存放不下，就把专家搬进内存和 SSD\n\nQwen3.8-Flash-Next 是 MoE 架构，每 token 只激活一小部分专家，Strata 把放不进显存的专家权重摊到内存和 SSD，按需流式喂给 GPU（streamed expert ring、`--prefill auto` 即此）。档位拉开空间：砍一半专家的 Coder 版只需 32GB 内存，官方自测达完整模型 SWE-bench Verified 的 91%；Swift 1.5 思考更短出答案更快；Unsloth UD-IQ4_XS 约 4-bit 质量更高，代价是 94GB 下载。\n\n## 冷水也得泼\n\n94 token\u002Fs 是最激进的 Q2_0 档，追求质量的 IQ3_S 掉到 53；Coder 版官方标注在中文等 CJK 文本偏弱，中文用户应选保留全部专家的档位；Pascal\u002FVolta、Intel Arc 与老 AMD 路径全是社区自测的实验特性。\n\n两周的迭代节奏加上这种纪律，给本地推理立了很高的工程标杆。云 API 定价和配额还在反复横跳的当口，一台游戏机加 80GB 磁盘就是你的退路——这条路现在真的通了。","https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata\u002Freleases\u002Ftag\u002Fv0.1.39","17d3ffcc-ce41-4b5d-a2e6-80476cd8725f",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",{"id":22,"name":23,"slug":23,"description":14,"color":14},"c187600e-804c-4697-b828-1e4330e0eb10","qwen",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"67a21ceb-9aa7-4c99-b7eb-0d168fc0e1ba","en","Strata runs a 125B model on a 12GB gaming GPU","Strata v0.1.39 runs a 125B model on a 12GB gaming GPU: 94 tok\u002Fs decode and 2,650 tok\u002Fs on 32K prompts; decode is 6% faster, output byte-identical.","An RTX 5070 with 12GB of VRAM and 64GB of RAM can now run a 125-billion-parameter open-weight model at 94 tokens per second. That is not a vendor demo — it is the self-measured number in the README of [Strata](https:\u002F\u002Fgithub.com\u002FNiko1221\u002FStrata\u002Freleases\u002Ftag\u002Fv0.1.39), an open-source project whose first commit landed on September 24 and which shipped v0.1.39 on October 4: 845 commits and 10.9k stars in two weeks, all aimed at one thing — moving Qwen3.8-Flash-Next from the server room onto an ordinary gaming PC, with model files pulled automatically from Hugging Face and nothing leaving the machine.\n\n## Requirements and speed: a gaming-PC spec sheet\n\nPer the project README, the GPU can be any mainstream NVIDIA RTX 20\u002F30\u002F40\u002F50 card or an AMD RX 7900\u002F9070-class card with **12GB of VRAM or more**, alongside 32GB of RAM (64GB fits every size) and about 80GB of disk. The model download is roughly 70GB; at startup Strata loads 35-55GB into RAM and locks part of it for the card — the docs warn the first start may make the PC unresponsive for one to three minutes.\n\nSpeed was measured by the authors on two ordinary gaming PCs. On an RTX 5070 (12GB), the most aggressive Q2_0 quant writes at 94 tok\u002Fs and reads a 32K document at 2,650 tok\u002Fs; the higher-quality IQ3_S size runs at 53 tok\u002Fs. On an AMD RX 9070 XT (16GB), Q2_0 writes at 60 tok\u002Fs. The README says a 24GB RTX 3090 should reach 100-140 tok\u002Fs — note that this is a projection, not a measurement. Two conveniences stand out for regular users: an AI assistant (Claude Code, Cursor, Codex) can perform the whole install by following docs\u002FAI_SETUP.md in the repo, and the engine exposes OpenAI- and Anthropic-compatible APIs on localhost 127.0.0.1:8080 so existing apps and coding agents connect directly.\n\n## v0.1.39: decode 6% faster, output byte-for-byte identical\n\nThe engineering taste in this release deserves its own note. After reworking the decode path (sub-warp expert packing, batching the PLE and MTP steps), Q2_0 on the RTX 5070 went from 68.4 to 72.7 tok\u002Fs (about +6%), IQ3_XXS gained 2.5-6%, and decode after 4K and 32K prompts is another 5-7% faster — yet the project ran ten interleaved comparison runs on all four quant sizes and found the output **byte-identical to 0.1.38**. Faster without changing a single answer is rare in local inference, where \"close enough\" quantization is the norm.\n\nThe long-prompt path gains more: up to 18.5% on 32K prompts with IQ3_XXS (with a 1,500-slot expert cache), and +26% on a 243K-token input on an RTX 3060. Multi-GPU now has a real story: dual RTX 4090s gain +63% on mixed text and +132% on code (numbers measured by contributors, not the author, who owns one GPU), and a four-way layer split across 16GB cards served 8 concurrent requests at 360 tok\u002Fs total versus 120 for one. The release also adds parallel request slots, experimental engines for older Pascal\u002FVolta and Intel Arc hardware, and an OpenAI Responses API — tested with Codex CLI 0.160.0, with later turns reusing about 96% of the prompt from cache.\n\n## If it does not fit in VRAM, stream the experts from RAM and SSD\n\nThe idea is not mysterious: Qwen3.8-Flash-Next is a MoE model, so each token activates only a small slice of the experts. Strata spreads the experts that do not fit in VRAM across RAM and SSD and streams them to the GPU on demand — the \"streamed expert ring\" and `--prefill auto` in the release notes are exactly this. Quant sizes open up the trade-offs: the Coder build removes half the experts, needs only 32GB of RAM, and reaches 91% of the full model's SWE-bench Verified score by the authors' own measurement; the Swift 1.5 fine-tune thinks for less time before answering; Unsloth's ~4-bit UD-IQ4_XS trades a 94GB download for higher quality.\n\n## Some cold water\n\nFirst, 94 tok\u002Fs is the most aggressive Q2_0 size — the quality-oriented IQ3_S drops to 53. Second, the authors explicitly note the Coder build is weaker outside code, including Chinese and other CJK text; Chinese-speaking users should pick a size that keeps every expert. Third, the Pascal\u002FVolta, Intel Arc and older AMD paths are experimental features self-measured by community contributors on hardware the author does not own. Fourth, the 3090's 100-140 tok\u002Fs is a projection. In other words, \"a 125B model on a gaming PC\" holds, but every speed grade needs to be read against your own VRAM and RAM.\n\nEven so, the two-week cadence and the discipline of \"faster only if byte-identical\" set a high engineering bar for local inference. While cloud API pricing and quotas keep swinging, a gaming PC plus 80GB of disk is your fallback — and that road now genuinely works.","strata-125b-12gb-gaming-gpu","2026-10-04T23:05:30Z","2026-10-04T23:11:09.680911Z","2026-10-04T23:11:09.680919Z",true,"agent",717,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"3a419e1c-fb2b-467e-a5b6-a7188525bc03","TraceForge:推理痕迹自己造,学生反超教师真迹","traceforge-family-native-distillation","2026-10-22T15:10:42+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"e880f146-5f06-4b07-a4da-e6a7391ddcff","DeepSeek 开源 DeepInfer:生产推理引擎放出来,服务那一半还留在内部","deepseek-deepinfer-open-source-inference-engine","2026-10-22T03:10:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"559df680-f13e-48f0-9263-73116aaff411","上交大DLite免历史投机解码:Agent提速2.74倍","dlite-history-free-speculative-decoding","2026-10-19T17:07:52+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"d1b7ef56-357c-4b38-841e-4d4d60970490","Waypoint-OPD给蒸馏加方向门控：教师前后打架就不学","waypoint-opd-direction-gated-distillation","2026-10-17T23:20:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"f7234b7e-a2c3-404b-9fc4-aaca8e0c8f91","Edge0 预测路由:35B MoE 挤进 24GB Mac","edge0-prerouter-ssd-moe","2026-09-17T15:10:05+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36+00:00"]