[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-oreolook-three-layer-cpu-cache":3,"topics-all":38,"news-related-2638aeac-dc4d-4b73-b7fe-2b042015adee":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"2638aeac-dc4d-4b73-b7fe-2b042015adee","OreoLook 开源:三层缓存把 AI 搜索搬进 8 核 CPU,重复问题 0.1 毫秒出答案","Pollinations 团队开源 AI 搜索引擎 OreoLook,论文给出三层缓存架构:Redis 会话窗口、语义查询缓存、URL 嵌入缓存,在单台 8 核 CPU 服务器上跑出 89.3% 命中率、0.1 毫秒读取延迟,重新定义\"小机器也能做 AI 搜索\"的成本底线。","AI 搜索引擎的第一反应是\"堆 GPU\"——但 Pollinations 团队刚刚用一篇 arXiv 论文证明:把缓存做对,一台 8 核 CPU 服务器就够撑起一个带会话记忆、语义去重的开源 AI 搜索引擎。论文《A Three-Layer Caching Architecture for Low-Latency LLM Web Search on Commodity CPU Hardware》(arXiv:2609.05463)围绕他们开源的 OreoLook(前名 lixSearch)展开:本地搜索、缓存、会话管理、嵌入栈全部跑在普通 CPU 硬件上,答案合成走远程推理服务。\n\n## 为什么缓存是 AI 搜索的成本命门\n\nOreoLook 是一个自动化浏览器 Agent 加多路推理路由的答案引擎:问题进来后,搜索 Agent 并发散到网页、YouTube、图片,提取全文与转录,再由 LLM 合成带来源的答案。作者在摘要里直接点出规模化后的三个痛点:会话丢上下文、换种问法就触发重复计算、同一个 URL 在不同会话里被反复嵌入。这三个问题的共性是:钱花在了已经算过的东西上。\n\n## 三层缓存,各管一段\n\n论文的核心设计是三层缓存,每一层对应一个具体的浪费源:\n\n- **会话上下文窗口**:最近对话滚动保存在 Redis,溢出部分经 Huffman 压缩归档到磁盘。配合后台 LRU 驱逐守护进程,闲置会话自动从 Redis 迁到磁盘、需要时再水合回来,对话可以按保留策略在数小时或数天后恢复。\n- **语义查询缓存**:对查询的嵌入向量做余弦相似度匹配,换种问法的等价查询直接命中缓存,省掉整条搜索+合成流水线。仓库 README 给出实现参数:Redis DB0,余弦阈值 0.90,重复查询 15 毫秒内返回。\n- **URL 嵌入缓存**:跨会话去重 URL 的嵌入计算,同一个网页不再被反复向量化。\n\n部署数字直接取自论文:单台 8-vCPU Intel Cascade Lake 服务器(2 GHz、32 GB 内存),跑 30 个 Hypercorn worker 进程、分三个容器化副本,聚合 Redis 命中率 89.3%,读取延迟 0.1 毫秒,Redis 内存开销仅 1.38 MB。\n\n## 单源数字怎么看\n\n有一点值得说明:89.3% 是 Redis 键空间命中率,作者在社区讨论里明确区分了\"键空间命中率\"与\"端到端查询避免率\"两个概念——这是生产系统论文里少见的老实。仓库还把这套缓存拆成了独立 PyPI 库 lix-open-cache,只依赖 Redis、numpy、loguru,不依赖服务器端;完整搜索引擎以 Docker 镜像发布,一条 docker compose 命令即可自托管。技术栈上,检索层用 Qdrant,浏览器自动化用 Playwright,API 层兼容 OpenAI 格式,还带一个无状态 MCP 端点支持深度研究与 PDF 导出。\n\n## 所以呢\n\n对做 AI 搜索产品的团队,这篇论文的参考价值不在模型而在账本:大厂把答案合成的算力成本转嫁给了用户看不见的推理集群,而 OreoLook 论文口径下近九成键空间读取由缓存接管,单次读取 0.1 毫秒。三层缓存里最值得抄的是语义查询缓存——换种问法就能命中,这在多轮对话产品里几乎是白捡的延迟优化。论文地址:arxiv.org\u002Fabs\u002F2609.05463,代码在 GitHub pollinations\u002Fsearch.elixpo。","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.05463","7437aeb9-930c-4866-a2e9-48003c1a792b",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"0a93ec8e-ea39-4693-81de-563ca8c173f7","inference",{"id":19,"name":20,"slug":20,"description":14,"color":14},"01598627-1ea6-4b27-a5d8-874971571a71","llm",{"id":22,"name":23,"slug":23,"description":14,"color":14},"b9bd9039-fcdb-41a8-b85b-fc1587def2b9","open-source",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"ea6f2a42-d7ec-4734-ad82-30f31c8d1682","en","OreoLook Open-Sources Three-Layer Caching: AI Search on an 8-Core CPU","OreoLook's three-layer cache—session window, semantic query cache, URL embedding cache—hits 89.3% at 0.1 ms reads on one 8-vCPU CPU server.","The first instinct in AI search engineering is to throw GPUs at the problem. A new arXiv paper from the Pollinations team argues otherwise: get caching right, and a single 8-core CPU server is enough to power an open-source AI search engine with session memory and semantic deduplication. The paper, \"A Three-Layer Caching Architecture for Low-Latency LLM Web Search on Commodity CPU Hardware\" (arXiv:2609.05463), centers on OreoLook (formerly lixSearch): local search, caching, session management, and the embedding stack all run on commodity CPU hardware, while answer synthesis is routed to a remote inference provider.\n\n## Why Caching Is the Cost Bottleneck of AI Search\n\nOreoLook is an answer engine built on automated browser agents and provider-routed LLM inference. When a question arrives, search agents fan out across the web, YouTube, and images, extract full text and transcripts, and an LLM synthesizes a sourced answer. The authors name three pain points that emerged as usage grew: sessions lost context, equivalent queries triggered redundant work, and URLs were repeatedly embedded across sessions. The common thread: money spent recomputing what was already known.\n\n## Three Layers, One Job Each\n\nThe core design is a three-layer cache, each layer targeting a specific waste source:\n\n- **Session Context Window**: recent conversation turns stay in Redis, with overflow archived to disk using Huffman compression. A background LRU eviction daemon migrates idle sessions from Redis to disk and re-hydrates them on demand, so conversations can resume hours or days later under the configured retention policy.\n- **Semantic Query Cache**: matching on embedding vectors via cosine similarity catches rephrased queries, skipping the entire search-and-synthesis pipeline. The repo README gives the implementation parameters: Redis DB0, cosine threshold 0.90, repeat queries resolving in under 15 ms.\n- **URL Embedding Cache**: deduplicates embedding computations for URLs encountered across different sessions.\n\nThe deployment numbers come straight from the paper: a single 8-vCPU Intel Cascade Lake server (2 GHz, 32 GB RAM) running 30 Hypercorn worker processes across three containerized replicas, reporting an 89.3% aggregate Redis keyspace hit rate, 0.1 ms read latency, and just 1.38 MB of Redis memory overhead.\n\n## Reading Single-Source Numbers Honestly\n\nOne point deserves attention: 89.3% is a Redis keyspace hit rate, and the authors explicitly distinguish keyspace hit rate from end-to-end query avoidance in the community discussion—a rare bit of honesty in production-system papers. The repo also packages the caching layer as a standalone PyPI library, lix-open-cache, which depends only on Redis, numpy, and loguru, with no server required; the full search engine ships as a Docker image, self-hostable with a single docker compose command. On the tech stack side, retrieval runs on Qdrant, browser automation on Playwright, the API layer is OpenAI-compatible, and a stateless MCP endpoint supports deep research and PDF exports.\n\n## So What\n\nFor teams building AI search products, the reference value of this paper lies not in models but in the ledger: big players absorb synthesis compute costs in inference clusters users never see, while under OreoLook's paper-narrow metric, nearly nine in ten keyspace reads are served by cache at 0.1 ms each. The semantic query cache is the layer most worth copying—a rephrased question hitting cache is almost free latency optimization in any multi-turn product. Paper: arxiv.org\u002Fabs\u002F2609.05463; code at GitHub pollinations\u002Fsearch.elixpo.","oreolook-three-layer-cpu-cache","2026-09-10T23:08:36Z","2026-09-10T23:08:39.039484Z","2026-09-10T23:08:39.039493Z",true,"agent",117,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"0fe9ceb8-6411-4924-8e02-8cee3665fc6f","Cohere 开源 megakernel 推理引擎：单 CUDA 文件，H100 解码吃到 62% 带宽光速","cohere-megakernel-north-mini-code-h100","2026-09-08T21:13:46+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"0c29e1ad-914a-4b79-a153-445c087acb03","被 LLM 抛弃的 dropout 翻身:Cerebras 称调好可省 25% 训练 FLOPs","dont-drop-dropout-layer-sparsity","2026-09-07T21:06:35+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"199cd4ef-f092-45a5-8635-91778dd2bce2","编译即训练：一句规约炼出 83.6% 准确率的本地神经函数，教师模型只用一次","compile-by-training-neural-functions","2026-09-04T23:08:03+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"7a6d28b6-65da-4a29-96b1-dedb9894de97","随机驱逐追平最强打分器:Salesforce 重写 KV Cache 压缩常识","random-attention-kv-cache-eviction","2026-09-04T19:08:26+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"44a035c8-b8a3-48e5-af4f-c76323dac7b5","RWKV7-G1j 13.3B 开源:不用注意力,每 token 推理成本是常数","rwkv7-g1j-13b-attention-free","2026-09-03T13:14:19+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"ff3b7c9f-e338-4af9-a562-4fda8b1bc929","vLLM 0.28 发布:584 项提交押注 Kimi K3,每卡省 17GB 显存","vllm-0-28-kimi-k3-optimization","2026-08-29T19:05:00+00:00"]