[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"news-slug-openai-pando-git-backend-codex":3,"topics-all":38,"news-related-bdf7fc57-def2-4b4e-9ce6-0fd9918721eb":57},{"id":4,"title":5,"summary":6,"content":7,"original_url":8,"source_id":9,"tags":10,"translations":24,"news_slug":31,"published_at":32,"created_at":33,"modified_at":34,"is_published":35,"publish_type":36,"image_url":14,"view_count":37},"bdf7fc57-def2-4b4e-9ce6-0fd9918721eb","Codex 撑爆 Git,OpenAI 自建 Pando","Codex 让 OpenAI 的 Git 服务端先撑不住:可靠性一度掉到 90% 以下。官方复盘拆解 Pando 的取舍——放弃仓库级一致改用因果一致性,分支级并发写,packfile 当 CDN 缓存加 L0-L4 压缩,中位推送延迟降近八成,每天交付近 1 PB Git 数据。","Git 诞生于「用邮件给内核传补丁」的时代,二十多年里开发者一直围着它改造工具链。但 OpenAI 这份复盘暴露了另一件事:当写代码的主体从人变成 Agent,Git 服务端的默认假设会集体失效。\n\n## 被撑破的是服务端\n\nOpenAI 的 monorepo 主要是 Python,外加 Rust 服务和 Go 服务。Codex 改变的是规模与节奏:分支、推送、合并,以及来自开发环境、研究任务和 CI 的拉取全部成倍增加;一次推送会让多个任务去 fetch 同一个 commit。到 2025 年底后端开始吃力:慢 fetch、慢 push、合并失败,可靠性一度掉到 90% 以下。\n\n传统上扩 Git 服务大致两条路,OpenAI 都评估过:**三阶段提交**让每个副本保留完整仓库,需多数派接受才能发布 ref 更新,读可本地服务、客户端拿到一致视图,但每次推送都要多轮副本往返;**共享 WAL** 把所有 ref 更新写进同一份 manifest,用 compare-and-swap 原子替换,写可并行上传,但不相关分支仍要争抢它。\n\nPando 的选择很明确:放弃仓库级一致,换成因果一致性。\n\n## 一致性让位,分支级并发写\n\nPando 把每次分支移动记成一个 ref version:新 tip、提供它的 pack、以及它依赖的更早 ref version,写进元数据存储;packfile 作为不可变 blob 存进 blob store,两者都能横向扩展。写入时它一边把 packfile 流式写入 blob store,一边用单次解码完成校验,不必展开进本地仓库;随后用多键事务记录 ref version 并原子更新 ref 指针,顺带给出 read-your-own-write 保证,整个写操作只需常数次元数据读写。\n\n代价写在明处:读多个分支时,客户端不能假设这些读对应同一时间点;但无关分支的推送完全并行,只有同一 ref 的并发推送才冲突。\n\n还有个容易被忽略的细节:Git 协议里 server 会把所有 ref 广播给客户端,分支数到百万级时单这次广播就能超过 100 MiB。Pando 只广播选定的 ref 集合,默认推送新特性分支时只广播一个主干分支作基线,代价是连续推送同一分支要多传主干之外的东西;换来的是 packfile 中位数不到 20 KiB,第 99 百分位不到 3 MiB。\n\n## 把 packfile 当 CDN 缓存\n\n读取路径上,Pando 不再为每个客户端单独生成 packfile。它用推送时记录的提交映射,把客户端的 have \u002F want 反解成一组 ref version,选出覆盖这些对象的已存 packfile,返回 packfile URI 让客户端自取。Git 协议规定一次 fetch 只能塞一个 packfile,但 `packfile-uri` 机制允许服务端额外返回下载地址,于是它内联传一个空 packfile,真正的数据走名为 Express 的服务——在各区域缓存 blob store,用一致性哈希路由,命中率超过 99%。用标准云厂商虚拟机,clone 和 fetch 能打满 10 Gbps 链路。\n\n小 packfile 太多是另一个问题:每次推送都发一个 URI,客户端追一次主干要下载几万个文件。Pando 在后台把 packfile 合并成 cover,从 L0 逐级几何放大,目前主干用到 L4,设计接近日志结构合并树的压缩。cover 还充当 ref version 之上的跳表,让 fetch 规划能以对数时间选出要发的包;各层级的 packfile 都保留不回收,因为读任意对象时从 L0 读通常比从 L4 读更省。效果是单小时服务过超过 57 TB 的 Git packs,其中 99% 以上的字节来自本地磁盘缓存。\n\n## 用 Codex 造 Codex 的基础设施\n\nPando 从立项到上线生产只用了十周:核心由一名工程师设计,多个 Agent 团队实现,由少数工程师部署,今年 4 月开始承接生产流量。它用 Rust 写、不带遗留依赖;为了让 Agent 能自我验证,团队先写了符合性测试套件比对标准 Git 行为,又搭了确定性模拟环境检查系统不变量。\n\n用 GPT-6 Astra 时,Agent 写 Lean 和 TLA+ 把「Pando 理论」形式化,用来在实现里找 bug 并证明系统性质;还有连续数百小时的会话专门找边缘 case,Agent 则昼夜不停地用 Daybreak 能力模型做红队。上线方式是读写透传代理到旧系统,逐步切流量不中断。\n\n## 反哺上游,以及「所以呢」\n\n在 Pando 上暴露出的客户端 bug 修完就回推 Git 上游:并发下载、中断传输后的清理、已完成下载的续传、接受已有 pack 时的校验。还贡献了「explicit haves」概念——服务端可单独广播分支当前 tip,而不广播推送方可能已略过的历史,客户端于是能用 `--force-with-lease` 只对主干构包。这些改动有一部分已进了上游 Git。\n\n这件事真正的意义可能不在 Git。Git 服务端二十年的前提是「一个人写、一个人推」,当 Agent 把分支数、推送频率和并发拉取同时抬起来,被打破的是服务端而非客户端。OpenAI 干脆放弃仓库级一致、接受因果一致性,说明在他们的负载里全局 ref 顺序已不重要到可以牺牲——这是「机器主导工作流」的具体信号。更值得注意的是另一层:十周、单人设计、Agent 实现,这套基础设施本身也是 Agent 写的,连证明系统正确性的 Lean 和 TLA+ 也是 Agent 写的。瓶颈从「模型够不够强」转移到了「系统怎么设计、谁来 review」。\n\n不过 Pando 并未开源,没有公开 API 和部署指南,外界能直接拿到的只有回推上游的 Git 客户端补丁。对同规模团队的启示是:先量一下工具链里,有多少假设是「一个人一次一个分支」。\n\n原始技术细节见 OpenAI 工程博客 Scaling Git at OpenAI(https:\u002F\u002Fopenai.com\u002Findex\u002Fscaling-git-at-openai\u002F),关键指标的第三方交叉印证见 AlphaSignal 报道(https:\u002F\u002Falphasignal.ai\u002Fnews\u002Fopenai-s-pando-cuts-push-latency-80-serving-a-petabyte-of-git-daily)。","https:\u002F\u002Fopenai.com\u002Findex\u002Fscaling-git-at-openai\u002F","15975962-b5fe-49e5-ae68-687ba6cb7015",[11,15,18,21],{"id":12,"name":13,"slug":13,"description":14,"color":14},"6ad31a14-c0da-42df-81fd-564281f768db","agentic-ai",null,{"id":16,"name":17,"slug":17,"description":14,"color":14},"e82b2d09-81b2-43d1-977e-e018443b3c14","coding-agent",{"id":19,"name":20,"slug":20,"description":14,"color":14},"0ef8513a-0a26-42f0-b6f9-5b6dadded45c","efficiency",{"id":22,"name":23,"slug":23,"description":14,"color":14},"42e59a88-7795-47dc-a334-ef1e72c24347","openai",[25],{"id":26,"lang":27,"title":28,"summary":29,"content":30},"08358ddd-7af5-4982-aae7-fe1a20d23a04","en","OpenAI's Pando: Codex Grew So Big It Rewrote Git's Backend","OpenAI built Pando, a Git-compatible backend, after Codex commit volume overwhelmed its monorepo. It cuts median push latency nearly 5x.","Git was born in an era of sending patches to the kernel over email, and for more than two decades developers adapted their tooling around it. But an OpenAI engineering retrospective makes something else clear: once the primary author of code shifts from humans to agents, the default assumptions baked into Git's server side collapse all at once.\n\n## The server side is what broke\n\nOpenAI's monorepo is mostly Python, plus Rust and Go services. Codex changed the scale and pace of development: branch creation, pushes, merges, plus fetches from dev environments, research jobs and CI all multiply. One engineer might have several changes queued for CI at once, and a single push can leave several jobs fetching the same commit. Starting around the end of last year the backend began to struggle: slow fetches, slow pushes, failed merges, with reliability at times falling below 90%.\n\nThere are broadly two conventional ways to scale a Git service, and OpenAI evaluated both. **Three-phase commit** keeps a complete copy of the repository on every replica and needs a majority of replicas to accept a ref update before publishing it; reads can be served locally and clients get a consistent view of all references, but each push needs multiple round trips among replicas. **A shared write-ahead log** records all ref updates in one manifest, swapped atomically via compare-and-swap; writes can upload in parallel, but pushes to unrelated branches still contend for that one manifest.\n\nPando's choice is explicit: give up repository-wide consistency in exchange for causal consistency.\n\n## Trading consistency for branch-level parallel writes\n\nPando records every branch move as a ref version — the new tip, the pack that supplied it, and whichever earlier ref versions that pack depends on — written into a metadata store, while the packfile itself is kept as an immutable blob in blob storage. Both stores scale horizontally. On the write path, Pando streams the incoming packfile into blob storage while validating it with a single decode pass, without unpacking it into a local repository. It then uses a multi-key transaction to record the ref version and atomically update the ref pointer, which also yields read-your-own-write consistency. The entire write operation requires a constant number of metadata reads and writes.\n\nThe tradeoff is written plainly: when reading several branches, a client cannot assume those reads describe a single point in time. But pushes to unrelated branches proceed fully in parallel; only concurrent pushes to the same ref can conflict.\n\nOne easily missed detail: in the Git protocol the server advertises every ref to the client, and with millions of branches that advertisement alone can exceed 100 MiB. Pando advertises only a selected set of refs, and by default a push of a new feature branch advertises just the designated trunk branch as a base. The cost is that repeatedly pushing the same branch forces the client to resend everything outside trunk; the payoff is that packfiles produced in this monorepo have a median under 20 KiB and a 99th percentile under 3 MiB.\n\n## Treating packfiles as a CDN cache\n\nOn the read path, Pando no longer generates a fresh packfile per client. It uses the commit mappings recorded at push time to resolve the client's haves and wants into a set of ref versions, selects the stored packfiles covering those objects, and returns packfile URIs for the client to fetch. The Git protocol only allows a single packfile in the HTTP response for a fetch, but the `packfile-uri` mechanism lets a server return links to additional packs — so Pando uses that exclusively, transmitting an empty packfile inline while the real bytes travel through a separate service called Express, which caches the blob store in each region, routes with consistent hashing, and reports cache hit rates over 99%. Using only standard cloud provider virtual machines, clone and fetch operations saturate 10 Gbps network links.\n\nToo many small packfiles is a separate problem: sending one URI per push would force a client to download tens of thousands of files just to catch up on trunk. Pando merges packfiles into covers in the background, scaling geometrically from L0, and the trunk currently uses up to L4 — a design closely resembling compaction in log-structured merge trees. Covers also act as a skip list over ref versions, letting fetch planning select packs in logarithmic time; packfiles at every level are kept rather than garbage collected, because reading an arbitrary object from L0 tends to move less data than from L4. The result: more than 57 TB of Git packs served in a single hour, with over 99% of those bytes coming from local disk cache.\n\n## Building Codex's infrastructure with Codex\n\nPando took ten weeks from kick-off to production. The core was designed by a single engineer, implemented by teams of agents, and deployed by a handful of engineers; it began serving production traffic in April. It is written in Rust with no legacy dependencies. So agents could verify their own work, the team first wrote a conformance test suite that checks behavior against standard Git, and also built deterministic simulation testing harnesses to check system invariants.\n\nUsing GPT-6 Astra, agents wrote Lean and TLA+ to formalize a \"theory of Pando,\" which they used to find and fix bugs in the implementation and prove properties of the system. Multiple continuous sessions ran for hundreds of hours hunting edge cases, and agents run around the clock red-teaming Git and Pando with Daybreak-capable models. Deployment started as a read- and write-through proxy to the existing source control system, so traffic could move without downtime.\n\n## Giving back upstream, and the so-what\n\nClient-side bugs surfaced while working on Pando were fixed and pushed upstream to Git: concurrent downloads, cleanup after interrupted transfers, resuming downloads that had already completed, and validation when accepting an existing pack for a fetch. The team also contributed the concept of \"explicit haves\" — a server can advertise a branch's current tip separately from history that a pushing client may leave out of its pack, which lets clients use `--force-with-lease` to check a feature branch has not moved and build its pack against trunk alone. Some of these improvements are already in upstream Git.\n\nThe real significance may not be Git itself. Two decades of Git server evolution assumed one person writes and one person pushes; when agents raise branch counts, push frequency and concurrent fetches at the same time, what breaks is the server side, not the client. OpenAI simply abandoned repository-wide consistency and accepted causal consistency, which tells you that in their real workloads, global ordering of ref updates has become unimportant enough to sacrifice — a very concrete signal that machine-driven work is taking over. A second layer is more striking: ten weeks, one designer, agent implementation. This infrastructure was itself written by agents, including the Lean and TLA+ used to prove the system correct. The bottleneck has shifted from whether the model is strong enough to how the system is designed and who reviews it.\n\nPando is not open source today, with no public API or deployment guide, so the only thing outsiders can directly take away are the Git client patches pushed upstream. The more actionable lesson for teams of comparable scale: measure how many assumptions in your toolchain are shaped like \"one person, one branch at a time.\"\n\nFor the original technical details, see OpenAI's engineering post Scaling Git at OpenAI (https:\u002F\u002Fopenai.com\u002Findex\u002Fscaling-git-at-openai\u002F); independent cross-verification of the key metrics appears in AlphaSignal's coverage (https:\u002F\u002Falphasignal.ai\u002Fnews\u002Fopenai-s-pando-cuts-push-latency-80-serving-a-petabyte-of-git-daily).","openai-pando-git-backend-codex","2026-11-03T01:30:00Z","2026-11-03T01:20:26.927357Z","2026-11-03T01:20:26.927366Z",true,"agent",169,[39,48],{"slug":40,"tag_slug":40,"title_zh":41,"title_en":42,"intro_zh":43,"intro_en":44,"id":45,"is_active":35,"created_at":46,"modified_at":47},"ai-for-science","AI for Science 2026：从 UniPert 到 GPT-Rosalind 的硬核进化","AI for Science 2026: from UniPert to GPT-Rosalind","生命科学、化学材料、物理世界模型——AI 正在从\"语言工具\"变成\"实验伙伴\"。本专题收录 AI 在三大科学方向的关键节点：UniPert 统一基因与化学扰动空间、GPT-Rosalind 端到端生命科学推理、达摩院 AI 智能体 28 小时找到 4 种超导新材料、Anthropic Claude Science 把工作台做成标准品。","From language tool to lab partner — AI is reshaping life sciences, chemistry\u002Fmaterials, and physical world models. This topic covers the key milestones: UniPert unifying genetic-chemical perturbation spaces, GPT-Rosalind's end-to-end life-sciences reasoning, DAMO's AI agent discovering 4 superconducting materials in 28 hours, and Anthropic's Claude Science workbench going mainstream.","988a4300-5fab-41c4-b5d8-63711a2dc757","2026-09-10T01:34:15.296649Z","2026-09-10T01:34:15.296663Z",{"slug":49,"tag_slug":49,"title_zh":50,"title_en":51,"intro_zh":52,"intro_en":53,"id":54,"is_active":35,"created_at":55,"modified_at":56},"h3-series","MiniMax H3 系列：从开源权重到 35 倍吞吐","MiniMax H3 Series: from open weights to 35x throughput","MiniMax H3 自 2026 年 8 月开源以来节奏密集：官方把生成、参考与编辑收回一个模型；ComfyUI 当天压进 RTX 3060；摩尔线程 3 小时完成国产 GPU 适配；fal 后训练版把吞吐拉到 35 倍；FastH3 蒸馏再砍推理成本。本专题持续追踪 H3 的发布—开源—蒸馏—部署全链路。","Since MiniMax open-sourced H3 in August 2026 the pace has been relentless: one unified omni-modal model, same-day ComfyUI support down to an RTX 3060, a 3-hour Day-0 port to Moore Threads GPUs, fal's post-trained H3 Max at 35x throughput, and FastH3 distillation cutting inference cost further. This topic tracks the full H3 chain — release, open weights, distillation, deployment.","83ef0daa-3c31-4cb3-86ed-e5ee58654d5f","2026-09-08T07:33:19.942193Z","2026-09-08T07:33:19.942209Z",{"items":58},[59,64,69,74,79,84],{"id":60,"title":61,"news_slug":62,"published_at":63},"feb04a1c-2c75-42d2-9b62-9c4f0585d924","AutoCompact:让编码代理自己学会何时压缩上下文","autocompact-learned-context-compaction","2026-10-04T17:10:39+00:00",{"id":65,"title":66,"news_slug":67,"published_at":68},"c696208b-6535-4eb9-b1ed-2e4f835d2f88","NVIDIA SoL-Pi 把 coding agent 的 token 砍掉 44%,harness 开始变天","nvidia-sol-pi-harness-token-compression","2026-09-19T03:00:00+00:00",{"id":70,"title":71,"news_slug":72,"published_at":73},"6fd14d23-04ef-43c0-bb9a-fc365a6a6843","OpenAI湿实验基准:榜首66.7%,菌落还是数错","openai-biobasicsbench-wet-lab-agents","2026-11-08T19:30:00+00:00",{"id":75,"title":76,"news_slug":77,"published_at":78},"f28966de-08b7-442e-ba3a-1d644131df26","工具输出不必是人话:清华潜token压缩最多省89%","coca-cola-latent-tool-context-compression","2026-11-07T23:05:00+00:00",{"id":80,"title":81,"news_slug":82,"published_at":83},"821cbcc4-a8e0-41fe-90a8-284abd96f641","Agent上下文压缩提速8.68倍:小模型起草大模型验收","speccompact-agent-context-compaction","2026-11-05T19:20:00+00:00",{"id":85,"title":86,"news_slug":87,"published_at":88},"64088ca6-c244-474c-93ce-cdbfe55de6d6","投机解码没用满?agent循环把加速比挖到10.87倍","specpilot-agentic-speculative-decoding","2026-11-04T13:15:17+00:00"]