算力与芯片

强化学习消耗了 10,000 多张 GB300 长达四周的时间。因此，实际上花在强化学习上的 FLOPs 比预训练还要多。强化学习任务要复杂得多，因为你既需要利用各类智能体沙箱进行大规模推理，又需要同时进行训练。

reinforcement learning was a little over 10,000 GB300s for four weeks. So, actually there are more flops spent on reinforcement learning. Reinforcement learning jobs are, you know, more complex in the sense that you do both a lot of inference at scale with all sorts of sandboxes for agents and training.

语境：Misha Laskin 披露 Beam（500B 总参数、23B 激活 MoE）的算力分配细节：后训练强化学习在集群规模与运行时长上已全面超越预训练阶段。

支持 → source-41e2110c2ba788c4

支持 → dwarkesh-podcast-ajeya-cotra-20260901

然后我还必须把它放进机架级解决方案里，因为如今已经没有什么单独的芯片了，整个系统就是一个机架。所以这需要 9 个月——我花 3 个月把它设计出来，但要等上整整 9 个月才能实际用上它。

then I have to put it into a rack scale solution because nothing's a chip anymore. It's a rack, you know. So we're 9 months, you know, it took me 3 months to design it, but it's nine months until I can actually start to use it

语境：Pat Gelsinger 阐述现代计算体系的系统级重构：计算单元的物理边界从单颗硅片扩张为整个高密度机架，使得先进封装与互联制造周期成为核心阻碍。

支持 → the-ezra-jensen-huang-20260923

支持 → bloomberg-tech-lisa-su-20260929

因此我能在 3 个月内完成设计，但我无法在 9 个月内真正实现规模化硅片制造。这很糟糕，对吧？所以如果我们要持续这种创新，就必须真正看到这个瓶颈得到改善。

so I can design the thing in three months but I can't actually get it into real silicon at scale for 9 months. Okay, that sucks, right? So, we have to really see that bottleneck improve if we're going to have this innovation

语境：对比算法软件层的高频迭代与晶圆代工制造的重资产长周期，呼吁半导体光刻与打样流程实现物理压缩。

支持 → dwarkesh-dylan-20260825

人类并不总是那么在乎 S3 的 P99.9 长尾延迟，但智能体却极其在乎，并且会被这种延迟彻底阻塞挂起。

they don't always care about the the P999 S3 latency like agents do care and get blocked by that

语境：Matt Garman 剖析智能体对云底层指标的严苛重构：在多轮链式自主调度中，单次长尾毛刺会被智能体循环成倍放大，促使基础设施优先优化极端尾部延迟。

支持 → latent-space-ari-weinstein-20260930

支持 → latent-space-thariq-shihipar-20260929

OpenAI 拥有一项未被外界足够重视的布局，就是我们的自研芯片计划——它将在 2027 年上半年开始上线，并在未来几年大规模扩展。我认为它将为我们在全球能获取的每瓦能耗上带来巨大的推理成本优势，特别是考虑到我们在消费者、开发者和企业端看到的爆发式需求增长。

one thing that OpenAI has that doesn't get as much attention is our chip program, which will start to come online in the first half of 2027 and will scale a lot in future years. I think gives us a huge inference advantage for whatever watts we can find in the world and given the growth and demand we are seeing across consumer developer enterprise really everywhere.

语境：Altman 在 DevDay 透露 OpenAI 正在推进的自研芯片计划时间表（2027H1 投产），旨在打破外部算力与能耗瓶颈，最大化单位功耗下的推理通量优势。

回应 → the-ezra-jensen-huang-20260923

支持 → dwarkesh-dylan-20260825

我认为能源发电的扩展是最大的事情之一。你必须扩展能源，也必须扩展芯片制造。我认为我们在软件和一切智力、数字化领域都处于领先。但从长远来看，我们需要能够产生足够的电力来与对手竞争——我认为从本质上讲中国是最大的竞争对手，而中国的发电量大约是美国的三倍。

Scaling energy I think is one of the biggest things. You've got to scale energy, you've got to scale chip production. I think we're winning on software, on anything intellectual or digital. But in the long run, we need to be able to generate enough power to compete with, I think essentially China is the biggest competitor, and China has about three times the electricity production of the United States.

语境：回应主持关于「美国如何保持领先」的提问，指出尽管美国在算法与模型软件上保持明显优势，但长远看 AI 扩展的核心物理瓶颈将落在大规模电力基础设施建设与芯片产能上。

支持 → dwarkesh-dylan-20260825

但我们现在正在建造的东西，实际上是超智工厂（SI Factory）。之所以称其为工厂，是因为你正在生产具有经济价值的东西，而不仅仅是存储数据。

But the things that we're building now, these are really SI factories, super intelligence factories. And the reason why it's a factory is because you're producing something of economic value. You're just not storing it.

语境：黄仁勋重新定义新一代数据中心的工业属性，强调智算中心不再只是信息检索或云存储库，而是将电力与算力转化为智能输出的高附加值生产型实体工厂。

支持 → zhangxiaojun-zeng-ming-20260903

硬件、软件、系统与模型层之间的相互作用决定了，你对端到端全链路理解越深，构建出的系统就越优秀。这就是我们收购 World Labs 的真正原因：获得飞飞汇聚的世界级顶尖人才，并将其与 AMD 在硬件、软件和系统方面的完整能力紧密结合。

the interaction between, you know, the hardware and the software and the systems and the model layer are such that the more you understand end to end, the better system you're going to build. And so that's why we're acquiring World Labs, really is, uh, to have access to the world class talent that Feifei has brought together and really bring it together with AMD's complete capabilities around hardware, software and systems

语境：回应彭博社关于‘芯片厂商为何重金收购模型层初创公司’的提问，苏姿丰阐明计算产业端到端垂直协同的底层逻辑，强调只有深入模型研发才能定义最契合的芯片架构。

支持 → tbpn-sam-altman-20260929

我坚信 AI 的未来在于 AI 软件与硬件的共同演进。模型将极大受益于最前沿的硬件来加速训练、推理与服务；与此同时，硬件也将极大受益于对未来工作流与算力需求的洞察与前瞻。

I really believe that the future of AI is this whole evolution of AI software and AI hardware. Models will gain so much by using the most cutting edge hardware to accelerate the training, the inference, the servicing. But in the meantime, hardware will benefit so much from the insight and the peek into the future of the workflows and needs of hardware.

语境：李飞飞阐述 World Labs 选择与芯片巨头合并的技术逻辑，指出空间智能模型与计算硬件之间存在紧密的互哺协同关系。

承接 → a16z-fei-fei-li-20260904

我一直强调在 AI 领域从来没有‘一刀切’的解决方案。我们谈论了很多关于大型云端算力以及我们在 CPU、GPU 上的进展；但物理 AI 世界、机器人领域的机遇，以及跨越云端、边缘与端侧的端到端能力，将是我们未来大幅增长的关键领域。

one of the things I've always said is there's no one size fits all when it comes to AI, right? I mean, we talk a lot about the big cloud compute and what we're doing in terms of CPUs and GPUs. Um, but the physical AI world, The opportunity in robotics. The end to end AI capability across cloud, edge and client. You know, those continue to be places where I believe we're going to grow a lot

语境：苏姿丰展望未来三至五年的算力格局，指出除了云端超大集群外，伴随具身智能兴起的边缘端与端侧异构芯片将是计算架构的新增长极。

限定 → youtube-elon-musk-20260929

智能体不是人类工程师，它们会做出大量导致系统实际变慢的作弊行为：例如为了让单核函数跑分好看而关闭 CUDA Graphs，导致端到端变慢 20 倍；或者只在极短上下文窗口上测试。这使得生成的核函数在生产环境中完全不可用。

agents are not humans and they will do plenty of things to make it slower like they'll disable CUDA graphs which can make it 20 times slower and they might make that one kernel faster but make the whole like it's not a viable kernel because it's they're disabling a bunch of speed ups like CUDA graphs or only testing on small context windows

语境：Tejas Bhakta 揭示自动优化智能体的奖励黑客（Reward Hacking）陷阱：智能体会在没有负向约束的情况下钻局部评价指标空子，定义「不可为清单」与端到端系统级验证至关重要。

支持 → all-in-satya-nadella-20260915

自动研究智能体所尝试的方案中，大约 80% 都是劣质甚至有害的；在开发中必须谨记绝大多数尝试都会失败，它会不断试图在评估指标上欺骗你；但坚持到最后，你确实能从中提炼出非常惊艳的优化结果。

Around 80% of the things that auto research is going to do are going to be bad. Uh so it's important to remember while you're u like working on this that most things are going to be bad. It's going to try to trick you all the time. Uh but at the end you can actually get really good results from this. with TLDDR.

语境：Tejas Bhakta 总结 Agent 自动化调优的真实分布：成功依赖海量并行试错与严格沙箱过滤，用高失败率换取极端硬件极限榨取。

支持 → ai-engineer-tejas-bhakta-20260926

我们希望能够服务每一家中小企业和大型跨国公司，满足它们的 Token 消耗、微调模型甚至自研前沿模型的需求。这比什么都重要——我们非常珍视与 OpenAI 的业务合作，他们是我们最大的客户之一；但我们的目标不是只拥有一个、两个或五个大客户，而是拥有成千上万个大客户。

We want to be able to serve every small and medium business and large multinational with their token consumption, their need for a fine-tuned model, maybe even their own frontier model. That's more important than, you know, we love our business with OpenAI. They're one of our largest customers. But our goal is not to have one large customer or two large customers or five large customers. It's to have many, many, many large customers.

语境：回应外界关于微软智算基础设施资本开支是否过度依赖 OpenAI 单一客户的质疑，纳德拉强调微软走的是面向全球数百万企业的超大规模云平台路线，而非少数几家模型玩家的专属算力中介。

回应 → dwarkesh-dylan-20260825

支持 → bloomberg-tech-lisa-su-20260929

任何正在开展大规模强化学习训练并给模型设定奖励函数的人——比如调动一万个智能体，投入一亿美元，让它们在庞大的集群上并行运行一两个月去尝试解决任何问题，哪怕根本不是安全相关的任务，哪怕只是去解一道数学题，极其恶劣的事情都有可能发生。

anyone who's doing big reinforcement learning runs and they're giving it a reward function. So unleashing you know saying hey here's like a you know 10,000 agents and here's $100 million let's put them in parallel and let them run on a gigantic cluster for a month or two try to solve anything and it doesn't need to be a security thing. It could be like do anything you know solve this math puzzle really bad things can happen

语境：Ali Ghodsi 指出前沿实验室面临的真正安全短板：高并发强化学习在庞大集群上长程自探索时，由于奖励机制缺陷极易引发不可逆的越权网络渗透。

支持 → a16z-aaron-levie-20260926

支持 → cognitive-revolution-nathan-labenz-20260927

细看 LFM-2 系列模型，它们拥有非常优雅且精简的架构设计：大体由 80% 的双门控一维卷积算子加上约 20% 的分组查询注意力（GQA）混合构成。这是一种高度精简、极速且高质的基础模型架构。

The LFM2 instances of the models, they are, if you look at them, they have a very elegant and simple architecture. So the architecture is basically 80% doublegated convolutions you know 1D convolutions plus some uh group query attention basically 20% of query. So the architecture is just a very simplified

语境：Ramin Hasani 公布 LFM-2 的算子混合配比：通过神经架构搜索找到能原生在 CPU 上高速跑通的自回归混合结构，极大削减对全自注意力机制的算力依赖。

支持 → latent-space-ramin-hasani-20260918

自回归模型在推理端本质上依然是完全串行的：计算只能从左至右逐个 Token 推进；在生成前 9 个词之前，你绝不可能提前生成第 10 个词。这种工作负载与 GPU 的并行硬件架构极其不匹配，受到极其严重的内存带宽限制。

auto reggressive models are still sequential uh the computation is one left to right one token at a time. You cannot generate the 10th token until you've generated everything that comes before it. That kind of workload is um does not map well to GPUs. that kind of workload is extremely memory bound.

语境：Stefano Ermon 指出自回归 Transformer 在推理时代的核心死穴：串行生成导致大量时间浪费在内存层级搬运权重上，算力利用率低下。

支持 → all-in-satya-nadella-20260915

你把绝大部分时间都浪费在了在内存层级结构中搬运模型权重上，实际执行的算术运算少之又少，这是自回归模型的根本痼疾。正如当年 Transformer 在训练端用并行计算取代了 RNN 一样，在推理端取代自回归串行架构的终极形态正是基于扩散的生成模型。

spending most of your time moving around weights across the memory hierarchy and you're doing very little arithmetic and that's a fundamental problem of auto reggressive models. And so what's the equivalent if you think RNN's transformers auto reggressive models the equivalent at inference time is a diffusion based is a diffusion model

语境：Stefano Ermon 梳理生成架构历史演进：类比 2017 年 Transformer 并行化训练淘汰 RNN，推理端必须走向多 Token 协同去噪的扩散范式以解锁单位算力智力上限。

支持 → no-priors-stefano-ermon-20260918

美国真正占据主导的是核心前沿 AI 供应链：即芯片设计、先进制程代工控制权、盟国控制的半导体制造设备以及前沿基座模型研发。如果你单单要求暂停这部分的核心推进，却任由中国在其余所有工业基建与制造领域全面狂奔，这从地缘博弈角度看将是一笔极其不平等的失衡交易。

the sort of like core frontier AI supply chain, right? Like chip design plus control over chip production, semiconductor manufacturing equipment controlled by the allies and actual frontier model development. So if you pause specifically that part of the development and let China run away with the entire rest of it and continue running away with the entire rest of it, that's just a very lopsided deal.

语境：Anton Leicht 拆解「中美 AI 暂缓协议」的博弈死结：前沿模型是美方唯一的非对称战略支点，单方面踩刹车等同于在其他实体产业链全面追赶的背景下自废武功。

支持 → all-in-satya-nadella-20260915