Modal CEO Erik Bernhardsson 解析 AI 算力为何不适合矩形容量合同

热烈欢迎 Basil Chatha 加入我们，成为最新的联合主持人！他来自 AI 工程领域的前沿部署方向。我们将和他一起推出一档新节目，也正好借 Modal 完成重磅 D 轮融资之际，采访了 Modal CEO Eric！

两年前，我们与 Erik Bernhardsson 聊过他对真正无服务器 AI 基础设施的愿景：开发者应当能够运行代码，而无需配置集群、预测负载，也无需为闲置资源付费。之后，CTO Akshat也加入我们，聊了 Modal 的基础设施原语，以及提供这些原语和智能体体验的技术栈。

过去 5 个月发生了很多变化。自完成 50 亿美元 C 轮融资以来，Modal 现在称其年化营收运行率已超过10 亿美元，并以160 亿美元估值完成 7.5 亿美元 D 轮融资，已启动近 30 亿个沙盒。在这次对话中，Erik 与 Vibhu 一起解释了推动增长的基础设施和经济逻辑，以及为什么算力需求的形态往往与购买算力所用的合同并不匹配。

在 YouTube 上观看完整对话 →

GPU 需求不是矩形

典型的容量合同会向客户提供固定数量的 GPU，使用期限也固定。将容量绘制为随时间变化的图表，就会得到一个矩形。真实工作负载则不同：它们会在一天中不断增长、激增和回落。按峰值需求购买足够的容量，就得为闲置硬件付费；购买得少一些，又可能无法满足需求。

Erik 将 Modal 的押注描述为一种期限转换：汇集数千家客户的需求，在底层锁定容量，再让每位客户按需使用。汇总需求越平稳，这个容量池就越有用。这也能很好地补充我们与 SF Compute 的 Evan Conrad 的对话：Evan 着重于让容量可以交易；Erik 则着重于让容量能够以更小、更灵活的份额供人使用。

另一方面也存在财务风险。如果服务商按当前价格锁定容量，而 GPU 价格之后下跌，就可能被昂贵的供给套牢。Erik 表示，他不认为这种情况会很快发生，但在构建业务时，这种可能性值得考虑。跳转至 GPU 价格讨论 →

险些被砍掉的沙盒

Erik 和联合创始人 Akshat Bubna 注意到，语言模型能够生成代码，而且总得有地方运行这些代码，于是 Modal 推出了沙盒。早期采用情况不温不火，团队一度讨论是否要关闭这项产品。随后，编程智能体、后台智能体和强化学习带来了新的需求。Erik 表示，沙盒业务已增长至 Modal 营收的约三分之一，这一结果是他在产品发布时没有预料到的。

这种增长也改变了基础设施必须具备的能力。智能体可能会创建环境、运行代码、检查结果，然后多次重试。启动和恢复这些环境的延迟，成了体验的一部分。在我们最近对 Akshat 的采访中，我们探讨了从开发者体验转向智能体体验的变化；这次 Erik 则补充了产品和商业层面的故事。跳转至沙盒讨论 →

快速启动是一项基础设施功能

只有容量能足够快速地到位，弹性才真正有意义。Modal 的无服务器 GPU 技术文章介绍了其背后的机制：可用 GPU 缓冲池、自定义文件系统、进程检查点和 GPU 内存快照。这些组件减少了启动另一个模型副本所需的工作。这是一个具体案例，展示了基础设施决策如何影响应用的响应速度和成本。

我们还讨论了 Erik 从 Spotify 推荐系统转向 Modal 的历程、公司如何采购算力、NVIDIA 的替代方案、驻场部署工程师，以及为什么 Modal 不强制进行代码审查。跳转至工程文化 →

节目注释

Modal——Erik 创立的 AI 基础设施平台。
Erik 的文章——他关于基础设施、市场和创业的文章。
Modal 的 D 轮融资公告——公司近期公布的增长情况和沙箱里程碑。
Modal 如何实现真正的无服务器 GPU——弹性推理背后的初创实践和扩展技术。
Modal 的资源求解器——它如何根据不断变化的价格和可用情况选择算力。
扩展至一百万个并发沙箱——关于沙箱扩展的工程实践记录。
我们对 Erik 的首次采访——最初的无服务器基础设施构想。
我们对 Akshat Bubna 的采访——面向智能体的基础设施和开发者体验。
可编程数据中心——Evan Conrad 谈 GPU 交易和实物交割。

时间戳

0:00 Modal 的业务
1:25 从 Spotify 到创立 Modal
4:19 在生成式 AI 热潮前创业
7:43 首批客户与融资
10:00 为推理和自动扩缩容而构建
13:21 Modal 从何处获取 GPU 算力
15:39 租用还是自建基础设施
19:45 GPU 价格下跌的风险
24:18 AI 需求是无限的吗？
26:12 NVIDIA 的替代方案
28:25 Modal 能从新云厂商身上学到什么
30:26 GPU 作为一种资产类别
31:55 Modal 的产品：函数和沙箱
36:03 小型训练任务的优势
41:21 智能体沙箱的变化
46:48 Anthropic 与沙箱安全
49:29 使用智能体测试安全性
52:51 智能体时代的开发者体验
57:45 Modal 的驻场部署工程师负责什么
59:55 Modal 的产品管理方式
1:03:29 可选的代码审查与工程文化
1:05:56 云端、本地部署和自带云
1:09:25 算力、融资与增长
1:13:07 扩大团队规模
1:17:28 Erik 的推文：临时沙箱
1:18:31 矩形 GPU 合约的问题
1:21:20 推理服务商与模型开发商
1:23:25 模型接下来将在哪些方面改进

文字记录

根据修订后的剪辑自动转录；姓名和措辞可能有误。

0:00 感谢你参加节目，来自 Modal 的 Eric Fern-Hartson。你能介绍一下这家公司，以及你们是如何走到今天的吗？当然。最简单的说法是，我们在构建 AI 基础设施。我们提供计算平台和存储平台，让 Dacagon、Suno、Runway、Cognition 以及许多其他客户能够构建 AI 应用。实际来说，这意味着大量推理、大量沙盒，有时还包括训练和微调，有时是批处理任务以及其他工作。所以，我们基本上构建了一套软件 SDK，让客户能够非常快速地完成构建。我们管理底层的所有容量，让这些东西更容易部署、扩展，并在生产环境中运行。你大概两年前参加过 Light and Space 的节目，从那以后发生了很多变化。当时你们有多大？可以分享哪些数据？相比之下，现在有多大了？我其实不记得当时的情况了，但可能至少增长了 50 倍，营收和用户数都是。嗯。我其实用它做了一个自己的副项目。不错。对，就是把 2D 图像转换成 3D 模型。对。所以，我只是在这里托管了几个模型。你觉得怎么样？有什么反馈吗？我的意思是，我很喜欢。不错。很好。我的意思是，对我来说，这就是为什么我会选择它，而不是任何竞争产品之类的。很好。那么，在创办 Modal 之前，你曾在 Spotify 工作过，也在 better.com 工作过，对吧？

1:30 没错。对。你在 Spotify 做什么？我在 Spotify 做过很多不同的事情，但我主要的贡献是构建了音乐推荐系统。我是在 2008 年、可能是 2009 年从学校毕业后加入的，在 Spotify 工作了七年。那段时间的大部分工作，基本上都是从零开始构建音乐推荐系统，组建了一个小团队，并推出了很多功能，比如相关艺人、Discover Weekly 以及其他许多功能。所以其中有大量矩阵分解，以及大量计算密集型工作。可以说，后来成为 Modal 起源的很多东西，就是在构建 AI 应用的过程中意识到：基础设施实在是太烂了。肯定有更好的方式来做这件事。所以，从某种程度上说，创办 Modal 的理由，就是构建一个我在 Spotify 工作期间一直想拥有的工具。对。所以，Modal 更多是你从 Spotify 学到的东西发展出来的，而不是来自 Better.com 的经历。Better.com 算是一次短暂的旁支经历。那是一段过山车般的经历，也是关于那家公司的一个很长的故事。我们之后再单独聊。但我曾担任 Better.com 五年的 CTO，并把技术团队从两三个人发展到四百人左右。所以，我学到了很多关于管理、组建团队和招聘的知识，也学到了很多关于开发者生产力的东西。我思考了很多工具之类的事情，也做了更多全栈方面的工作。但后来，我又回到了自己真正热爱的领域，也就是基础设施、AI 和机器学习。Better.com 当时是做什么的来着？抵押贷款。好的。对。你是怎么进入这个领域的？

3:00 我的意思是，我们没必要在这上面花太多时间。对，没事。就像这个人，Zach Frankel，我当时并不认识他，后来他招我加入 Better 担任 CTO，当时公司有10个人。你知道，Vishal 这位创始人，不知为何，是个很有意思的人，非常非常聪明，也有点疯狂，不过大多是积极的那种。我们相处得非常好。所以，我们在2015年开始着手做这件事，我花了五年时间发展那家公司。后来你知道，事实证明做抵押贷款挺难的，疫情之后利率又迅速上升，给公司造成了很多问题。不过那时候我已经离开了。你离开是在2020年、2021年左右？对，差不多吧。好，所以你是在有了模型的想法之前离开的，还是离开之后才开始做？很难说。我离开 Better 时脑子里有很多零散的想法，而且我其实不确定自己是否想创办公司。那时候我基本上就是，你知道，疫情期间，我几乎算是休了个小长假，家里有两个孩子。我是说，我现在还是有孩子，但我的意思是，当时他们大概分别两岁和四岁，所以我决定陪陪他们，休息一阵子，同时也做些兼职，试着折腾一些东西。我不确定，也许应该做些咨询，然后我开始做些东西。我对这个想法特别兴奋，它最终成了 model。但实际上两者都有，你知道，model 应该是我做的第30个项目，我当时就……之前还有几次尝试，都不了了之。

4:30 不过我的想法是，第一个点子让我觉得挺酷的，像是我应该做这个。这可能会成为一个很棒的平台。其他点子也类似，都是开发者工具吗？最初的想法是做某种网页抓取工具，但后来不了了之。不过现在回想起来，也许它本来能做成。那是在 ChatGPT 发布之前吗？是的，是的。2021 年。哦，好的。所以 ChatGPT 是 2022 年底发布的，对吧？之后我就开始着手构建了。也许先稍微回顾一下背景：我在 Spotify 工作时，发布过一个名为 Kaluigi 的开源框架，它用于工作流编排。它和 Airflow、Prefect 之类的工具属于同一类别，不过这些类别现在某种程度上已经有些过时了，人们不太常提起。但有一段时间，它确实获得了很多关注。总之，我最终没有把它做成一门生意。不过几年后，也就是 2021 年，我在考虑商业点子或要做的事情时，又想到了工作流编排。我开始尝试各种想法，比如在云端运行代码，也就是基本上对云端运行的代码进行编排。我意识到，归根结底，核心难题不只是编排计算机，而是执行计算。于是我觉得，这实际上是个更大的待解决问题。从某种意义上说，我把它暂时搁置了，转而去做另一件事，想着以后再回来处理工作流调度。但总之，我再也没回去做了。长话短说，最后我创立了 Modal，重点是想办法让人们能轻松地在云端计算机上运行代码。起初，目标相当通用。

6:00 实际上，从某种意义上说，我们最近又兜了一圈，但我们曾专注于许多不同的应用。我们的想法大概是：让大家能非常轻松地在云端运行代码。可以是 GPU 代码、AI、机器学习，也可以是数据相关的东西。我们花了很多精力思考如何扩展规模、如何打造出色的开发者体验。我们很早就意识到，要做到这一点，就必须深入技术栈，于是我们构建了文件系统和容器运行时，也解决了许多棘手的工程难题。你提到了 ChatGPT，但它其实还不算真正的杀手级应用。不过，如果你还记得，那年秋天差不多同一时间还推出了另一款产品：Stable Diffusion。后来证明，Stable Diffusion 成了我们当时正在打造的产品的杀手级应用。那时我们只有四五个人。于是我们开始看到很多人来找我们，想要运行 Stable Diffusion 模型。我们由此意识到，生成式 AI 推理或许确实是一个很适合我们的应用场景。你是怎么找到用户的？我就是个发推上瘾的人。以前比现在还严重，现在我忙着创业，容易分心，但我一直发很多推文，也写了很多博客。所以我觉得自己有一点关注者。早期推动增长的，很大一部分就是我不停地发推谈模型和我正在做的东西，让大家兴奋起来，还写很长的博客文章，聊云计算的发展方向之类的。我把这叫作创始人主导的营销。我总觉得，你得发挥自己的长处

7:30，而我的长处大概就是发些胡言乱语，长期泡在网上。所以，你知道，我们现在仍然在这方面做得不错。我觉得我们的增长很大程度上是因为我们擅长社交媒体，也经常在线。不错。你们多久才有了第一个客户？花了挺长时间。另外，你们早期就融资了吗？我们过了好一阵子才融资。实际上，我有一段时间都是自掏腰包，甚至一直到招来最初几名员工之后也是如此。从某种意义上说，这挺好的，因为这让我对钱特别节省，也会关注投资回报。我很长时间都没给自己发工资。后来有一度这成了问题，因为我需要钱，但我们还是过了好一阵子才融资。直到 2022 年初，我们才完成种子轮融资。那时我已经为这个项目工作了大约一年。大约五年前的现在，也就是 2021 年 8 月左右，Akshah 加入，担任 CTO 和联合创始人。但直到 2022 年底，我们才开始获得可观的收入。我想这就是我为你感到骄傲的原因。是啊。对我们来说，Stable Diffusion 是主要推动因素。我们的月收入从 2022 年 11 月左右的 100 美元，到了次月就达到了 2,000 美元。然后到了 2023 年 1 月，突然就达到 30,000 美元。那时我终于觉得胸口压着的那块大石头落了地。我心想，也许我们真的找对方向了。感觉我们确实找对方向了。之后我们在 2023 年增长得非常快。

9:00 从那之后，我们基本上一直在快速增长。那个月收入达到 30 K 的时候，是因为有几个大客户，还是客户群很多元？只有一个。我不确定自己有没有提过那是谁，也不知道他们是否知道这件事，不过我觉得他们应该猜到了。那是一家位于纽约、名叫 Mage 的公司。他们做的是 Stable Diffusion 相关业务。所以，在 2023 年第一季度左右，我想他们大概占了我们 80%、90% 的收入。但之后，我们的收入来源就开始变得更多元了一些。到了 2023 年底，我们很快又迎来了一家大客户。他们有一段时间也占了我们很大一部分收入。不过随着时间推移，我们现在的收入来源确实已经非常多元化了。是的。说到 Stable Diffusion，我记得自己用它来跑过 ComfyUI。我猜很多人都在用 ComfyUI，比如在上面运行 ComfyUI 工作流。有些人是这样，但我觉得使用这些模型的人通常会做更底层的事情，比如复杂的操作。人们喜欢这些模型的一点是，它们一直都很容易折腾。他们会部署自己的定制工作流，使用适配器、自定义推理和大量优化。我们为让开发者体验变得更好而做的所有工作，结果也非常适合运行生产负载。因为事实证明，自动扩缩容实际上是个非常类似的问题：你希望容器能够非常快速地扩容和缩容。所以在早期，我们恰好非常适合承接这类工作负载。那时候没有自动扩缩容工具吗？为什么你们得自己开发这些工具？

10:30 总的来说，我的论点是，每一个新时代的技术都会遇到工具的局限性。回顾一下，无论是 Web、云、移动设备、PC，还是 AI，都是如此。我认为，工具往往是存在的，但通常并不是特别好用。为了从中获益，真正加速发展并开始为这个新世界构建产品，我们往往需要重新思考很多工具。云计算时代就是如此，我们必须发明 Kubernetes、Docker 之类的东西。类似地，对于生成式 AI，你可以基于上一代工具构建这些东西。比如，你可以在 Kubernetes 和 Docker 之上构建，但这些技术是为完全不同类型的工作负载打造的。它们面向的是非常稳定的后端工作负载，请求数量也很稳定；运行时利用率相当低，因为 CPU 通常很便宜。所以人们并不太担心成本。但现在我们正在进入一个全新的世界，你需要管理昂贵得多的 GPU 的容量和利用率。你需要通过网络将非常大的模型加载到 CPU 和 GPU 中。结果发现，很多这类工具根本不是为此打造的。因此我们觉得必须重建其中很多东西。我提到过文件系统，但过去几年我们还花了很多时间，围绕 CPU 构建了一些非常酷的东西。

12:00 快照、GPU 快照和审计，我们基于自动扩缩容器和调度器等各种容量管理工具进行了构建。我们觉得必须这么做，因为你不能在旧软件之上，为新一代软件构建新功能；你不能在旧东西上面套一层，就指望实现新东西。某种程度上，你得从头开始。所以说到自动扩缩容，你们在平台上有没有遇到什么奇怪的行为？扩容时有没有什么东西出问题？我的意思是，容量一直以来时不时都会成为挑战。你们如何获得数百、数千乃至数万块 GPU？这始终很有挑战性。我们很早就不得不采用多区域架构。回过头来看，我觉得那是个影响深远的决定：我们当时下定决心，咬牙把它建了起来。我们一直把容量管理看作一个全球统一的资源池，而不是各个区域分别运行的独立集群。因此，从一开始就以这种方式来思考，是我们做出的一个影响重大的决定。我不确定具体有哪些问题。显然，我们遇到过 bug，也发生过宕机。我多么希望模型能做到 100% 正常运行，但很遗憾，最近我们基本达到了四个或五个九的可用性。不过，在 Model 的发展历程中，我们当然也曾遇到过一些扩容问题。是啊，GPU 市场——不过在说这个之前，你们底层用的是 AWS、GCP 以及所有这些云服务提供商吗？

13:30 比如你们也会购买基础设施吗？不，我们来聊聊云容量吧。我觉得这方面其实变化很大。最初几年，我们几乎所有容量都来自超大规模云服务商提供的现货和按需容量，主要是现货容量，分布在各个不同的区域。最初我们用 AWS，也用 GCP，后来还用了其他服务商。我们很早就开始从 Oracle 租用一些容量，因为我们想要八张 A10 和八张 A100，而他们给出的价格非常优惠。后来我们又增加了几家云服务商，比如签订预留实例协议。但直到去年年底，我们的大部分容量仍然是现货容量或一年期预留容量。现货的意思就是你当下租用。没错。对。而且现货容量也可以被抢占，我们必须处理这种情况。不过，这种模式在 Agria 还算行得通，因为你可以从许多地区（基本上是数百个）汇总容量。所以，这种模式一直运作得很好，直到我们再次开始快速扩张。我的意思是，首先，Modal 开始飞速增长，我们需要更多容量；但从根本上说，云市场也发生了很大变化。所以过去一年，为了获得所需容量，我们不得不与更多云服务商合作。目前我们合作的云服务商将近 25 家。很多时候，为了获得想要的容量，我们不得不签订三年以上的合约，而且要预付一大笔钱。你们会锁定三年期的价格，对吧？

15:00 所以，过去六到九个月里，我们不得不花大量时间重新思考如何获取容量。我们得建立一整个采购部门，基本上有四个人全职负责采购容量、联系 Neoclouds、洽谈交易、审核容量。我们还得建立一整个工程团队，基本上有 10 个人专门管理全球各地的裸金属容量，处理网络问题、故障硬盘、GPU 从总线上掉线等各种问题。到目前为止，我们不拥有任何容量，所有容量都是租来的，通常签的是三年期合约。这是一个重大决定：我们应该购买，还是继续租用？其实我并不特别固执于某一种做法。我觉得租用挺好的，因为它有一定的资本效率。如果你购买，首先就会面临这些运营上的复杂问题：你得与供应商打交道，还要处理电力、电缆以及各种各样的事情。但它在财务上也很复杂：你要怎么筹资？要不要融资，然后拿 10 亿美元去买硬件？这是一种会造成很大股权稀释的做法。也可以借债，但如果借债，通常就需要锁定长期收入来支付利息。这也不是件简单的事。不过，我们会研究经济效益，也会研究运营方面的……

16:30 复杂性，我们来看看其中的风险。嗯，这是一个复杂的权衡，我们也愿意购买。我们还会考虑其他一些结构。我们这么做是为了获得 2027 年所需的算力。这需要做很多工作，探索各种不同的途径，才能把算力落实到位。那么，你们是不是得雇很多财务人员来处理这些事情？说真的，我在这件事上花了很多时间，我们也确实投入了很多时间。我认为随着时间推移，毕竟这些硬件资产非常昂贵。而且，我觉得这规模着实惊人。你已经开始看到，AI 投资占美国 GDP 的比例达到几个百分点；过去几周，美国的利率也开始飙升。就现阶段而言，你看看 Neoclouds 就会发现，这些公司在金融工程方面非常精明。所以，我们会研究这些问题，也会考虑所有选项；我们自己可能也不得不采取类似做法。我不太希望这样；我更愿意专注于做一家非常精干的软件公司，把我们擅长的事情做好。总的来说，我不认为软件公司持有硬资产是件好事。这就像要不要拥有自己的办公室一样，我觉得这未必是大多数公司的正确选择。当然，为了获得你想要的算力，一切选项都可以考虑。是啊，而且我猜算力供应还会受限一段时间。

18:00 所以你多少得这么做。你同意这一点吗？还是觉得这只是某个阶段的现象？我倒不一定同意。我的意思是，Nvidia 每个月都在交付几十万块 Blackwell 芯片，对吧？我不想说市场是 100% 有效的，但一般来说，只要你愿意出高价，就能买到 GPU 算力，对吧？如果你愿意比其他人出更高的价，当然，Anthropic 和 OpenAI 也在尽可能多地采购。但从根本上说，市场上一直有新的供给。所以我不一定认为这是一种资源被完全垄断的局面。价格确实在上涨，我觉得值得对此反思一下。硬件通常会像受重力作用一样——最终，硬件价格总会下降。所以现在我们多少处于一个有点反常的时期，硬件价格竟然在上涨。你知道，很难说这种情况会持续多久，可能再持续六个月，也可能再持续36个月。很难确切知道会持续多久。但从根本上说，我不确定说“你需要买下产能”是不是正确。这在我看来有点像疫情期间那场严重的厕纸短缺，或者类似的情况，对吧？至少有点像挤兑银行那种情形。但话虽如此，产能紧张的局面还会持续一段时间，价格也在上涨。幸运的是，我觉得这对我们来说未必是坏事。价格上涨也有好处，因为稀缺也意味着我们可以

19:30 向我们的客户收取更高费用。而且，你知道，我们有很多预订都锁定了价格，这意味着在某种程度上，我们实际上已经相当于做多了 GPU 价格。所以我不一定太担心价格上涨。我更担心的是价格真的崩盘。我觉得短期内不会发生这种情况。但我认为这对市场上的很多参与者来说会是件相当糟糕的事。为什么会很糟糕？我觉得对那些锁定了大量供应的人来说会很糟糕。是啊。因为他们付了太多钱，你知道，所以存在高点买入的风险，对吧？再说一次，我不太担心这一点。我觉得这不会在短期内发生。但就像我说的，从非常长远来看，硬件价格往往会下降。所以，值得对此进行反思，对吧？对。就像现在，我觉得你的算力容量受到限制，而这在经济学上会带来一种稀缺租金，对吧？谁获得了这笔稀缺租金？我认为主要流向两个方面：大部分归 Nvidia，但当然，Neocloud 在一定程度上也能分得一杯羹。总的来说，任何拥有算力容量的人都能获得更好的经济效益。如今算力容量受限，这一点无可否认。从长远来看，我觉得这未必完全成立，对吧？拥有 GPU 总能带来一点溢价，但这值得冒险吗？你知道，把这些非常昂贵的

21:00 资产放在资产负债表上，值得冒这个风险吗？我不太确定，不过我们拭目以待。这件事还需要几年时间才能尘埃落定。从长远来看，我们可能会拥有一部分，租用一部分，也可能两者结合，或许还会做些别的疯狂事情，比如合资企业，谁知道呢？也可能搞一些复杂的 SPD，诸如此类。嗯，一切皆有可能。你们如何与 OpenAI 和 Anthropic 竞争，抢购 GPU 算力？我猜他们会想把所有能买到的都买下来。他们确实在这么做。我觉得你看到的是，嗯，很多价格上涨都是由此导致的。话虽如此，NVIDIA 出货量也很大。我觉得在某种程度上，NVIDIA 也更倾向于适度分散供货。他们不希望客户集中在少数几个大买家身上，就像你看到超大规模云服务商获得的配额也受到了一些限制。我觉得 NVIDIA 也希望看到更多 Neocloud 和其他替代云服务商获得更多配额。我认为我们在市场上看到的某些情况也说明，小型云服务商与其合作的风险稍高一些，反而给我们带来了更多机会。当然这只是推测。我的感觉是，也许 NVIDIA——抱歉，Anthropic 和 OpenAI——更倾向于从那些成熟可靠的供应商那里采购。其中一部分原因是他们必须进行大量训练，所以需要非常大的集群。不过，市场上还有很多算力容量，你知道，在亚美尼亚就能租到 32 个节点。

22:30 你可以在印度尼西亚获得128个节点。你知道，我们就喜欢这种东西。这样我们就能去世界各地获取大量容量较小的资源，价格也相当不错。是不是只要有人能提供一定数量的 GPU，你就什么都不在乎了？你不在意它有多可靠吗？会不会只是某个人把设备放在某个破旧仓库里？有时确实是个破旧仓库。感觉就是这样，但我们基本上确实有一份规格表。和供应商沟通时，我们做的第一件事就是把规格表发给他们。我们会说，我们需要达到这个 IOPS 数值。CPU 至少要达到某个要求，得有这么高的 PassMark 分数之类的。我们还要了解最低的南北向带宽，以及出口和入口带宽。所以我们确实有一份要求清单。通常我们会尝试进行一段时间的验收测试，先拿一个节点，持续运行和测试；在某种程度上，这也需要一点信任。有时我们会签下金额高达数千万或数亿美元的大额合同，开始与一家我们还不完全了解的供应商合作。随着时间推移，我们就会逐渐看清情况。我觉得，我们内部多少也有一套自己的评估标准。你知道，SemiAnalysis 有 ClusterMax，我感觉我们内部几乎也有自己的 ModalMax 分数。我知道自己更喜欢哪些供应商，哪些没那么喜欢，而且……

24:00 而且我们多少更倾向于和那些我们见过、可靠性非常好的供应商达成更多交易。不仅如此，可靠性还不是全部；当我们遇到支持问题时，他们的响应速度如何？他们能否在我们要求的时间内修复问题？如果服务中断，他们能多快修好？诸如此类。你是否同意，对智能的需求基本上是无限的？如果是这样，为什么不尽可能多地买入容量？如果买多了，还可以转卖给别人。我觉得，当有某种稀缺资源时，人们很容易会说：“那就尽可能多买。”但我想，首先，如果我们今天买下这些容量，我们没法全部用上。我们可以用上很大一部分，对吧？但是你可以把剩余的卖给其他想用的人，对吧？我觉得难点在于，如果市场上发生了什么变化，很多因素会同时出现相关性。比如价格下跌、利用率下降，以及转售容量的市场也在差不多同一时间消失。我现在就在这里，每天都在非常积极地购买容量，也在筹钱购买，所以总的来说，我非常看好这个市场，但我确实想指出，存在一些尾部风险。而这些尾部风险一旦发生，情况可能会变得非常糟糕。部分原因可能是，我年纪大了，过去见过这种情况；我第一份工作其实是在1999年，所以我经历过互联网泡沫危机，也经历过全球金融危机之类的事情。而且，我想打造一家能够持续经营20

25:30 到 30、50 年以后，对吧？所以，你知道，我想我们当然可以现在就去买下惊人数量的算力，试图垄断市场。但你知道，你听说过亨特兄弟吗？这是交易界的传奇故事。他们试图垄断白银市场，结果惨败。你知道，所以这就是垄断市场。我觉得一般来说，这可能不是个好主意。不过，总体而言，我们确实非常、非常积极进取，但我们正在加快步伐，采取积极的计划，基本上与我们的需求增长相匹配，也符合我们希望在明年年底达到的运行容量或运行速率。你们只买 NVIDIA 的东西，还是也会从其他类型的公司购买算力，比如用来训练的？我们正在考察。我觉得替代加速器开始变得非常有意思。讽刺的是，过去五年，甚至可能十年来，我每年都在说，明年就会是非 NVIDIA 加速器的时代，结果我总是判断错。但我确实认为，2027 年可能终于会成为非 NVIDIA 加速器的时代。我们拭目以待，看我今年是不是也会判断错。不过，我觉得这些产品在某些方面已经开始取得非常不错的成绩。我认为，TPU 显然已在 Google 内部得到广泛应用，而且似乎效果很好。我不太了解它们在训练方面的表现，感觉我看到的还没那么多，但你知道，那个团队非常优秀，我知道他们正在开发很多软件方面的东西。

27:00 你知道，硬件在某些方面很难。软件就比较容易。这让我觉得，如果我们只要解决这些加速器的软件层问题，我认为市场对它们的需求就会很大。而且我认为，从根本上说，NVIDIA 是一个很大的瓶颈，我们可能会看到更多替代加速器。话虽如此，供应链更下游也有瓶颈。我是说，还有 TSMC 和 ASML，它们或许是这一切更深层次的瓶颈。不过从长远来看，我认为垄断不会永远持续下去。因此，我认为市场上有更多替代选择会更健康。虽然我很喜欢 NVIDIA，但我想明确说明这一点：他们显然是一家了不起的公司，产品也非常出色。我确实希望看到市场更加多元化。还有 Cerebras、Grock 这些公司。对。你怎么看这些供应商？我觉得它们相当令人兴奋。而且你也看到，它们在某些工作负载上表现很好。有些人正在尝试有趣的混合式、解耦推理方案，比如用 NVIDIA 做预填充，用 Cerebras 做解码之类的。哦，我还真没听说过。对。还有 Grock，因为很多情况下，不同任务有的受计算能力限制，有的受内存限制。所以你可以把它们搭配起来。我觉得这相当令人兴奋。在实际部署这些方案时，事情会变得非常复杂。但我确实认为，这些更复杂的硬件配置有很多机会。你从一些 Neocloud（比如 Nebius 之类的）那里学到过什么吗？比如它们的做法与

28:30 你喜欢自己在做的事情吗？我觉得 Neoclouds 有一件事非常有意思：它们怎么能扩张得这么快？我觉得最疯狂的例子其实是 N scale，它一开始几乎默默无闻，后来却有了数千亿美元的订单储备，尽管它几乎没有任何算力。Neoclouds 怎么能增长得这么快？我的意思是，答案基本上是这样的，可能显而易见，但它们会去找 Anthropic 或 OpenAI，或者某个大型买家之类的，然后说：嘿，你想买五年的芯片吗？签这份合同，我们给你一个优惠价格。接着，它们拿着这份合同去找华尔街，说：看，我们有这个大型买家，他们承诺购买价值 100 亿美元的芯片。我们现在只需要监控它。然后它们就去融资。接着，它们去找 Super Micro 和 DAL，或者英伟达之类的公司，拿到硬件，然后安装。然后，现在它们就把算力交付给客户了，对吧？所以，这是一种非常巧妙的金融架构，你基本上可以预售自己还没买到的东西。这意味着你可以极快地扩大规模。这就是我们在这些 Neoclouds 身上看到的情况：它们确实能够非常、非常有效地扩张。我觉得 Modal 因为一直更注重 PLG，而且没有同样的能力去锁定长期合同，所以从财务角度来看，要以那么快的速度扩张就更有挑战性一些，尽管我觉得从长远来看，我们会……

30:00 同样的观点。但我觉得，看到这些 Neoclouds，尤其是像 CoreWeave 这样的大型公司如何运作，确实很有意思。他们在融资方面非常老练，也很懂得如何把 GPU 作为贷款抵押品，以及诸如此类的事情。这有什么意义？我记不清是谁了，好像有家金融公司说要把 GPU 变成一种资产类别。哦，我想起来了，应该是 Nvidia。他们说要把 GPU 变成一种资产类别。这是否只是因为他们需要融资，正在想办法筹钱？这到底有什么意义？我的意思是，有不少初创公司。我不知道你想的是不是这个。有 Silicon Data、computer exchange、Orn，还有其他几家，基本上都在尝试建立价格指标和指数、交易衍生品之类的。我觉得这很有意思。就算力而言，作为一种资产类别，它确实面临不少挑战。它显然不像其他资产类别那样同质化。网络、地点、磁盘、CPU，各种设备和可靠性都各不相同。不过话说回来，石油也有类似的问题，但我们还是把石油变成了相对标准化、可交易的资产。这一切还有待观察。我确实认为，考虑到我们如今 AI 数据中心和 GPU 部署的规模，我们需要更聪明地进行风险对冲、分担风险，以及……

31:30 有更多类似保险之类的东西。所以我非常高兴看到大家都在认真思考这个问题。话虽如此，我不太确定我们是否会看到 GPU 变成类似亨利枢纽天然气或布伦特原油那样高度商品化、可交易的资产。好。我想把话题转到 Modal 沙盒上。好。在此之前，我想先问一下，Modal 有哪些不同的产品？我想，主要产品显然是 GPU。然后还有 Modal 沙盒。还有别的吗？我认为我们有两款主要产品：第一是 Modal Functions，第二是 Modal 沙盒。我们在 2021 年推出了 Modal Functions，理念是提供易于部署、易于扩展的无服务器函数，并由我们管理所有容量。事实证明，它非常适合推理，也非常适合管理 GPU 和运行这类任务。不过，我们实际上从来没有专门的推理产品或 GPU 产品。我们从未把 Modal 看作一家新型云服务商，我认为公司有陷入这个方向、变成新型云服务商的风险，也就是只销售我认为相当固定的方案，比如“给你这些 GPU，用三年”。Modal 一直很擅长让你部署某项服务，并根据需要扩展规模。你的工作负载可能有很强的周期性，也可能会突然激增，而容量管理都由我们负责。美妙之处在于，我们有成千上万的客户，

33:00 我们可以把这些容量汇总起来，从而获得非常平稳的工作负载，也就是非常平稳的总需求，这样我们就能非常高效地管理它。所以，Modal Functions 是核心产品。我们在 2021 年推出了它。正如我提到的，我们在 2023 年开始看到产品与市场的契合。现在，它大约占我们收入的三分之二，并为许多应用提供支持；实际上，它主要用于推理，但也可以用于各种其他用途。Modal Sandboxes 是我们在 2023 年推出的。实际上，它和 Functions 类似，过了一段时间才看到产品与市场的契合。为什么要推出它？我们意识到，很多我们交流过的人都在用 LLM 输出代码。这是我和 Akshar 有一天吃午饭时聊到的。我们想，也许他们需要一个地方来运行这些代码。然后我们意识到，其实我们已经有很多基础设施，可以直接接收并运行代码。因为我们已经构建了管理容器和隔离的基础组件。我们已经有 G visors、隔离组件之类的东西。所以我们对这个想法很兴奋。Akshar 写了一个原型，基本上用了一两天就发布了。起初几乎没什么增长。我不是想轻描淡写，因为我们确实看到了一些增长。也有一些……你知道的。而且那是 2023 年。对，是 2023 年。有一些人开始使用它。我们有一些早期用户，觉得它在某些场景下挺好用。但总的来说，在 2023 年夏天之前，它带来的收入几乎可以忽略不计。两年后，我们甚至还讨论过：我们真的需要这个产品吗？

34:30 我们该关掉它吗？然后它就开始爆发式增长了。所以是 2023 年。我觉得 2023 年真正发生的事情是，语言模型在代码生成方面变得非常出色。是在 2023 年。2023 年。我的意思是，大概就是那个时候。我觉得那时它们从能够重构函数、编写代码片段，发展到了可以生成完整应用。没错。于是我们开始看到，很多增长来自氛围编程，你知道，像 Loveable 这样的公司，还有其他一些公司。然后，你知道，像 ramps 这样的公司开始将模型用于后台智能体。接着，我们看到许多 Neolabs 开展诸如在模型沙盒中进行强化学习等工作，带来了巨大的增长。所以，沙盒业务现在约占我们营收的三分之一。你知道，我们基本上把它视为我们的第二款产品。对我来说，这真的令人兴奋，因为早在模型发展的初期，我就一直知道，模型的意义在于打造一整套产品工具箱。我们一直希望它能像云服务一样，你有存储和计算能力，还有各种不同的使用方式。我们一直希望成为一家多产品供应商，但直到现在，这一点在实践中才真正实现。我觉得现在已经很清楚了。我们有函数，也有沙盒。我们还在开发很多其他东西，包括训练、批处理，以及其他一系列功能。所以对我来说，发展成为一家多产品公司，这种演进令人振奋，因为这一直是我们愿景的核心。早在最初的几版演示文稿里，我就放了好几张幻灯片来讲这个。但直到最近，这一点才真正为外界所知。

36:00 你们有没有正在开发、你认为会成为下一个类似这样的产品？对市场有什么预测吗？或者类似的想法？我是说，我对训练非常看好。我认为训练算是一个服务不足的市场。显然，大家都非常关注规模极大的训练任务。但我们实际上看到，较小和中型训练领域也有很多需求。所以我本周晚些时候会稍微谈谈这个。我们正在组织一场会议。我认为批处理是一个非常有意思的市场。我们看到很多需求，来自那些需要……的客户。前几天有个客户需要转录 5,000 年的音频。他们有数百万个音频文件。我们当时有很多可用算力，于是就让他们处理了这些文件，花了几天时间。我认为这里有一些非常有意思的应用场景。我也非常看好低延迟领域。我认为，一旦我们让音频、语音到语音模型，也就是语音模型开始运作，就会有很多……我预计这会是一个非常有意思的市场。当然，还有所有这些智能体相关的东西，对吧？某种程度上，我对“智能体”这个词的感觉很复杂，因为我觉得从某些方面来看，所有软件最终都会变成智能体。所以对我来说，“智能体”到底意味着什么，几乎成了一个问题。我认为从某种意义上说，随着时间推移，所有软件都会由 AI 驱动。但显然，确实存在一系列构建模块，可以用来开发更多由 LLM 驱动的软件，而这些似乎是不同应用场景共有的。

37:30 我们的想法是，你知道，我对模型的看法是，我们从事的是构建基础模块的业务。我们想为你提供出色、通用、经过实战检验、用途广泛且可组合的基础模块。我觉得智能体用例方面也有一些非常有意思的东西。你提到训练很有意思，因为我记得在另一个播客里，大概一两年前，你说你们并没有特别专注于训练。市场上发生了什么变化？其实我们断断续续地开发训练产品已经好几年了，所以我不会说我们从来没关注过它。但这就像是……我的意思是，我们现在有 150 名工程师，不过一两年前，工程师人数少得多。因为规模较小，我们不得不更加专注一些。因此，多年来，推理一直是我们的唯一重点。不过我们现在规模更大了，也有机会在其他市场下注。我也对继续扩展业务领域很感兴趣，比如后训练、强化学习。我觉得这方面有些令人振奋的进展。我还觉得，在存储、构建数据仓库和特征存储、实时流处理管道等方面，也有很多其他有意思的事情。总之，我觉得 Modal 的长期愿景基本上就是：你想打造一些很酷的 AI 产品，我们都能帮上忙；我们拥有所有这些基础模块，也会帮你把它们组合起来。

39:00 软件的妙处就在于，只要有合适的构建模块，你几乎什么都能做出来。这就是我们想打造的公司和产品类型。你说的训练，是更关注大型企业进行大规模训练吗？不，不，他们会去做那些大型超大规模云服务商，或者说 Neo 云巨头的交易，拿到 100,000 个 GPU 之类的。但我认为有一个服务不足的小众市场，比如研究人员迭代，或者训练较小模型的人。你知道，文本和语音、文本模型、计算生物技术，实际上都是我们看到大量需求的市场。我们在这个领域有很多客户，训练的模型规模相对适中。比如可以用 64 个 GPU 训练，或者训练推荐系统，以及很多传统机器学习模型。有很多事情并不需要 100,000 个 GPU。我觉得我们暂时不会专门专注于那个市场。但我认为，在规模稍小的市场里，有很多需求尚未得到满足。搭建训练集群非常困难。是啊。你有没有注意到，比如从 GLM 5.2 或者也许是 Kimi 2.5 之后，很多人在用这些模型并对它们进行微调？如果说有变化的话，我看到的其实是相反的趋势。我觉得几年前微调的价值主张是：开源模型不够好，但如果你拿一个小模型，在特定领域进行微调，就能在质量和成本方面达到前沿水平。

40:30 如今，我想说开源模型已经非常出色了，基本上可以直接替换使用。比如，不用前沿模型，改用 Kimi K3、DeepSeek 4.1 或 GLM 5.3，成本低得多，效果也几乎能媲美前沿模型。所以我们仍然看到很多对后训练的需求。我确实认为这是一个非常有意思的领域，我们也在开发一些相关产品，包括 SFT、强化学习之类的。我还想说，微调已经不再是必做的事情了，它完全是可选的。开源模型直接开箱即用，效果也很好。回到沙箱的话题，我刚才忘了问：你们为自己提供的沙箱做了哪些改进，是现有沙箱做不到的？没有太多。我认为我们一直非常擅长的一点是——我们都是基础设施极客，对吧？我们很擅长扩展规模、保障可靠性，以及做好很多相关工作。我认为这在很大程度上解释了为什么，在我看来，虽然我没有数据，但我们在这个领域的市场份额远远领先。我们确实很擅长运行数十万甚至数百万个沙箱。是啊。我想说，为了应对这种规模，我们不得不多次重写系统。

42:00 但我们一直都非常擅长这一点。随着时间推移，还有其他一些事情。我的意思是，总体来说，沙箱领域并不是一个差异化程度很高的领域，但我们正在开发一些非常酷的东西。随着时间推移，我们不得不增加对 VM（虚拟机）的支持。我们之前的沙箱产品基于 GVisor 构建，但现在也提供完整虚拟化的机器。事实证明，在很多特定用例中，你需要完整的 Linux 兼容性。我们还不得不构建其他东西，比如更好的基础功能，以支持更具交互性的操作，例如在沙箱之间来回传输数据、处理文件系统。我们还花了很多时间在快照上。我认为，长期来看，很多能带来差异化的东西会是一些非常酷的功能，比如沙箱分叉、创建快照、恢复快照，以及实时迁移。总的来说，我认为沙箱的新颖之处一直在于它的短暂性：你可以启动一个只运行几秒钟的沙箱。这一点很吸引我，我想在这方面继续深入探索。我认为这里有很多很酷的事情可以做，比如将代理与运行框架解耦，使其不再受沙箱限制；让代理控制多个沙箱；分叉沙箱；创建快照；以及运行寿命非常短的任务，比如每条命令启动一个沙箱之类的。我认为这里还有很多很酷的事情可以做。我们已经在这方面花了很多时间，比如处理快照。但我认为这里还有很多值得开发的酷东西。你已经提到快照好几次了。

43:30 Like what do you see as like the use case for snapshotting? Like what would that give us that we don’t get right now? I think the big thing for me is like today a lot of sandbox workloads are kind of like lift and shift. Like people think a lot about sandboxes almost like, you know, I have my dev box, now I’m going to shove it in the cloud, I’m going to have an agent control it, maybe I’ll run the harness inside the sandbox. I don’t know if like long term that’s like the right way to think about, you know, on like compute for agents. I think a much better model would be you’re going to have this like very long running agents, which almost like to me are like kind of durable execution workflows. Like they’re going to run, you know, 24 seven, you know, every once in a while they’re going to run commands and, and you know, having the ability to like spin up a sandbox in, you know, 100 millisecond, run a few commands and then like shut it down and like snapshot it. I think the faster you can get that like, you know, wake up, you know, go to sleep cycle, the more you can sort of create sandboxes that only run for like a very brief period and that will make compute much more efficient. And it will also enable you to do some really cool things, you know, like forking and moving things around and, and, and, you know, having like imagine, you know, like Claude code instead of, you know, having sort of a very linear flow, like sequential flow, like kind of branching and, you know, running hundreds of agents at the same time. I think a lot of like CI is like another like illustrative example, like for most of the, the history of CI, you know, talk about like GitHub actions and things like that.

45:00 Like you had a very sequential flow, like, you know, you slowly install a bunch of stuff, then you run the test suite and then, you know, generates some reports or whatever. And it’s like, you can break it up into shards, but it’s kind of annoying. Like what I would love to do is like take your entire test suite and just run every single test in its own sandbox, right? Like, and just paralyze the shit out of that around like a thousand sandboxes. Why can’t that be done? I mean, the problem is like there’s too much, there’s too much overhead around like, you know, starting and stopping sandboxes. So I think the more like you create like, you know, frictionless, like starting and stopping, the more you can enter a world where like you can do this like extremely bursty, like large scale, like run as much as you think you want in parallel. So, you know, I’m an infrageek, like I said, like I get very excited about these things. You know, once we figure out all these primitives and the right building blocks, I think it’s going to unlock a lot of really cool stuff that, you know, we previously could not do with software. Why hasn’t that already been built? I would assume even previously people would want to parallelize as much of their CI as possible. You can’t, but the overhead, it’s like Amdahl’s law or whatever. Right? Like the overhead of parallelization right now is very high because like right now, like you can get up actions, for instance, like we parallelize in the modal test suite, right? But like what that basically means is like now if you take your test suite and you split it up in two, like now each of those shards both need to do all the installs and the test setup. And that takes a long time. Like it can take minutes, but let’s say you can do the test set up once

46:30 and then you snapshot the container and now you clone it and now you can run the test suites independently. I think a lot of the benefits cannot be, you can’t materialize those benefits until you get the friction and the overhead down, the transaction cost. I think you guys have a partnership with Anthropic and they’re using sandboxes for something, like modal sandboxes. Yeah, I mean the managed agents with Anthropic, this is sort of bring your own sandbox integration. So you can use modal for those, yeah. How did that come out? Like come to be? I think Anthropic is like trying to create like a thriving ecosystem and community and so I think they make money from inference, right? Okay. I think it’s very much in their interest to promote, for everything else, as much of an ecosystem as possible and they want to have the largest possible surface area as long as you use Anthropic for inference. So I think it makes like very rational sense for them to partner with us as a sandbox provider. Also, I feel like we have to bring up the hugging face incident. Are you okay talking about that? Yeah, yeah, I mean it’s all there. Yeah, yeah. So yeah, could you just briefly describe like what happened and I think they were using the modal sandbox but it wasn’t your, you know, it wasn’t like a fault of the sandbox. Yeah, exactly. I forget all the details of the exploit but I mean, you know, I think what you’re referring to is the fact that like modal was involved. Yeah. And so, you know, just to comment on that, like basically the agents running inside,

48:00 you know, that exploited hugging face, they discovered a vulnerability in an app that was deployed on modal and they used that to sort of, it’s almost like a springboard for getting access, getting full internet access is my understanding. I don’t remember the full details. To be clear, this is an app running a modal that was hacked. Like it was not modal that was hacked. So it’s like AWS, a lot of people run stuff on AWS, occasionally those apps get hacked. Does that mean AWS got hacked? Like no, of course not, right? And so there’s a lot of people running apps on modal and, you know, one of those apps got hacked. And I think, but like there are some lessons we can learn and I think one of the things we’ve been reflecting on is like, could we have more sensible, you know, security restrictions by default, I think to use the AWS analogy and just as another analogy, you know, S3 used to be this, like, you know, it used to be very easy to accidentally make an S3 bucket public and AWS realized like, you know, it doesn’t necessarily cause it, you know, poses security problem for AWS, per se. But a lot of their customers accidentally would, you know, like leak data. And so S3, it’s much harder now to make S3 buckets public. And so in a similar way, it’s like, we’re exploring like, you know, can we make security a little bit more restricted by default without necessarily, you know, destroying the magic that makes modal good. And I think there’s also, you know, it’s kind of a separate sidetrack is like, agents are getting really good at cybersecurity. So we’ve actually also, as a sort of, you know,

49:30 separate sort of learning from this incident, like we started using agents internally quite a lot to try to find vulnerabilities so far hasn’t found anything, but like, they’re very good at like, probing into like very deep parts of the system and exploiting, trying to exploit things and trying to escalate and get out of sandboxes. So, you know, it’s, it’s, it’s something we’re exploring very actively right now. How exactly do you get the agents to like run through the code base? You’re just like, hey, like find security vulnerabilities? Well, no, you just like, you just put them in a sandbox and tell them like, can you try to explain? Just run through the escape from the sandbox. It’s like, escape the room, but for agents. Cool. When did you start doing that? We started doing it a few months ago. And it’s pretty exciting. Like when Fable, I guess, came out. Yeah, roughly around the same time. Okay. I mean, my view on like cybersecurity is that, you know, I actually think they’re very good at finding vulnerabilities, but I think you can also use them defensively very well. So I, I, I think, you know, companies like modal and many other companies right now, like, I think, you know, if you go to talk to like pretty much every system, I’m sure they’re all thinking about the same thing. Like how do we use agents defensively to, to, to, you know, fix all our vulnerabilities? So over time, actually, in a way, like I’m not that nervous. I think if anything, like agents and cybersecurity, all the breakthroughs in cybersecurity will, will make the world safer, if anything. Do you think that like all of the cybersecurity, the big cybersecurity companies are just cooked? I don’t know if that’s true, because I mean, you’re asking something that’s like very far

51:00 outside of my like core expertise. But when I think about like cybersecurity, a lot of what you’re selling is effectively fear. And that’s almost like a branding and like marketing strategy, as much as, you know, obviously the core products. But I think, you know, brand and, and things like that are actually like arguably like more valuable in the air of AI. So I wouldn’t necessarily jump to the conclusion that they’re cooked. I think they have, you know, deep relationships with the enterprise customers and they have, you know, big sales teams are very good at telling, you know, spooky stories and, and things like that. And I think they’re going to continue to be very good at that. I just ask, because like, yeah, you’re doing it yourself, you’re not going to one of like the Palo Alto networks or whatever. But I guess like it’s only like giant enterprises that care about that. Also, they can blame somebody if something goes wrong. That is true. Yeah. Yeah. You kind of want to hire in many cases, just like an established vendor in order to blame them. I mean, to some extent, it does reduce a little bit the need for us to do, to bring in pen testers like human pen testers, which most startups like do on a regular cycle. So maybe previously, you know, we would do it, I actually don’t know what the cycle was, but, but I think we used to do it like once every six months, roughly. Maybe now we only need to do it once every year. I don’t know. But, but like, I think with all of these things, you know, humans can also use them to get more efficiently. So like, you know, like, I don’t know, I’m kind of a believer also that like humans with AI is even more powerful now. So like, maybe there’s an argument that like we can bring in pen testers

52:30 who are really good at using agents and, you know, find vulnerabilities even faster. I don’t know. I’m not that, you know, I’m an optimist. I started a company. So like, you know, I’m generally like optimistic about everything. But like, I certainly don’t think humans are cooked or companies are cooked. Like I think in the era of AI, a lot of companies are going to figure out, you know, how to use AI to be even more efficient. I want to transition a little bit to like developer experience, because like early on, I mean, even now, like you guys focus a lot on that. I mean, you’ve talked a lot about that. Does developer experience even matter anymore? I think it still matters. And I think you’re sort of, you know, like the context, I think you’re official, like the thing you’re talking about is like, now with agents, right? Like the, the developer experience, like the docs are already there. The agents are going to look at the docs, like all that stuff. To that point, though, like almost all the things that make developer experience good, also makes the agent experience good is kind of what we’ve seen in practice, which is like things like having good docs, having descriptive error messages, having fast feedback loops, right? Like, I do think there’s certain things have changed. The other day, I was like setting something up with Cloudflare, and it’s like, I find Cloudflare like notoriously annoying to work with sometimes. And, and I threw an agent at it, and it was able to like just configure a bunch of like, you know, things and pollute me for me. And, you know, and so I will say like agents does, they do make it a little bit easier to work with things that are like harder to work with in the past, you know, like the sort of human toll of dealing with the AML as arguably gone down a little bit, because you can just like,

54:00 you know, tell your agent to deal with it. But, but for the most part, I think, you know, the things that make developer experience good also make agent experience good. I think to some extent, also like infrastructure over time, anyway, becomes, it needs to be more programmatic. I think that’s always been the thesis about sandboxes, is that, you know, like things that you previously had to configure using YAML, that should just be an SDK. That should be more like imperative, like launch a sandbox, like do stuff with it, tear the sandbox down, right? Like, rather than sort of declarative using, using, you know, some sort of DSL, it also pushes like a little bit of a different interface that maybe previously didn’t exist. I think in either way, like SDKs are as relevant as they were, as they always were, if not more. Is there anything you changed in your product? Just because like, you know, most people are just running agents on it? We do. I mean, we also, we actually do run a lot of internal tests where we have agents, we tell agents to like, build a mobile app and run it. And I mean, a very concrete thing that is kind of funny is that we have actually seen that like, because of agent training data, it’s very hard to deprecate things now. We see that like, agents often like, love using, you know, old versions of modal syntax and it’s kind of harder to nudge them away from that. So we’ve actually had to be a little bit more conservative going forward, like, you know, over the last year or so with, with how we deprecate and remove features, it takes, we have to be very careful to update every single example that’s out

55:30 there online. Yeah. And, and, and, you know, skills, right? Yeah, there’s like stuff like that. But, but like, there’s also an insane amount of like modal code that’s now like online too that goes into the training data. I’ve seen like that to be like kind of a, you know, a little bit of like an interesting example where like, another thing by the way too is like, there’s obviously like to sort of a similar to like search engine optimization, there’s like agent engine, you know, whatever people call it, agent optimization. We often like, run internal benchmarks, we’re like, ask an agent, like, hey, like, can you deploy a stable diffusion for us? Use whatever tool you want. And we see if it recommends modal or not. And those things are very useful for us to sort of track over time. Or like, hey, spin up some sandboxes or like, hey, like rank all the sandbox vendors. So we spent a lot of time like doing those things to make sure. What do you do about that? Yeah, you say, hey, like, you know, build this stable diffusion thing, does it recommend modal or not? Like, yeah, what do you do about that? Yeah, I think that’s harder. I think, you know, it’s like search engine optimization, I think it’s primarily through content is like, you just have to put out a lot of content. You can also to some extent, like sort of see the perception, I think, a little bit more like you can ask an agent like, you know, how does, you know, rank all the sandbox vendors by reliability or rank all the sandbox vendors by price and sort of see, and it does feel like agents are a little bit more prone to like, misunderstanding pricing and, and, you know, things like that. So we also spend a lot of time like just making content more clear

57:00 and more like, you know, understandable for agents, because we know they’re all indexing, like everything that’s out there and that’s talking about modal. Have you thought about just like, making a bunch of GitHub repos and having the agents like write a bunch of modal code in there? Have you tried that? Slop cannon. I think the truth is like, that is probably something you can do. My guess is like LM training data today is probably quite insensitive to like, what’s like AI output in the first place and you can probably like create a lot of slop. Like we thought about it, like, I don’t love that idea. But like in theory, like much like, you know, SEO in the early days and still to some extent, like auto-generated content is like a very effective strategy. I think it’s possible. Like, you know, you could, you can do those things too. Yeah. I want to switch to FDEs. Apparently, you guys hire like forward-appoid engineers. Like, what do you do? We’re in a new era of like, building applications and, and, and, you know, things that we couldn’t build, you know, just a few years ago. And we see a lot of companies that, you know, they need a little bit of help, right? Like, you know, and so we started investing in FDEs a couple years ago, but it’s really been this year, it was kind of scaled it up a lot. I think we started this year having like four FDEs now. I think we have like 24 or something like that. So we’ve grown it a lot. And they do it, you know, it’s more, it’s not exactly the pound term model, but they do a lot of like heavy, you know, sort of customer engagement. Like they work with a lot of customers on performance tuning, especially with LMS. We see that there’s so much benefit from like, you know, sitting down

58:30 with customers and training draft models and tweaking, you know, parameters and DLM and SG Lang. They often, you know, fix bugs or not fix bugs, but like, you know, when customers run into issues, like they often work with customers trying to figure out like, how do we work around that? Or another thing we’ve used FDEs a lot for is like, you know, kind of proactively, you know, if we see certain customers, you know, over provisioning with model, like customers requesting too much CPU or too much memory, for instance, like that’s kind of expensive. And like, we try to catch those things and we try to sit down with those customers like, yo, like you’re spending a lot of money on model right now, we can actually help you save money. And then of course, like with sort of active deals, you know, we helped them migrate code from other platforms, things like that. I think, you know, we never have the intention of becoming like service heavy business. I think that the goal for FDEs is like, you know, A is to be a part of, you know, the sort of sales process and like closing deals. B is to serve customers and do some like, you know, customer support. But like, I actually think like the most important thing is like, C is like bridging gaps in the product that we later want to turn into like real features. So I think to a large extent, you know, anything an FDE does, you know, my question is always like long term, how do we turn this into product so we don’t need an FDE for this? Yeah. Do you guys hire PMs?

1:00:00 Product managers? Yeah. No. Would you say these FDEs are kind of like pseudo ish PMs? It’s a little bit of a different function. Like I think certainly on the more like, you know, customer facing side they are, but there’s like, you know, on the more like inward, you know, internal side, like this is sort of separate part of like product management, which is more like, what are the next building blocks Modal should invest in and like, you know, and then there’s like something in between like, you know, looking at like maybe 10, 20 different customers that are all asking for like slightly different things, but maybe what are the patterns? Like how do we turn this into product? We do have people doing product with us and it’s sort of evolved, you know, we’ve had people basically like become product managers after previously being either DevRel or engineers and showing strong aptitude on doing thinking very strategically about what we have to build. So I’m actually not like anti product management. I do think that a lot of companies in the past hired product management product managers too eagerly. And I think there’s a risk that you kind of, I can sort of see it in a lot of products like certain products like you can tell like, there’s just like eager product managers that just thought of like, kind of like, wigity things and just ship them. I think doing product management at an infra company really well is very, very hard because I think it takes like a very deep, you know, level of strategic thinking thinking about like, what are the foundational building blocks? Where do we want to be in one to two years? Because the truth is building infrastructure is very hard. And how do we make sure we don’t end up with like 35 features that are like,

1:01:30 kind of similar looking, but all do like a little bit slightly different things. That sort of level of thinking is something that, you know, is very rare to find. And for that reason, we haven’t aggressively tried to build up a product management team yet. Yeah, that makes sense. How would you say your teams have been evolving maybe like since Genialite? Because I would assume, yeah, I mean, you obviously worked at like Spotify, you had like a certain team structure, have your team structures like evolved a lot now that, you know, we have coding agents. In a funny way, like, yes, I do think team structures have changed a lot, but I’m actually not convinced it was like Genialite driving that. I think in particular, like one thing I’ve seen a lot is that there’s been sort of a focus, like a swing back towards being in IC that I actually think is very, very positive for the space as a whole. I think, you know, if you look back at the space like 10 years ago, 20 years ago, everyone wanted to be in management because that was the way you would get more money and more recognition and more prestige. I think that shift actually started even before Gen AI, but I do think it’s like heavily accelerated with AI. Like, you know, we make it very clear to everyone at Modal that like manager or RC, it’s like there’s no difference in prestige. Like everyone is just a member of technical staff. You know, it’s about ultimately your contributions and we see a lot of X managers, you know, becoming ICs and because they want to go back and build code. And so, and you know, and there’s been sort of shift away from like, you know, non-technical managers,

1:03:00 everyone, including me, I still write a lot of code at Modal and Akshath too and pretty much everyone in the tech team and a lot of people who are not in the tech team still write code. So that to me is like, you know, a recent phenomenon, but I actually don’t know if it’s like necessarily cost by Gen AI, but I think Gen AI has like greatly accelerated it because it made it even lower the cost of like writing code is even easier. So if you have like non-technical people like committing code, like how do you, how do you review that? Like how do you guys do code reviews? We do code reviews, but it’s, we think of it as optional. I mean, for most, my view is like code reviews in large part is like people, in many cases, trying to just escape the accountability of like, you know, merging things into production. And if you have that like, you know, if you have that culture, like then to me, that’s a sign that you need to invest way more in, you know, CD and CI and things like that is CI in particular, but also CD, containers integration, because it’s deployment. We do code reviews. Like, I mean, like when I write code, like occasionally, like I’m asking, you know, authors, like, is this in line with how you think about things? I hope I’m not changing the architecture fundamentally, but we also make it clear internally that like code reviews are quite optional. And, you know, if you’re just changing docs, if you’re just doing, you know, some basic stuff, like don’t wait for a human to look at it, especially in the era of AI, why do you need a human to look at every piece of code? You don’t have to. But that does require you to have, you know, very good controls, you know,

1:04:30 internally around, you know, testing things. Yeah, and exactly. Like, you know, infra companies, like we need one more nine than the average startup, right? In terms of uptime. But like, you know, these are investments you just have to make. Like, you have to build systems that are very resilient to like, you know, you have to have rate limits everywhere. You have to decouple systems. You have to make sure that like, nothing is a single point of failure. They have to have extremely good test coverage. Agents are actually really good at writing tests these days, which was like, you know, a few years ago, I thought like, one of the major things missing in agents, now they’re actually quite good at like, because I think it takes a little bit of like, sort of kind of like, need to step out of what you’re trying to do for a second and be almost a little bit like devil’s advocate to write good tests. But now they’re actually quite good at writing tests too. So I think, I think all these things help. Do you guys use like frontier models for coding or do you guys have you guys experimented with like, open source ones? We experiment with everything. But yeah, I mean, you know, initiative is a lot of cursor, then, you know, of course, Claude and Codex and now a lot of people are using, you know, night shift and all these open source stuff or other stacks. So I don’t necessarily have like a strong point of view, like I want people to experiment and try different things like we, you know, use frontier models, use open source models, we’re trying to do a little bit more dog floating internally. Now that we’re actually running inference for a lot of the open source models, like a very large scale. I think it’s also helpful to like use certain internal endpoints for a lot of agentic development. So we try to promote that a little bit. Something I read maybe like, I don’t know, a year ago, maybe like two years ago,

1:06:00 you were saying that you didn’t think that there was going to be like a big on-prem business for you guys. Have you maybe like changed your opinion on that? No, I have not changed my opinion. I think, moral to me, I’m a Claude maximalist and maybe to take a step back, like when the cloud came in 2008, 2009 or whatever, my first reaction was like, that’s crazy. Like why would I want to take my code and run it on someone else’s computer? That’s like insane. And then like six months later, I was doing it. And I think, you know, like took me six months to change my mind. It took probably enterprises like another 15 to 20 years. But like you look at, you know, vendors like Snowflake or I mean AWS and like, I think there’s many examples of Datadog, et cetera. Like the cloud is clearly winning, right? Like and the multi-tenant systems are clearly winning. And you know, to me, on-prem looks increasingly like sort of a legacy business model. And I actually think, you know, BYOC, you know, sort of VPC peering, like all of these things, you know, like to me, they reflect sort of an outdated security model of like, we need to protect everything inside our own network, right? Like using the network as a mode. And that I don’t think is true anything anymore, right? Like we’ve sort of moved on to more like actually robust security paradigms, like, you know, zero trust and things like that. And so to me, BYOC and those things, like they look increasingly

1:07:30 as like outdated, at least from a security perspective. From our perspective also, like the fact that we have full control over our entire infrastructure stack, we rent all the hardware, we deploy, we manage all the infrastructure, we deploy and like manage health and observe, you know, all the metrics and all the hosts. It means we can iterate very fast and build a greater product. And I think that’s very, very valuable versus, you know, customers in the past who had enterprise editions that they had to deploy on customers environment, they had to be very, you know, they have to think very carefully about release management and having the like, we don’t care. Like if we want to introduce a new database, like we can introduce a new database because we have control over it. Like we don’t need to think about what customers have access to. So to me, multi-tenancy is, is, yeah, has so many different benefits. I mean, that being said, like, you know, if we were to work with like, you know, some very large Fortune 500 customer in the future and, you know, they ask for BYOC, you know, we’ve done this in the past, we’ll give them a number. Like, you know, that number is very high and no one said yes to it so far. But like, who knows? Like, you know, at the right price, like I would still build it. And, and I think there’s like maybe separately, like another thing which I think is interesting, which is like, in theory, BYOC might also be a way to work with and get capacity in certain ways. Like we sometimes work with Neolabs. And those Neolabs have already have very large capacity reservations. And, you know, the question is like, you know, how do we, how do we get them to run modal when they already have capacity? And BYOC might be a solution. We can still keep the control plane in our environment.

1:09:00 And the other flip side of that is like, we could maybe even run like other people’s stuff on that capacity. So there’s some like, kind of fun stuff you could do in theory where like, you can sort of, you know, like kind of aggregate all that capacity into one big pool. So we’re excited about some of those ideas. But in general, like I feel like on-prem, I don’t know if we’re ever going to do. But BYOC, like maybe bring your own cloud. I can say BYOC. I should explain. Let’s say hypothetically, you are going to raise, I should tend more money like soon. What would you use that money for? I mean, like I mentioned, like in this capacity situation, the, a lot of the money, a lot of the cash flow, I should say, that’s required is capacity, prepayments, right? And so that to me represents the best use of cash today. We’re ramping up capacity very aggressively. Maybe we move into, you know, other types of deployment models in the future, ownership or whatever. But I think there’s always going to be a cash component of that, like some sort of down payment or prepayment or some sort of that. So that to me is the greatest use of cash right now. There’s of course other things like, you know, salaries and office spaces and stuff like that. But that actually in the grand scheme of like our financials, like it’s basically a rounding error, right? Yeah, interesting. Do you think that’s going to change? No, I think, you know, the sort of operating leverage that we have in this business is kind of insane. Like, you know, we have 200 employees, you know, they cost a lot of money, but, you know, the

1:10:30 GPUs cost a lot more money. Yeah. If you had to say, like, what are the top, like, I don’t know, like three things that are your biggest focus or the business’s biggest focus right now, like, what would you say those are? Capacity and the sort of financial complexity. And, you know, in general, like, how do we, how do we think about that space? How do we think about pricing and packaging? How do we think about the sort of tension between ownership and renting? Should we be going a more neocloud direction? Should we go in the opposite direction? The sort of counter position, like, that’s probably the biggest thing that I think a lot about. The second biggest thing I think a lot about is probably around, like, more like product strategy. Like, I think a lot about, like, what is the next building blocks? Like, if we, you know, do more like, agentic stuff, like, what are the things we need, you know, are there certain storage primitives or compute primitives? Or in general, like, what is the thing that the world is going to wake up and realize they need over the next few years? And ideally, we have that at that point. We’ve been pretty good in general at, like, you know, anticipating demand and building products in advance of it with the serverless functions in the sandboxes. So what’s the next thing? Then I would say, like, actually the third thing I think a lot about is, like, more on, like, culture and company building and stuff like that. I mean, we recently crossed 200 people. We’re growing extremely fast. We’ll probably hit 250 by the end of the year. Like, when you’re growing very fast, you have 250 people that are all trying to figure out how they can have the most value to modal and, you know, making sure that they have the right context, that they have the right motivation, that they’re excited to come into work every day because they get to build something incredibly cool

1:12:00 and they know, you know, what makes modal successful, what they can do, aligning the incentives and making sure that, you know, we, we, people have like a very positive culture where they have the sort of implicit trust to do a lot of stuff, but also the accountability to like, you know, deliver the best things and, you know, and be held accountable to like, if they’re not doing it, to me, those are like, you know, super critical things around this scale. Like, when you’re like 10, 20, 30 people, you can all just like, kind of sit in a room and like, it’s all kind of implicit and you kind of just like, feel it. People can sort of read, read me like, they can tell me when I get a little bit annoyed. They can tell when I’m happy, but like, now we’re 250 people and like, people don’t necessarily see it and, you know, of course, there’s like a lot of new people that are like, maybe a little bit nervous to like, talk to CEO and like all this stuff. So like, you know, how do I remain approachable and how do I, like, it’s, it’s, there’s like the Dunbar number, which I think is like more sort of, you know, faked than, than anything, like at least like the story behind it. But like, it is actually kind of real still, like there’s something around like 150, 200, we’re like, laying the foundation of the culture is very, very important. How many people are you guys trying to hire over the next year? We’ve grown about four acts on an annualized basis in the last 12 months. I think that’s like an unsustainable growth rate on a relative basis. I’ve seen that, you know, you can double the team every year and that’s not super hard to do, but four acts, what you start to see is, you know, cultural drift, you start to see islands of different working styles.

1:13:30 You start to see people doing like very speculative things like that. That’s something I always watch out for in the engineering team. You go and talk to the engine and engineer and I’m like, what are you working on? And it turns out they’re like building something like kind of like fringe and like, is this really what you should spend time on? So growing very fast has that cost and you know, it has the cost of ramping people up. So I don’t know. I think by the end of the year, maybe 250, about the end of the next year, you know, but like, you know, we’re going to keep growing like maybe two and a half, three acts. So it’s very likely we’ll end up like next year, like 700 people, something like that. I’m going to assume most of those are engineers. Most engineers. We’re wrapping up sales a lot too. Okay. What kinds of engineers? It’s primarily systems engineers. I mean, we’ve always been like very deep in the sort of, you know, Linux and file systems and scalable systems and networking and cloud and stuff like that. So that’s the core of what we’re looking for. Like people are really good at those things. We’re also looking a lot at like, you know, more like, because we moved up in the stack a little bit, like we’re doing a lot of work in the LMA in front of space. So now we’re dealing with a separate set of concerns around, you know, CUDA and like GPUs and and and MFU, you know, like maxing out the tokens that you can get per GPU and a lot of that things. And, you know, even though I actually like will say a lot of the work in that space, in my opinion, turns out is like more systems driven still, because it turns out like KV caching and routing is actually like, is still like very like systems like like these are very like systems have problems. But yeah, a little bit

1:15:00 more like sort of GPU performance, that kind of stuff. We’re starting to build up a research team. We’re very excited about, you know, moving into stuff around like, you know, post training and distillation and better methods for quantization for your customers or for internally. Both. I think it’s like a little bit like with the FDEs. Like I think we initially will do a lot of stuff with our customers to optimize their models. But over time, figure out how do we turn those into products? I think we’ll sort of arguably fail in a certain way. If every time a customer needs to optimize a model, we have to go and work with them. And, you know, long term, it has to be turned into products. And so with the FDEs and with the research team, initially, you’re going to have to do a lot of work with the customers and we are that. But over time, we’re also building a lot of internal tooling and a lot of over time internal products that are fully self-serve. So we’re working a lot of stuff right now in post training space, but also in the GPU performance space. So a lot of cool stuff. A lot of cool stuff. Back to the product strategy stuff that you were talking about. Yeah. Yeah. Like what are your thoughts? Like what do we need over the next year, two years? I think it’s incredibly hard to say. And if I said like, I know exactly what we need to build, I think I would be like delusional. I think what we’ve seen so far, and what I kind of believe in on a more general level is like, we need better building blocks. Like this is a new era of software and we’re unlocking a whole new type of applications that could not be built a few years ago.

1:16:30 Doing all the GNI stuff, code that, LMS that output code that needs to run like in an isolated way. Like there’s so many like new primitives that are needed to work with GPUs and work with LMS. And we built some building blocks so far and that’s unlocked a bunch of applications that the world has seen over the last few years. Not just stuff with us, of course. But I think over the next few years, there’s going to be so many more cool things. And so I mentioned some stuff. Like I’m very bullish on training. I’m very bullish on fast jobs. I’m very bullish on certain things around like agents and low latency and things like that. But like in the end, we consider ourselves like an infrastructure shop, right? Like we’re not trying to build, you know, very narrow things. Like we try to build like very general purpose building blocks and put them in front of customers. And then those customers will combine those building blocks and build some incredible applications that we could have never anticipated. I want to go through some of your tweets. This will be the last part of it. Okay. And we can just like, maybe you want to double click on anything. A lot of long running persistent sandbox stuff right now. Today feels lift and shift. I suspect sandboxes will evolve from dev box into cloud to something more ephemeral, one environment for every command. And now I’m shouting in between so you can pause, resume, clone. Yeah. I mean, we talked about this a lot. Yeah, we talked a bit about it, right? Like I’m fully, I think, I think the median sound box in modal today is like eight seconds long or something like that. Like, which is kind of incredible

1:18:00 when you think about it. And over time, I think it might get even shorter. We want to support long running sandboxes too. Like I think they’re awesome, like very valid use cases around long running sandboxes. But like my vision is like, you know, the world needs also, we need to figure out this like how can we just massively parallelize things and reducing the overhead of starting and stopping sandboxes? I don’t know if I ran an IP already start up engaged in catfax arms race for the competitor, it would be nice to coordinate and paste arms race a bit. No comment. The biggest downside of GPU scarcity is not that it’s hard to get GPUs, but that almost all capacity is sold as three years as a three year rectangle is starting three to six months in the future. That’s not what demand looks like for most users. I think we talked about that a little bit as well. Yeah. And you know, maybe just go a little bit deeper now. Like one thing I think modal has always been very, very good at is fully on demand capacity, right? Like if a customer comes to us and need 100 GPUs, like we can typically give them that like immediately, you don’t even have to talk to salesperson. And a lot of workloads in real life, like they don’t look like these like flat, you know, rectangular workloads, like they scale up and down throughout the 24 cycle, they maybe grow over time, they maybe have certain burst periods. And to me, that is a challenge with a lot of customers today. Like the, and I do think it’s a legit problem for, for the space is like the only, the prime way to get capacities to, to buy these rectangular blocks. It’s not great. I think the world needs to, and that’s fundamentally, I mean, I’m very biased, but like that’s why I think modal can play a very large role in that sort of, you know, whatever you want

1:19:30 to call it, sort of duration transformation of offering more flexible capacity. And we can do that because we can aggregate many, many thousands of customers and the demand gets very smooth and aggregate. So that’s a lot of the value prop of modal is exactly that. Would you say that’s a differentiator? Like the, oh really? Okay. Yeah, massive differentiator. Actually, in fact, like, I think, modals, like if you want to be like very sort of reductive, like you can break down modal’s value prop into two things. What is a developer experience, the ability to like iterate and like build things and not have to think about infrastructure. The other thing is the capacity stuff. The more and more I think about modal and like where we’re going in the long run is like we’re building, like we’re aggregating the world’s compute capacity. And we’re very good at slicing it out like very granularly, because we’ve built all this work around, you know, scaling up and down extremely fast, you know, all this stuff around GPU, snapshotting, working some, some ways to make it much, much faster. And that model works at very large scale, like really, really well, because you can aggregate lots of customers. Every month I keep saying or saying box revenue might apply to it, but it keeps going up 30% every month. It’s kind of nuts. Like I don’t understand, you know, like every month I’m like, well, you know, sandboxes grew a lot, but like clearly the kind of plateau and then they keep growing even more. So I, I, I don’t know. And a lot of VCs, like they’re like, how did you know sandboxes was going to be big? Like, and the truth is like, I didn’t like, like, I thought it was going to be bigger than it was in 2023 for sure. But like the fact that sandboxes is a third of our revenue today, it’s, it’s insane. And, you know, we’re obviously very happy about it. And we have like a very large team investing

1:21:00 in sandboxes. You know, I don’t want to like trivialize it. Like we obviously, I do want to give ourselves like a lot of credit for, for seeing sandboxes for what it is today a little bit back in even 2023. But I think, you know, if like, I probably would have never, you know, taken a bet that revenue would be this large from sandboxes and the market demand would be so much. There’s an interesting structural reason why inference providers should exist separately from model builders. The best model keep changing a lot inference providers keep can easily reallocate resources between models since overall demand is quite smooth. So I mean, if you think about all the Chinese labs and, you know, in the future, hopefully more US labs too, every time a new lab releases, you know, the latest and greatest open source models, like you see big shifts in demand, you know, for these models. And so I think a challenge with, you know, if all these labs independently like held capacity, they would see when they launched a new model demand would go up a lot. And then another lab launches a better model and then the demand goes down. You know, but, but of course, like in the background, they’re working on the next version. So like, I think, you know, there’s like a structural reason, like I said, like why you’re going to start, you’re seeing demand kind of shifting to like the latest and greatest Chinese lab. I mean, not like, it’s not like, you know, this is like, everything just moves overnight because, because a lot of models and prompts and stuff like, like a lot of things that get sort of there’s like a certain stickiness to it. But I do think, you know, there’s an argument that it’s very easy for model to like, you know, right, today we’re running, you know, X number of 1000 GPUs for one model. Tomorrow we need to shift like, you know, 50% of that to another model.

1:22:30 Sure, that’s just like, that’s what the system is built to handle. No one even needs to tell us to do that. It just magically does that. So you agree that basically the LLMs are commodities? And if you agree, then where do you think the business for the big labs is going to be? I’m not sure if that’s necessarily true, right? Like, is Taylor Swift the commodities? Like, she releases a big album and goes on a tour and makes a lot of money. And then, you know, then there’s like a down the air when she goes and like records, I don’t know, this is best analogy. And then she releases a new album, you know, so I don’t know, like she still has like a lot of brand and there’s a lot of, you know, power and like makes a lot of margin. So like, as long as the labs like repeatedly can launch new models, and as long as the margin that they have on those models are such that they can sort of pay off amortized like that the cost of training those models, it could still be like a decent business, I think. So you don’t think that the intelligence is like plateauing? No, I don’t think so. I mean, I think it remains to be seen, right? Like, models are today are like very good at like doing things that can be trained very efficiently, you know, using reinforcement learning. So we’ve seen that initially with, you know, coding and writing simple snippets over time, it’s gotten really good at also writing production like level code. It’s gotten really good at math. It’s gotten really good at cybersecurity. Like these are all like very quite verifiable things. I still think it remains to be seen, you know, our model is going to be really good at whatever, cooking food or I don’t know, like things that I have like more

1:24:00 like diffuse targets, maybe longer feedback loops or whatever, I think humans probably like my feeling is like will be quite good and quite competitive with models for a long time in those domains. But you know, certainly, you know, AI models, they’re getting very good as I’m sure everyone can agree with. Yeah, so obviously, like the models are getting really good at coding, they’re already really good at coding. What other applications do you think maybe that aren’t as big right now are going to become huge over the next few years? For the LLMs? I don’t know, but I think anything that has like kind of a verifiable computational bio, I think you mentioned. Yeah, computational bio, I think those are like, not exactly verifiable, but there’s like certain ways you can sort of run simulate, like you can use an LLM to output predictions, then you can do like computational like, like methods to like simulate molecular dynamics and get like kind of feedback loops. I mean, like I think like material science is like maybe another area where like you have a little bit of that loop where you can sort of, you know, synthetically manufacture certain materials and measure the properties and use kind of put an agent in the loop and like, you know, do things. So, you know, I have a physics background, I’m pretty excited about that. So drugs, materials, what else? I’m drawing a blank, but I’m sure there’s many other things. Cool. I mean, those are all the questions that I had. Is there anything you wanted to talk about that I haven’t? I’m good. This is great. This is a lot of fun. Awesome. Thank you.