正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Next up on First Pass: A conversation with Simon Mo from Inferact on Kimi K3. We discuss: - The basics on the K3 model (it's huge!) - Why open weight models are important - The engineering complexity of working with a model this large - Innovative licensing models And more!

Jamin Ball

92,789 subscribers

13,707 次观看 • 2 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

2 条评论

Palak Goel 的头像

Palak Goel2 个月前

@simon_mo_ @inferact @simon_mo_ thanks for coming on again!

Jamin Ball 的头像

Jamin Ball2 个月前

More of Simon's thoughts below

相关视频

vLLM runs on half a million GPUs at any given moment. Most people have never heard of it. Simon Mo, co-founder and CEO of Inferact and lead maintainer of vLLM, sits down with a16z’s Matt Bornstein and Elena Burger to discuss what it takes to actually run open models in production, the advantages of open models, Simon’s mission at Inferact, and more. 00:00 Intro 01:46 When open source became critical infrastructure 08:55 Day zero model releases, and the drama behind them 14:59 What Kimi K3 actually buys you 18:56 Why open model licenses are changing 22:24 The pharmaceutical analogy for funding model training 26:16 If GPUs got 99% cheaper 29:08 Why vLLM, OpenRouter, and Ollama all started before ChatGPT 35:42 Building a company on an open source project 39:48 The inventor of RoPE removing RoPE Simon Mo Matt Bornstein Elena

vLLM runs on half a million GPUs at any given moment. Most people have never heard of it. Simon Mo, co-founder and CEO of Inferact and lead maintainer of vLLM, sits down with a16z’s Matt Bornstein and Elena Burger to discuss what it takes to actually run open models in production, the advantages of open models, Simon’s mission at Inferact, and more. 00:00 Intro 01:46 When open source became critical infrastructure 08:55 Day zero model releases, and the drama behind them 14:59 What Kimi K3 actually buys you 18:56 Why open model licenses are changing 22:24 The pharmaceutical analogy for funding model training 26:16 If GPUs got 99% cheaper 29:08 Why vLLM, OpenRouter, and Ollama all started before ChatGPT 35:42 Building a company on an open source project 39:48 The inventor of RoPE removing RoPE Simon Mo Matt Bornstein Elena

a16z

210,878 次观看 • 2 个月前

Hacktron founder s1r1us on why frontier models are still far ahead of open models for real-world hacking: "The frontier models are insanely capable. There's a huge gap. You don't even want to use Kimi K3 when you have access to GPT 5.6." "We can't really use Opus 5 anymore. Every time there's a new model release, you kind of don't want to work with the previous model." "Open models, we don't really use them. It's incredibly hard to hand hold them to do the exploitation. They're not yet there." Hacktron AI Harsh Jaiswal

Hacktron founder s1r1us on why frontier models are still far ahead of open models for real-world hacking: "The frontier models are insanely capable. There's a huge gap. You don't even want to use Kimi K3 when you have access to GPT 5.6." "We can't really use Opus 5 anymore. Every time there's a new model release, you kind of don't want to work with the previous model." "Open models, we don't really use them. It's incredibly hard to hand hold them to do the exploitation. They're not yet there." Hacktron AI Harsh Jaiswal

MTS

15,156 次观看 • 19 天前

Inferact Co-founder Simon Mo on the trajectory of open and closed models: "What really differentiates open-weight models from closed-weight models, right? In the end, there's not much differentiation. It's more about the distribution strategy and go-to-market strategy... Capability-wise, I don't really see a big gap, not even today." "It's not about just source data. It's not about where to get the data from. Rather, it's who can build the best environment and who can make the most optimization and algorithmic choices to leverage all this learning from this environment." "So the next year is all gonna be about that. It's about how open-weight model labs are differentiating and really getting the model to meet the real world." "Project out in a year, there's not gonna be any difference." Simon Mo

Inferact Co-founder Simon Mo on the trajectory of open and closed models: "What really differentiates open-weight models from closed-weight models, right? In the end, there's not much differentiation. It's more about the distribution strategy and go-to-market strategy... Capability-wise, I don't really see a big gap, not even today." "It's not about just source data. It's not about where to get the data from. Rather, it's who can build the best environment and who can make the most optimization and algorithmic choices to leverage all this learning from this environment." "So the next year is all gonna be about that. It's about how open-weight model labs are differentiating and really getting the model to meet the real world." "Project out in a year, there's not gonna be any difference." Simon Mo

a16z

33,782 次观看 • 2 个月前

With Kimi K3 Day-0 on vLLM: Open Frontier Intelligence for Everyone 🚀 At 2.8 trillion parameters, Moonshot AI's Kimi K3 is one of the most powerful open-weight models ever released. Starting today, you can serve it on vLLM the moment the weights are public. What K3 brings: 🧠 2.8T-parameter Mixture-of-Experts (16 of 896 experts active per token) 📚 1M-token context window 🛠️ Native multimodal understanding, including vision ⚡ Kimi Delta Attention: a hybrid of linear and full attention that makes million-token context affordable Huge thank you to Kimi.ai AI for the model release and partnership, Inferact for leading the vLLM optimizations, and to our partners at NVIDIA, AMD, and the broader vLLM community. 1/6

With Kimi K3 Day-0 on vLLM: Open Frontier Intelligence for Everyone 🚀 At 2.8 trillion parameters, Moonshot AI's Kimi K3 is one of the most powerful open-weight models ever released. Starting today, you can serve it on vLLM the moment the weights are public. What K3 brings: 🧠 2.8T-parameter Mixture-of-Experts (16 of 896 experts active per token) 📚 1M-token context window 🛠️ Native multimodal understanding, including vision ⚡ Kimi Delta Attention: a hybrid of linear and full attention that makes million-token context affordable Huge thank you to Kimi.ai AI for the model release and partnership, Inferact for leading the vLLM optimizations, and to our partners at NVIDIA, AMD, and the broader vLLM community. 1/6

vLLM

104,533 次观看 • 2 个月前

📣 Kimi.ai's Kimi K3, an open-weight model, is now generally available and rolling out in GitHub Copilot. The model shows frontier-level abilities on agentic coding with highly cost-effective pricing. It is hosted by Fireworks. Learn more. 👇

📣 Kimi.ai's Kimi K3, an open-weight model, is now generally available and rolling out in GitHub Copilot. The model shows frontier-level abilities on agentic coding with highly cost-effective pricing. It is hosted by Fireworks. Learn more. 👇

GitHub

236,944 次观看 • 2 个月前

Inferact CEO Simon Mo says open and closed models are converging. The moat is environment: "What really differentiates open-weight models from closed-weight models? In the end, there's not much differentiation. It's more about the distribution strategy and go-to-market strategy. Capability-wise, I don't really see a big gap, not even today." "One of the most important parts is just the data. It's about who gets what data, and then what are the environments you are building to let the model improve on itself." "For Moonshot, they have built some of the best environments for front-end coding... the ability for this model to code and then see what the rendered code is, and then continue looping in this iterative process." "Now, this is about their environment to improve the model. It's not about just source data. It's not about where they get the data from. Rather, it's who can build the best environment and who can make the most optimization and algorithmic choices to leverage all this learning from this environment." "The next year is all gonna be about that: how open-weight model labs are differentiating and really getting the model to meet the real world." Simon Mo

Inferact CEO Simon Mo says open and closed models are converging. The moat is environment: "What really differentiates open-weight models from closed-weight models? In the end, there's not much differentiation. It's more about the distribution strategy and go-to-market strategy. Capability-wise, I don't really see a big gap, not even today." "One of the most important parts is just the data. It's about who gets what data, and then what are the environments you are building to let the model improve on itself." "For Moonshot, they have built some of the best environments for front-end coding... the ability for this model to code and then see what the rendered code is, and then continue looping in this iterative process." "Now, this is about their environment to improve the model. It's not about just source data. It's not about where they get the data from. Rather, it's who can build the best environment and who can make the most optimization and algorithmic choices to leverage all this learning from this environment." "The next year is all gonna be about that: how open-weight model labs are differentiating and really getting the model to meet the real world." Simon Mo

a16z

39,424 次观看 • 1 个月前

Next up on First Pass: A conversation with Jesse Zhang from Decagon about how everyone is wrong about open source AI Decagon runs ~90% of their workloads on open source models, however their spend on open models fell to 11% (from 19% a year ago).

Next up on First Pass: A conversation with Jesse Zhang from Decagon about how everyone is wrong about open source AI Decagon runs ~90% of their workloads on open source models, however their spend on open models fell to 11% (from 19% a year ago).

Jamin Ball

16,087 次观看 • 1 个月前

2 years ago, Anthropic CEO Dario Amodei warned the U.S. Congress: “The scaling of open-source models is going down a very dangerous path…” Today, Kimi K3 is already competing with frontier models. Seeing what this model can do, I think we’re getting closer to open-source models reaching Mythos-level capabilities with little to no guardrails. That’s a pretty scary thought.

2 years ago, Anthropic CEO Dario Amodei warned the U.S. Congress: “The scaling of open-source models is going down a very dangerous path…” Today, Kimi K3 is already competing with frontier models. Seeing what this model can do, I think we’re getting closer to open-source models reaching Mythos-level capabilities with little to no guardrails. That’s a pretty scary thought.

m0h

148,005 次观看 • 2 个月前

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

learned a lot from this conversation with Simon Mo and Matt Bornstein. biggest takeaways for me: -there are a lot of reasons why we should like open-weight models. a lot of these arguments stop at handwavy things like "what if the labs stop releasing frontier models to the public" or "it's lower cost." but simon's position as lead maintainer of vLLM and CEO of Inferact give him authority to talk about some of the other, more interesting and concrete reasons to pay attention to open-weight models, namely that they allow end-users to calibrate latency / other performance metrics with way more customizability than what any of the frontier closed-source labs offer (and without the fear that your job might be met with a refusal at some random point where you're deep in a 2 hour job) -re: the above point...for this reason, a lot of US companies (inferact included!) choose to use open-weight models over their closed-source alternatives. this also isn't limited to internal workloads / research - on a recent a16z podcast the team at Decagon spoke about how something like 90% of their customer service ai agents run on open-weight models that they've fine-tuned. -we should really appreciate how many companies/teams came out researchers fascinated by the wave of very small open-weight models that were being distilled from e.g. gpt-3.5 and earlier models in 2022/2023 (prior to the release of chatGPT!). these small models motivated the development of pagedattention, which then led to vlmm/inferact (at other layers of the stack with similar origin stories, you can look at teams like openrouter or ollama). in other words, we have open-weight models to thank for a bunch of the orchestration infra we now rely on. i think yet another, indirect, way we can point to open-source/weight infra pushing the frontier forward. anyway, a lot more in this convo, it was a lot of fun!

Elena

12,922 次观看 • 2 个月前

Next up on First Pass: A conversation with Yash Patil from Applied Compute. Customized models are coming. But how can enterprises get ready for this future? We dug in on questions like that with Yash

Next up on First Pass: A conversation with Yash Patil from Applied Compute. Customized models are coming. But how can enterprises get ready for this future? We dug in on questions like that with Yash

Jamin Ball

11,292 次观看 • 8 天前

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Announcing the Open Model Series: a new series on open source AI models, with merve of Hugging Face. Hugging Face is the open model hub: it hosts model weights, datasets, evals, & demo apps, and maintains the open-source libraries (like transformers) used to run and train them. I think of it like GitHub, but for AI models. You can download open models and run them on your own machines. In this series we'll cover what open source and open weight models are, how they work, how to run them yourself, and why they're important. By the way - this video is edited entirely with Claude! I can make a tutorial on how I did it. If there's a question you want answered about open models, leave it in the comments. We will read them all. Episode 1 is out soon.

Harper Carroll

30,392 次观看 • 2 个月前

Kimi K3 just generated a Red Dead Redemption 2-style open-world game... 🤯 A few months ago this would've sounded impossible. Now open-weight models are building explorable worlds with terrain, gameplay systems, and impressive visual fidelity from a single prompt.

Kimi K3 just generated a Red Dead Redemption 2-style open-world game... 🤯 A few months ago this would've sounded impossible. Now open-weight models are building explorable worlds with terrain, gameplay systems, and impressive visual fidelity from a single prompt.

WorldofAI

94,901 次观看 • 2 个月前

Benchmark’s Peter Fenton says 90%+ of tokens could come from open weight models in the next 18-24 months, pressuring frontier model margins. Full convo at 12p PT/3p ET on the livestream, plus Aravind Srinivas on Perplexity’s new orchestrator model and Jeffrey Morgan on why enterprises are moving toward AI models they can download and control

Benchmark’s Peter Fenton says 90%+ of tokens could come from open weight models in the next 18-24 months, pressuring frontier model margins. Full convo at 12p PT/3p ET on the livestream, plus Aravind Srinivas on Perplexity’s new orchestrator model and Jeffrey Morgan on why enterprises are moving toward AI models they can download and control

Deirdre Bosa

64,460 次观看 • 3 个月前

I WAS WRONG, FABLE-5.1 MIGHT ACTUALLY BE THE BEST MODEL TO EXIST ON THIS PLANET FOR FRONTEND TOO 😭 Tried recreating the exact same game with Fable-5.1 and Kimi K3. Fable-5.1 outclassed Kimi K3 on UI and gameplay. Smoother mechanics, better interactions, and noticeably more polished overall. Kimi was still competitive, especially with way fewer jitters during gameplay. Cost: - Kimi K3: ~$11 - Fable-5.1: ~$18 That’s ~60% more cost for Fable, but the quality difference was actually noticeable. Kimi is still ridiculously competitive for the price.

I WAS WRONG, FABLE-5.1 MIGHT ACTUALLY BE THE BEST MODEL TO EXIST ON THIS PLANET FOR FRONTEND TOO 😭 Tried recreating the exact same game with Fable-5.1 and Kimi K3. Fable-5.1 outclassed Kimi K3 on UI and gameplay. Smoother mechanics, better interactions, and noticeably more polished overall. Kimi was still competitive, especially with way fewer jitters during gameplay. Cost: - Kimi K3: ~$11 - Fable-5.1: ~$18 That’s ~60% more cost for Fable, but the quality difference was actually noticeable. Kimi is still ridiculously competitive for the price.

aditya

367,938 次观看 • 1 个月前

Kimi K3 is insane🤯 It built an Animal Crossing-style game, and it generated a fully playable experience with the cozy aesthetic, interactions, and gameplay loop in a single shot. Open-weight models are starting to rival the best closed models for game generation. The pace of progress is getting ridiculous. 🌱🎮

Kimi K3 is insane🤯 It built an Animal Crossing-style game, and it generated a fully playable experience with the cozy aesthetic, interactions, and gameplay loop in a single shot. Open-weight models are starting to rival the best closed models for game generation. The pace of progress is getting ridiculous. 🌱🎮

WorldofAI

451,208 次观看 • 2 个月前

TokenRouter is giving away 50 million free Kimi K3 tokens with no trial period and no credit card, accessible via any OpenAI-compatible tool in 2 minutes. → Sign up on TokenRouter and create a new API key → Add it to any OpenAI-compatible tool with the URL → Set the model to moonshotai/kimi-k3-free and you're running

TokenRouter is giving away 50 million free Kimi K3 tokens with no trial period and no credit card, accessible via any OpenAI-compatible tool in 2 minutes. → Sign up on TokenRouter and create a new API key → Add it to any OpenAI-compatible tool with the URL → Set the model to moonshotai/kimi-k3-free and you're running

0xMarioNawfal

51,457 次观看 • 1 个月前

DeepSeek V4.1 Flash vs Kimi K3 This is not one-shot from DeepSeek btw. The first result was boxy and not realistic, so I iterated with a $6 cap. Kimi did this in one shot, costing $6.95. Not as good as Kimi K3, but it did better than the other Flash models I’ve tried.

DeepSeek V4.1 Flash vs Kimi K3 This is not one-shot from DeepSeek btw. The first result was boxy and not realistic, so I iterated with a $6 cap. Kimi did this in one shot, costing $6.95. Not as good as Kimi K3, but it did better than the other Flash models I’ve tried.

Bhavy☄️

34,313 次观看 • 28 天前

Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding. To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8. We are open sourcing it today. 1/2

Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding. To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8. We are open sourcing it today. 1/2

Inferact

243,166 次观看 • 14 天前

Kimi K3 is the first open source model to earn the BridgeMind stamp of approval. One shot Horror House game. One shot full-stack app. Beating Fable 5 in early BridgeBench arena matches. #3 on the Intelligence Index. Moonshot AI just caught the frontier. Full Kimi K3 review below.

Kimi K3 is the first open source model to earn the BridgeMind stamp of approval. One shot Horror House game. One shot full-stack app. Beating Fable 5 in early BridgeBench arena matches. #3 on the Intelligence Index. Moonshot AI just caught the frontier. Full Kimi K3 review below.

BridgeMind

32,017 次观看 • 2 个月前

Thanks to CNN for having me on to discuss Kimi K3. I told Paula Newton K3’s large size, limited uptime, and the erosion of the U.S. software moat are all redefining the U.S.-China AI race: We are in an industrial systems competition to build and install vast compute.

Thanks to CNN for having me on to discuss Kimi K3. I told Paula Newton K3’s large size, limited uptime, and the erosion of the U.S. software moat are all redefining the U.S.-China AI race: We are in an industrial systems competition to build and install vast compute.

Ryan Fedasiuk

14,702 次观看 • 2 个月前