Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Inferact cofounder Simon Mo: “I fundamentally believe that open source, especially how vLLM itself is structured, is critical to the AI infrastructure in the world. And what we want to do with Inferact is to support, maintain, steward and push forward the open source ecosystem.” “Open source moves so... show more

a16z

936,334 subscribers

29,383 views • 6 months ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

a16z

87,993 views • 6 months ago

In an AI world, effort is no longer the bottleneck—ideas are. Open-source AI enables anyone to have access to build anything they want without depending on the one-sided gates that closed-source offers. This is amplified by the open innovation that open-source development enables.

In an AI world, effort is no longer the bottleneck—ideas are. Open-source AI enables anyone to have access to build anything they want without depending on the one-sided gates that closed-source offers. This is amplified by the open innovation that open-source development enables.

Sentient

17,392 views • 5 months ago

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

We co-led Inferact's $150M seed round to support them in their mission to build the inference engine for all current and future AI. In this episode of The Investment Memo, Lightspeed's Bucky Moore and James Alcorn sit down with Simon Mo (Co-Founder & CEO Inferact) to cover: - How vLLM grew to 60K+ GitHub stars - Why inference is shifting to the majority of compute - How vLLM evolved from a research project into the industry standard - Why building a company was the next step to push open-source inference forward 00:00 Introduction 02:03 The investment memo 04:47 Latency vs throughput vs cost 06:19 Paged attention explained 08:04 The evolution of attention 09:42 Growing the vLLM open source community 11:41 Working with hardware vendors 14:45 Deploying vLLM at large scale 16:03 Inferact's culture of openness 18:45 Building an open ecosystem and horizontal stack 19:45 Inferact's approach to fundraising 22:14 What is the future of inference? Simon Mo Bucky Moore James Alcorn

Lightspeed

27,040 views • 6 months ago

ELON: CHINA HAS THE BEST OPEN SOURCE AI MODELS RIGHT NOW “The best open source models are generally from China, which is bizarre. I think the second best one, or maybe it’s better than second best, is Grok 2.5. The open source model is actually very good, and we will continue to open source our models.” Source: The All-In Podcast

ELON: CHINA HAS THE BEST OPEN SOURCE AI MODELS RIGHT NOW “The best open source models are generally from China, which is bizarre. I think the second best one, or maybe it’s better than second best, is Grok 2.5. The open source model is actually very good, and we will continue to open source our models.” Source: The All-In Podcast

Mario Nawfal

1,163,582 views • 9 months ago

Tobi talking about making pi-autoresearch open-source: "I do not believe in software as owned" "Open-source is about making gifts to the world"

Tobi talking about making pi-autoresearch open-source: "I do not believe in software as owned" "Open-source is about making gifts to the world"

David Cortés

66,390 views • 3 months ago

We are now Cult.Repo. The original creators behind the iconic Open Source documentaries (Vue, React, GraphQL) are back together! Honeypot is now called Cult.Repo and is independently-owned. We are on a mission to document the complete history of every major open source language ever created. With 250K subscribers and over 15M views, we're building on our foundation as creators of the acclaimed Open Source Origin Story documentary format. Coming in 2025: Python Software Foundation, Vite ⚡ documentaries and much more. Our independence is core to our mission. We believe the human stories behind open source deserve authentic, in-depth storytelling. Our new model is powered directly by: - Community support from viewers - Sponsorships from organizations that champion open source This transition represents our commitment to independent, meaningful tech storytelling that honours the collaborative spirit of open source itself.

We are now Cult.Repo. The original creators behind the iconic Open Source documentaries (Vue, React, GraphQL) are back together! Honeypot is now called Cult.Repo and is independently-owned. We are on a mission to document the complete history of every major open source language ever created. With 250K subscribers and over 15M views, we're building on our foundation as creators of the acclaimed Open Source Origin Story documentary format. Coming in 2025: Python Software Foundation, Vite ⚡ documentaries and much more. Our independence is core to our mission. We believe the human stories behind open source deserve authentic, in-depth storytelling. Our new model is powered directly by: - Community support from viewers - Sponsorships from organizations that champion open source This transition represents our commitment to independent, meaningful tech storytelling that honours the collaborative spirit of open source itself.

Cult.Repo

70,793 views • 1 year ago

Open research has a purpose: to facilitate open-source AI through an open economy that serves us all 🌎 We’ve converged years of open research into creating the GRID—a comprehensive, production-grade open-source AI stack that enterprises are adopting every day. This is the future of open-source 😁

Open research has a purpose: to facilitate open-source AI through an open economy that serves us all 🌎 We’ve converged years of open research into creating the GRID—a comprehensive, production-grade open-source AI stack that enterprises are adopting every day. This is the future of open-source 😁

Sentient

56,660 views • 7 months ago

Unitree G1 Open Source Dataset In order to promote the development of the global embodied AI industry, the Unitree G1 robot operation data set is open sourced, adapted to a variety of open source solutions, and continuously updated: Open source data collection: Open source learning algorithms: Open source datasets and models: #AI #Teleoperation #OpenSourceDataset #Unitree #EmbodiedAI #Humanoid #DataCollection #AGI

Unitree G1 Open Source Dataset In order to promote the development of the global embodied AI industry, the Unitree G1 robot operation data set is open sourced, adapted to a variety of open source solutions, and continuously updated: Open source data collection: Open source learning algorithms: Open source datasets and models: #AI #Teleoperation #OpenSourceDataset #Unitree #EmbodiedAI #Humanoid #DataCollection #AGI

Unitree

148,925 views • 1 year ago

.Bill Gurley “I don’t think politicians understand how powerful open source is as a competitive weapon. If you have ten companies all doing open source, the minute an idea is shared, it's available to the other ten and they absorb it immediately. I think the odds that the U.S., with two proprietary models, can stay ahead of China, with ten open source ones, is very low.”

.Bill Gurley “I don’t think politicians understand how powerful open source is as a competitive weapon. If you have ten companies all doing open source, the minute an idea is shared, it's available to the other ten and they absorb it immediately. I think the odds that the U.S., with two proprietary models, can stay ahead of China, with ten open source ones, is very low.”

Codie Sanchez

65,669 views • 4 months ago

"This is just like the space race." - Doug Kelly America must invest in open source and closed source AI to win the future.

"This is just like the space race." - Doug Kelly America must invest in open source and closed source AI to win the future.

American Edge Project

215,647 views • 1 year ago

🛰️ Many countries can't afford billion-dollar space programs. Spacecoin is changing that. We plan to open source our satellite design and technology so that anybody who wants to participate in our open network can do so. Open-source technology makes satellite communications more accessible to all, and we’re leading the charge.

🛰️ Many countries can't afford billion-dollar space programs. Spacecoin is changing that. We plan to open source our satellite design and technology so that anybody who wants to participate in our open network can do so. Open-source technology makes satellite communications more accessible to all, and we’re leading the charge.

Spacecoin™ 🛰️

913,408 views • 7 months ago

It has to be open source. It has to adhere to crypto values. That is the only way to scale. Jon Kol explains Hyperlane's open source ethos ⏬

It has to be open source. It has to adhere to crypto values. That is the only way to scale. Jon Kol explains Hyperlane's open source ethos ⏬

Hyperlane ⏩

10,667 views • 1 year ago

Eron Wolf gives a talk on why open-source is failing and what we can do to fix that

Eron Wolf gives a talk on why open-source is failing and what we can do to fix that

FUTO

13,468 views • 1 year ago

Jason Calacanis: Anthropic, OpenAI and others are trying to kill OpenClaw Why? Because an open source agent platform is an existential threat to frontier model companies. @jason: “ People are going to say I'm a conspiracy theorist, but the number one goal, I believe, in the large language model, frontier model space, is to kill (OpenClaw). This is a giant movement to stop it, because this is the equivalent of having an open source Android-like player in the market, and that could be incredibly disruptive. Because, I believe, open source is going to win the day on the large language models and take 90% of the token usage, and I think the entire frontier model space could be undercut by open source. And I think they realize that SLMs, the smaller language models that are verticalized now, that will run on desktops and laptops and are even starting to run on the top ones, that is their biggest competitive threat, and I hope it happens.”

Jason Calacanis: Anthropic, OpenAI and others are trying to kill OpenClaw Why? Because an open source agent platform is an existential threat to frontier model companies. @jason: “ People are going to say I'm a conspiracy theorist, but the number one goal, I believe, in the large language model, frontier model space, is to kill (OpenClaw). This is a giant movement to stop it, because this is the equivalent of having an open source Android-like player in the market, and that could be incredibly disruptive. Because, I believe, open source is going to win the day on the large language models and take 90% of the token usage, and I think the entire frontier model space could be undercut by open source. And I think they realize that SLMs, the smaller language models that are verticalized now, that will run on desktops and laptops and are even starting to run on the top ones, that is their biggest competitive threat, and I hope it happens.”

The All-In Podcast

64,611 views • 3 months ago

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit

.Marc Andreessen 🇺🇸 says the winner of the US-China AI race could dictate what all 8 billion people on Earth use — but open source introduces a third possibility: "The stakes are basically: what is the world going to run on?" "In the long run, somebody's gonna win. And the world will either be running on American AI or be running on Chinese AI." "The open source thing is super fascinating because it throws a wrench into all of this. It raises a third possibility that neither the US nor China are going to be the platform — it may just be open source." Marc Andreessen with Jeetu Patel at Cisco AI Summit

a16z

101,985 views • 5 months ago

The US government is becoming more involved in controlling AI models, while open-source Chinese models are improving at a rapid pace. Jordi Visser explains why this should push more people to open-source AI in the coming months.

The US government is becoming more involved in controlling AI models, while open-source Chinese models are improving at a rapid pace. Jordi Visser explains why this should push more people to open-source AI in the coming months.

Anthony Pompliano 🌪

48,359 views • 1 month ago

Open source software is GREAT. But "open source" AI is NOT like software - it's VERY different. Rob Miles cuts through the bullshit: ROB: Oh, hey, Meta. I heard Llama's weights leaked. That's rough, man. Information security's hard. How you holding up? META: Oh, we're great. Yeah, we're fine. We... actually, that was deliberate. We meant to do that. ROB MILES: Oh, really? META: Yeah... well, the second time anyway. It's called open source. Look it up. ROB MILES: Oh. Well, I love free and open source software, but do those principles really apply to network weights? How does that work? META: Open source is good for users because it lets them read the source code and see what the program is really doing and how it works. ROB MILES: Wait, have you found a way to tell how a model works by looking at its weights? META: No. But, it lets developers all over the world spot bugs in the code and submit patches. ROB: Wait, people are fixing bugs in Llama's weights? META: Well, no. People can fine tune it themselves, though. ROB: ?? Other companies offer fine tuning through APIs. ... So, hang on, if you can't actually read the code and know what it's doing, then network weights are effectively a compiled binary. So, in what sense is this open source? Why not call it like public weights? Why call it open source at all? META: I love open source. ROB: Well, I know a lot of your employees do, but you don't love anything. You're a giant corporation. What's in it for you? META: I love, love open source.

Open source software is GREAT. But "open source" AI is NOT like software - it's VERY different. Rob Miles cuts through the bullshit: ROB: Oh, hey, Meta. I heard Llama's weights leaked. That's rough, man. Information security's hard. How you holding up? META: Oh, we're great. Yeah, we're fine. We... actually, that was deliberate. We meant to do that. ROB MILES: Oh, really? META: Yeah... well, the second time anyway. It's called open source. Look it up. ROB MILES: Oh. Well, I love free and open source software, but do those principles really apply to network weights? How does that work? META: Open source is good for users because it lets them read the source code and see what the program is really doing and how it works. ROB MILES: Wait, have you found a way to tell how a model works by looking at its weights? META: No. But, it lets developers all over the world spot bugs in the code and submit patches. ROB: Wait, people are fixing bugs in Llama's weights? META: Well, no. People can fine tune it themselves, though. ROB: ?? Other companies offer fine tuning through APIs. ... So, hang on, if you can't actually read the code and know what it's doing, then network weights are effectively a compiled binary. So, in what sense is this open source? Why not call it like public weights? Why call it open source at all? META: I love open source. ROB: Well, I know a lot of your employees do, but you don't love anything. You're a giant corporation. What's in it for you? META: I love, love open source.

AI Notkilleveryoneism Memes ⏸️

107,362 views • 2 years ago

FROM THE LIVE SHOW: Matt Hicks (Matt Hicks), President & CEO of Red Hat, on open source and collaboration. "I love the model that open source provides because it provides this incredible collaboration." "This is why we're passionate about smaller models, because they allow that collaborative, open source ethos to thrive a little bit better than the frontier models."

FROM THE LIVE SHOW: Matt Hicks (Matt Hicks), President & CEO of Red Hat, on open source and collaboration. "I love the model that open source provides because it provides this incredible collaboration." "This is why we're passionate about smaller models, because they allow that collaborative, open source ethos to thrive a little bit better than the frontier models."

Forward Future

12,538 views • 9 months ago