正在加载视频...

视频加载失败

返回首页

Jesse Pollak on the ecosystem forming around tokenized inference "There's this really cool tokenized inference market that Venice has innovated, where they've taken inference and turned it into a token called DIEM where you can buy it and you get $1 of inference per day" "That innovation of tokenizing...

MarketBubble's profile picture

Market Bubble

96,433 subscribers

25,780 次观看 • 6 天前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”
0:52

IN NEWS: Baseten raises a $150M series D round. Tuhin Srivastava (Founder & CEO, Baseten) on the future of inference: “I think the token price goes down and inference should get cheaper over time. And that really just means there is going to be more inference.” “Every time we lower prices or optimize models to make it cheaper, four months later customers are spending more anyway.” “Inference prices will go down, but if the world is run by AI in 10 years, there is going to be a lot of inference. It better be cheap.”

TBPN

17,057 次观看 • 10 个月前

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering
27:58

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 次观看 • 4 个月前

Free, instant access to 1M inference tok/day is excited to launch Cerebras Supernova! Developers get 1M free inference toks/day, and if you're a startup, get up to 3 months FREE of increased context windows, rate limits, and 245M tok/day. All on the world's fastest inference provider. This is the most generous free tier that exists. It really is land of the free...(credits)🇺🇸
0:33

Free, instant access to 1M inference tok/day is excited to launch Cerebras Supernova! Developers get 1M free inference toks/day, and if you're a startup, get up to 3 months FREE of increased context windows, rate limits, and 245M tok/day. All on the world's fastest inference provider. This is the most generous free tier that exists. It really is land of the free...(credits)🇺🇸

Sarah Chieng

122,158 次观看 • 1 年前

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."
1:02

"Inference, if you look at it as a market, will be much, much bigger than cloud computing was pre-ChatGPT." Lightspeed’s Bucky Moore says inference is an underrated investment category in AI, and expects the market to break up into large, specialized platforms for each modality: "The GPU supply crunch that we're seeing right now is largely, as Dylan Patel has said on the show before, due to the fact that not only these consumer products, but also B2B products like Claude Code and Codex are just really taking off and creating insane demand for inference." "We're talking hundreds of billions in spend every year. And if that's true, I think there will be very, very large inference platforms built in each modality." "So there will be an inference platform for real-time video models, there will be an inference platform for open-source and custom language models, there will be an inference platform built specifically for long-running agents." "So I think we're just going to see that industry, which today looks like one industry, break up into many because of how big it is and how much room for specialization there is."

TBPN

33,278 次观看 • 4 个月前

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!
0:56

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 次观看 • 2 年前

I'll let Noam Brown (OpenAI research) answer Yann's “told you so” again. There is nothing to add. There is no wall, there is no end in sight! "Now, we are in a world, like I said earlier, where the amount of compute that's going into pretraining for things large language models is very, very high. But the inference costs are very low. And there was a reasonable concern among various people that we were going to start seeing diminishing returns from AI progress because the costs and the amount of data that you need for pretraining would become so astronomical. And I think the really important takeaway from o1 is that that wall doesn't actually exist, that we can actually push this a lot further. Because, now, we can scale up inference compute. And there's so much room to scale up inference compute."
0:46

I'll let Noam Brown (OpenAI research) answer Yann's “told you so” again. There is nothing to add. There is no wall, there is no end in sight! "Now, we are in a world, like I said earlier, where the amount of compute that's going into pretraining for things large language models is very, very high. But the inference costs are very low. And there was a reasonable concern among various people that we were going to start seeing diminishing returns from AI progress because the costs and the amount of data that you need for pretraining would become so astronomical. And I think the really important takeaway from o1 is that that wall doesn't actually exist, that we can actually push this a lot further. Because, now, we can scale up inference compute. And there's so much room to scale up inference compute."

Chubby♨️

132,808 次观看 • 1 年前

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."
2:32

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

TBPN

30,486 次观看 • 2 个月前

.carried_no_interest says Manus is one of the most underrated AI products today, and that it was a genius acquisition by Zuck: “There are simply things that you can do with Manus that no other inference provider or product can do. I would reveal them, but that’s alpha for me.” “What Zuck realized is — 'I don’t have to care about inference or who’s giving it to me, because [Manus] figured out some really special stuff around how an LLM processes data in and around a web browser.'” “And now he’s already integrating it with Ads Manager. It’s a great bet.”
3:23

.carried_no_interest says Manus is one of the most underrated AI products today, and that it was a genius acquisition by Zuck: “There are simply things that you can do with Manus that no other inference provider or product can do. I would reveal them, but that’s alpha for me.” “What Zuck realized is — 'I don’t have to care about inference or who’s giving it to me, because [Manus] figured out some really special stuff around how an LLM processes data in and around a web browser.'” “And now he’s already integrating it with Ads Manager. It’s a great bet.”

TBPN

37,131 次观看 • 4 个月前

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.
0:25

AI answers your questions in seconds, but behind that speed is something called inference—the compute-intensive process where trained models generate responses. At AWS, we've built custom chips like Trainium, intelligent routing systems, and unified infrastructure to make inference faster and more affordable. As AI agents handle complex multi-step tasks, inference accounts for 80-90% of AI computing power. We're engineering at planetary scale to keep those milliseconds reliable.

Amazon Web Services

27,021 次观看 • 5 个月前

Ansem shares his thesis on $VVV “Venice AI is essentially a private inference platform… you can access all of the top state-of-the-art models, all the best open source models but through Venice, it doesn’t store any of your information” “They’ve done really well. They have over 2 million users. I think they have 50k daily actives on their app and they’re making money from the subscription” “One of the cool features of Venice that I thought was interesting is they have a second token that’s called DIEM… if you have 1 DIEM, you get to use $1 of compute per day for that 1 DIEM that you have” “It’s a really interesting model. I think it’s honestly attracted a lot of people to stake $VVV. We don’t have a lot of real protocols there in the crypto and AI space that aren’t vapor and are actually doing well”
2:27

Ansem shares his thesis on $VVV “Venice AI is essentially a private inference platform… you can access all of the top state-of-the-art models, all the best open source models but through Venice, it doesn’t store any of your information” “They’ve done really well. They have over 2 million users. I think they have 50k daily actives on their app and they’re making money from the subscription” “One of the cool features of Venice that I thought was interesting is they have a second token that’s called DIEM… if you have 1 DIEM, you get to use $1 of compute per day for that 1 DIEM that you have” “It’s a really interesting model. I think it’s honestly attracted a lot of people to stake $VVV. We don’t have a lot of real protocols there in the crypto and AI space that aren’t vapor and are actually doing well”

Market Bubble

142,486 次观看 • 2 个月前

Erik Voorhees on Venice and the convergence thesis: "Venice is a private, uncensored version of ChatGPT with crypto ethos baked in." "DIEM token holders get a dollar per day of free compute, zero marginal cost inference for agents." "Cryptocurrency was actually just being built for the AI robots. Humans were always awkward with it. Agents won't be."
1:44

Erik Voorhees on Venice and the convergence thesis: "Venice is a private, uncensored version of ChatGPT with crypto ethos baked in." "DIEM token holders get a dollar per day of free compute, zero marginal cost inference for agents." "Cryptocurrency was actually just being built for the AI robots. Humans were always awkward with it. Agents won't be."

The Rollup

47,691 次观看 • 2 个月前

INTERVIEW - Venice Is Here to Win: How Private AI Takes On OpenAI and Anthropic David Hoffman sits down with Jon ShapeShift and Jesse Proudman to unpack how Venice is taking on OpenAI and Anthropic with private AI. They talk about: - why AI is becoming the biggest data honeypot ever - private Grok access through Venice - unrestricted AI as a consumer wedge - how non-crypto users still feed the $VVV economy - $DIEM, tokenized inference, and AI agents --- TIMESTAMPS 0:00 Intro 1:12 Why Private AI Matters 3:01 The AI Data Honeypot 6:07 Venice’s Consumer AI Ambition 8:01 Beating the Big AI Labs 10:19 Model Aggregation and Agentic Chat 12:29 Open Source, Closed Source, and Private Grok 17:14 Who Actually Needs Private AI? 21:58 Venice’s User Base 23:53 What Drove Venice’s Recent Growth 27:21 $VVV, $DIEM, and Tokenized Inference 29:00 Where Venice Gets Its Compute 32:17 Bonding Curves and DIEM Monetary Policy 39:14 Real Tokenomics, Not Just a Pie Chart 45:46 AI Agents Need Inference to Exist 52:54 ShapeShift DNA Inside Venice 55:53 Regulation, AI, and Individual Rights
58:52

INTERVIEW - Venice Is Here to Win: How Private AI Takes On OpenAI and Anthropic David Hoffman sits down with Jon ShapeShift and Jesse Proudman to unpack how Venice is taking on OpenAI and Anthropic with private AI. They talk about: - why AI is becoming the biggest data honeypot ever - private Grok access through Venice - unrestricted AI as a consumer wedge - how non-crypto users still feed the $VVV economy - $DIEM, tokenized inference, and AI agents --- TIMESTAMPS 0:00 Intro 1:12 Why Private AI Matters 3:01 The AI Data Honeypot 6:07 Venice’s Consumer AI Ambition 8:01 Beating the Big AI Labs 10:19 Model Aggregation and Agentic Chat 12:29 Open Source, Closed Source, and Private Grok 17:14 Who Actually Needs Private AI? 21:58 Venice’s User Base 23:53 What Drove Venice’s Recent Growth 27:21 $VVV, $DIEM, and Tokenized Inference 29:00 Where Venice Gets Its Compute 32:17 Bonding Curves and DIEM Monetary Policy 39:14 Real Tokenomics, Not Just a Pie Chart 45:46 AI Agents Need Inference to Exist 52:54 ShapeShift DNA Inside Venice 55:53 Regulation, AI, and Individual Rights

Bankless

59,022 次观看 • 1 个月前

Lightchain AI – Decentralized Inference Demo 🍂🥧 + Blockchain Rewards In this video, we showcase how Lightchain AI’s decentralized inference engine (AIVM + PoI + Federated Learning) runs a real-world prompt “What is the best way to make pumpkin pie for thanksgiving dinner?” 💰 Blockchain Rewards in Action This demo doesn’t just show inference — it shows how inference providers are rewarded for their work. The provider that processed this prompt was automatically rewarded on the Lightchain testnet. 🍁 Why It Matters Unlike centralized AI, Lightchain AI distributes inference across a global network of participants. Nodes earn blockchain rewards for providing compute, ensuring a trustless, transparent, and economically sustainable ecosystem. With pay-to-prompt mechanics and DAO governance, Lightchain AI creates a fair AI economy for all. 🌐 Learn more at-
1:37

Lightchain AI – Decentralized Inference Demo 🍂🥧 + Blockchain Rewards In this video, we showcase how Lightchain AI’s decentralized inference engine (AIVM + PoI + Federated Learning) runs a real-world prompt “What is the best way to make pumpkin pie for thanksgiving dinner?” 💰 Blockchain Rewards in Action This demo doesn’t just show inference — it shows how inference providers are rewarded for their work. The provider that processed this prompt was automatically rewarded on the Lightchain testnet. 🍁 Why It Matters Unlike centralized AI, Lightchain AI distributes inference across a global network of participants. Nodes earn blockchain rewards for providing compute, ensuring a trustless, transparent, and economically sustainable ecosystem. With pay-to-prompt mechanics and DAO governance, Lightchain AI creates a fair AI economy for all. 🌐 Learn more at-

Lightchain AI

18,665 次观看 • 10 个月前

Erik Voorhees explains how Venice's dual token model actually works "You can stake 100 VVV to get pro access to Venice. We take a bunch of our revenue, buy the token, and burn it" "We want to take that to its extreme and try to buy every last token we can and burn it all" "You lock up VVV to mint DIEM. Each DIEM you hold gives you $1/day to use on any AI model you want" "We built it so AI agents could hold an asset and get marginally free inference" "It changes the economic calculation. We thought that'd be an interesting design space to explore"
2:23

Erik Voorhees explains how Venice's dual token model actually works "You can stake 100 VVV to get pro access to Venice. We take a bunch of our revenue, buy the token, and burn it" "We want to take that to its extreme and try to buy every last token we can and burn it all" "You lock up VVV to mint DIEM. Each DIEM you hold gives you $1/day to use on any AI model you want" "We built it so AI agents could hold an asset and get marginally free inference" "It changes the economic calculation. We thought that'd be an interesting design space to explore"

Market Bubble

134,785 次观看 • 1 个月前

$NVDA CEO: “Demand for inference will go up by a billion times.” $AMD will be one of the biggest winners as it has the upper hand in inference thanks to larger memory bandwith and capacity of its chips. Stock is still at 22x 2027 earnings. How can’t you be bullish on $AMD?
0:20

$NVDA CEO: “Demand for inference will go up by a billion times.” $AMD will be one of the biggest winners as it has the upper hand in inference thanks to larger memory bandwith and capacity of its chips. Stock is still at 22x 2027 earnings. How can’t you be bullish on $AMD?

Oguz Erkan

282,229 次观看 • 3 个月前

Gavin Baker: “I think Anthropic is worth $3 trillion today.” Gavin laid out the case on the most recent All-In Podcast: – They will end 2026 with over $100B in revenue – They will likely end 2028 with $200-$300B in revenue – At that scale, Anthropic will be very profitable because compute will be inference-dominated – Anthropic reportedly has 85% gross margins on inference – The market has already absorbed Anthropic’s valuation, it will just shift from private to public @jason: “I'm sorry, did you say Anthropic is worth $3 trillion?” Gavin: “Yeah, I think that is roughly where it would probably trade as a public company.” Travis Kalanick: “Wow. Oh man. Holy sh*t.” Gavin: “They're going to end this year with well over $100 billion (in revenue). So what's the '28 number? Is it $200 billion? Is it $300 billion?” “It's probably not going to trade at 10 times that number, and it will be very profitable at that scale because it'll be inference-dominated and people are reporting they have 85% gross margins on inference.” “But in terms of the market absorbing this, like the market's already absorbed it. It's just shifting from private to public.” “And so in the scale of global capital markets, these seem like really big numbers, but you're just moving from the private markets to the public markets, which are even bigger.”
0:55

Gavin Baker: “I think Anthropic is worth $3 trillion today.” Gavin laid out the case on the most recent All-In Podcast: – They will end 2026 with over $100B in revenue – They will likely end 2028 with $200-$300B in revenue – At that scale, Anthropic will be very profitable because compute will be inference-dominated – Anthropic reportedly has 85% gross margins on inference – The market has already absorbed Anthropic’s valuation, it will just shift from private to public @jason: “I'm sorry, did you say Anthropic is worth $3 trillion?” Gavin: “Yeah, I think that is roughly where it would probably trade as a public company.” Travis Kalanick: “Wow. Oh man. Holy sh*t.” Gavin: “They're going to end this year with well over $100 billion (in revenue). So what's the '28 number? Is it $200 billion? Is it $300 billion?” “It's probably not going to trade at 10 times that number, and it will be very profitable at that scale because it'll be inference-dominated and people are reporting they have 85% gross margins on inference.” “But in terms of the market absorbing this, like the market's already absorbed it. It's just shifting from private to public.” “And so in the scale of global capital markets, these seem like really big numbers, but you're just moving from the private markets to the public markets, which are even bigger.”

The All-In Podcast

84,353 次观看 • 27 天前

AI Bull vs Bear #1: 60-70% gross margins on AI inference. Gavin Baker Gavin Baker of Atreides ran the math on BG2 Pod: inference revenue exits 2026 above $200B, and he calls $300B a low estimate for 2027. Ed Zitron's Ed Zitron bear case says costs scale with revenue, so the curve never bends. Baker's number says the unit economics already work at the inference layer. The catch: that is gross margin, not the whole P&L. It does not answer OpenAI's $21B operating loss. Where the bull margin math meets Zitron's receipts:
5:16

AI Bull vs Bear #1: 60-70% gross margins on AI inference. Gavin Baker Gavin Baker of Atreides ran the math on BG2 Pod: inference revenue exits 2026 above $200B, and he calls $300B a low estimate for 2027. Ed Zitron's Ed Zitron bear case says costs scale with revenue, so the curve never bends. Baker's number says the unit economics already work at the inference layer. The catch: that is gross margin, not the whole P&L. It does not answer OpenAI's $21B operating loss. Where the bull margin math meets Zitron's receipts:

Podcast Alpha

15,261 次观看 • 1 个月前

Litman: I think there is no other inference but the owner of the LA Times has decided to curry favor with Trump.. I think they cowered and are worried about their personal holdings.. that is a really shameful capitulation. I felt I couldn't be a part of it and had to resign.
1:45

Litman: I think there is no other inference but the owner of the LA Times has decided to curry favor with Trump.. I think they cowered and are worried about their personal holdings.. that is a really shameful capitulation. I felt I couldn't be a part of it and had to resign.

Acyn

407,943 次观看 • 1 年前

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.
34:46

Ex-vLLM core contributor explained how to make LLM inference 10x cheaper in 34 minutes - better than $3000 inference optimization bootcamps. request comes in -> check LMCache -> hit? load KV cache from CPU/SSD/remote -> skip prefill -> serve. That loop is why Bloomberg and other production stacks now push 300 terabytes of KV cache per week. LMCache + vLLM + CPU/SSD/remote storage + zero-copy CUDA kernels - that's the stack. Watch and save it, then wire the KV-offload into your inference stack.

h100envy

60,618 次观看 • 15 天前

Sam Altman: “By the end of this year, for $100–$1,000 of inference and a good idea, you’ll be able to create software that would have taken teams of people a year to do. That magnitude of economic change is very hard to wrap your head around.”
1:25

Sam Altman: “By the end of this year, for $100–$1,000 of inference and a good idea, you’ll be able to create software that would have taken teams of people a year to do. That magnitude of economic change is very hard to wrap your head around.”

vitrupo

579,103 次观看 • 6 个月前