Загрузка видео...

Не удалось загрузить видео

На главную

Releasing kyutai pocket TTS inference in webgpu today — it’s open-source, here’s a demo from my phone It took a while and a bunch of code to implement streaming inference from scratch!

ekzhang1's profile picture

Eric Zhang

21,504 subscribers

22,085 просмотров • 5 месяцев назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Kyutai Speech-To-Text is now open-source! It’s streaming, supports batched inference, and runs blazingly fast: perfect for interactive applications. Check out the details here:
0:53

Kyutai Speech-To-Text is now open-source! It’s streaming, supports batched inference, and runs blazingly fast: perfect for interactive applications. Check out the details here:

kyutai

66,264 просмотров • 1 год назад

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik
1:21

Releasing moondream-zig! It is a fast, implementation of moondream2 inference on the CPU written from-scratch in Zig :) moondream-zig provides 1.5-2x faster inference compared to huggingface on the same device. moondream vik

snow

29,245 просмотров • 1 год назад

BOOM! Starting today you can use open source frontier LLMs in Visual Studio Code with HF Inference Providers! 🔥 Use your inference credits on SoTA llms like GLM 4.5, Qwen3 Coder, DeepSeek 3.1 and more All of it packaged in one simple extension - try it out today 🤗
1:52

BOOM! Starting today you can use open source frontier LLMs in Visual Studio Code with HF Inference Providers! 🔥 Use your inference credits on SoTA llms like GLM 4.5, Qwen3 Coder, DeepSeek 3.1 and more All of it packaged in one simple extension - try it out today 🤗

Vaibhav (VB) Srivastav

41,185 просмотров • 9 месяцев назад

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering
27:58

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 просмотров • 3 месяцев назад

releasing v0.1 of Papercode today! - a fun platform for getting back to implementing papers from scratch - you can implement papers from scratch using leetcode style interfaces :D - auth, good storage coming soon - this is a barebones structure at this point, so please report any bugs in my dms please do share and use it well :) link in replies
0:26

releasing v0.1 of Papercode today! - a fun platform for getting back to implementing papers from scratch - you can implement papers from scratch using leetcode style interfaces :D - auth, good storage coming soon - this is a barebones structure at this point, so please report any bugs in my dms please do share and use it well :) link in replies

neural nets.

17,523 просмотров • 6 месяцев назад

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference
0:57

Before Fable 5 was shut down, it pushed Gemma 4 to 255 tok/s on WebGPU. Some didn't believe it was real. Today we're releasing the demo and kernels it wrote for you to see yourself. Run it locally in your browser. Agentic kernel optimization is the future of on-device inference

Xenova

358,050 просмотров • 7 дней назад

Introducing Open TTS Tracker! 🗣️ *sound on* A one-stop shop to track all open access/ source TTS models! Ranging from XTTS to Pheme, OpenVoice to VITS, and more... ⚡ For each model, we compile: 1. Souce-code 2. Checkpoints 3. License 4. Fine-tuning code 5. Languages supported 6. Paper 7. Demo Help us make it more complete! Let's 2024 the year of open TTS models! ❤️
1:27

Introducing Open TTS Tracker! 🗣️ *sound on* A one-stop shop to track all open access/ source TTS models! Ranging from XTTS to Pheme, OpenVoice to VITS, and more... ⚡ For each model, we compile: 1. Souce-code 2. Checkpoints 3. License 4. Fine-tuning code 5. Languages supported 6. Paper 7. Demo Help us make it more complete! Let's 2024 the year of open TTS models! ❤️

Vaibhav (VB) Srivastav

68,037 просмотров • 2 лет назад

I spent my summer building TinyTPU : An open source ML inference and training chip. it can do end to end inference + training ENTIRELY on chip. here's how I did it👇:
0:27

I spent my summer building TinyTPU : An open source ML inference and training chip. it can do end to end inference + training ENTIRELY on chip. here's how I did it👇:

surya

354,929 просмотров • 10 месяцев назад

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:
0:53

⚡️Today marks a big milestone for Nebius. We’re launching Nebius Token Factory, the evolution of Nebius AI Studio, built to make open-source AI production-grade. Token Factory transforms raw open models into governed, scalable systems with dedicated inference, sub-second latency, 99.9% uptime and zero-retention compliance. It’s where inference, post-training and governance converge, turning raw compute into reliable intelligence. Run AI inference at scale:

Nebius Token Factory

259,764 просмотров • 7 месяцев назад

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.
0:47

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.

The Linux Foundation

32,930 просмотров • 1 год назад

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇
0:32

Introducing TTS WebGPU: The first ever text-to-speech web app built with WebGPU acceleration! 🔥 High-quality and natural speech generation that runs 100% locally in your browser, powered by OuteTTS and Transformers.js.🤗 Try it out yourself! Demo + source code below 👇

Xenova

19,666 просмотров • 1 год назад

Yesterday: iPad demo Today: iPhone demo Same headless Mac mini setup, but now you can manage AI agents from your pocket. Here’s a quick demo of how I remotely connect from anywhere with my iPhone to monitor what’s happening, and take over when needed
1:18

Yesterday: iPad demo Today: iPhone demo Same headless Mac mini setup, but now you can manage AI agents from your pocket. Here’s a quick demo of how I remotely connect from anywhere with my iPhone to monitor what’s happening, and take over when needed

Matt Ronge

4,706,428 просмотров • 2 месяцев назад

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein
44:14

Today we announced a $150M seed round in Inferact, a new startup led by the maintainers of the vLLM project. Inferact will support the vLLM open source project through dedicated financial and developer resources and build what they see as the next generation commercial inference engine. Cofounders Simon Mo and Woosuk Kwon joined a16z GP Matt Bornstein for a conversation on how vLLM came to be, what they’ve learned from building it, and what they’re now doing at Inferact. They cover how vLLM began as a side project, why open source is essential to the world’s AI infrastructure, why inference is getting harder, and more. 00:00 Introduction 11:41 Community and collaboration in vLLM 19:19 Understanding inference engines 24:27 Cluster scale and GPU deployment 31:19 Belief in open source AI 35:45 Founding Inferact 40:00 The future of Inference at scale Simon Mo Woosuk Kwon Matt Bornstein

a16z

87,993 просмотров • 5 месяцев назад

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!
1:51

New course: Efficient Inference with SGLang: Text and Image Generation, built in partnership with LMSys LMSYS Org and RadixArk RadixArk, and taught by Richard Chen Richard Chen, a Member of Technical Staff at RadixArk. Running LLMs in production is expensive, and much of that cost comes from redundant computation. This short course teaches you to eliminate that waste using SGLang, an open-source inference framework that caches computation already done and reuses it across future requests. When ten users share the same system prompt, SGLang processes it once, not ten times. The speedups compound quickly, especially when there's a lot of shared context across requests. Skills you'll gain: - Implement a KV cache from scratch to eliminate redundant computation within a single request - Scale caching across users and requests with RadixAttention, so shared context is only processed once - Accelerate image generation with diffusion models using SGLang's caching and multi-GPU parallelism Join and learn to make LLM inference faster and more cost-efficient at scale!

Andrew Ng

97,357 просмотров • 2 месяцев назад

Kyutai TTS and Unmute are now open source! The text-to-speech is natural, customizable, and fast: it can serve 32 users with a 350ms latency on a single L40S. Try it out and get started on the project page:
1:12

Kyutai TTS and Unmute are now open source! The text-to-speech is natural, customizable, and fast: it can serve 32 users with a 350ms latency on a single L40S. Try it out and get started on the project page:

kyutai

171,391 просмотров • 11 месяцев назад

Demo of Tether's QVAC running local inference on a mobile device at incredible speed, via llama.cpp + LLAMA 3.2 1B parameters. QVAC is a generalized inference and fine-tuning runtime able to adapt to any device, from smartphones, to laptops and servers. Lots of models being supported already. More to come. No limits. Infinite intelligence. Coming soon
0:53

Demo of Tether's QVAC running local inference on a mobile device at incredible speed, via llama.cpp + LLAMA 3.2 1B parameters. QVAC is a generalized inference and fine-tuning runtime able to adapt to any device, from smartphones, to laptops and servers. Lots of models being supported already. More to come. No limits. Infinite intelligence. Coming soon

Paolo Ardoino 🤖

42,015 просмотров • 11 месяцев назад

Introducing Epic’s version control system: Lore! Built from scratch and open-sourced, we made scalability and performance core tenets from day one. We’re releasing Lore as open source today.All source code, along with documentation of data formats and protocols are released under an MIT license and free to use. Help shape it into something even more powerful, together:
0:52

Introducing Epic’s version control system: Lore! Built from scratch and open-sourced, we made scalability and performance core tenets from day one. We’re releasing Lore as open source today.All source code, along with documentation of data formats and protocols are released under an MIT license and free to use. Help shape it into something even more powerful, together:

Unreal Engine

224,105 просмотров • 7 дней назад

Anthropic CEO Dario Amodei on Open-Source AI Models. "I don't think open source works the same way in AI that it has worked in other areas. Primarily because with open source you can see the source code of the model. Here we can't see inside the model, it's often called open weights instead of open source to kind of distinguish that. But a lot of the benefits, which is that many people can work on it and that it's kind of additive, don't quite work in the same way. So I've actually always seen it as a red herring. When I see a new model come out I don't care whether it's open source or not. If we talk about Deep Seek I don't think it mattered that Deep Seek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That's the only thing that I care about. It actually doesn't matter either way. Because ultimately you have to host it on the cloud. The people who host it on the cloud do inference. These are big models, they're hard to do inference on. When I think about competition I think about which models are good at the tasks that we do. I think open source is actually a red herring. It's not free. You have to run it on inference and someone has to make it fast on inference." --- From 'Alex Kantrowitz' YT channel
1:40

Anthropic CEO Dario Amodei on Open-Source AI Models. "I don't think open source works the same way in AI that it has worked in other areas. Primarily because with open source you can see the source code of the model. Here we can't see inside the model, it's often called open weights instead of open source to kind of distinguish that. But a lot of the benefits, which is that many people can work on it and that it's kind of additive, don't quite work in the same way. So I've actually always seen it as a red herring. When I see a new model come out I don't care whether it's open source or not. If we talk about Deep Seek I don't think it mattered that Deep Seek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That's the only thing that I care about. It actually doesn't matter either way. Because ultimately you have to host it on the cloud. The people who host it on the cloud do inference. These are big models, they're hard to do inference on. When I think about competition I think about which models are good at the tasks that we do. I think open source is actually a red herring. It's not free. You have to run it on inference and someone has to make it fast on inference." --- From 'Alex Kantrowitz' YT channel

Rohan Paul

944,205 просмотров • 7 месяцев назад

Here’s Caleb missing open receivers from a clean pocket
1:56

Here’s Caleb missing open receivers from a clean pocket

Steven Patton

316,877 просмотров • 4 месяцев назад