Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

I spent my summer building TinyTPU : An open source ML inference and training chip. it can do end to end inference + training ENTIRELY on chip. here's how I did it👇:

surya

3,563 subscribers

355,648 views • 11 months ago •via X (Twitter)

Health & Wellness Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

🚀 Excited to release LongLive 2.0! 🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference. ⚡45.7 FPS generation speed on 5B model⚡ ✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment. 🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference. 🙌 Welcome to check it out, try it, and share feedback! 🔗 Code: 📰 Paper: 🎥 Demo: #LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA

Yukang Chen

58,880 views • 2 months ago

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

What is AI inference engineering, why is it such an in-demand skill, and how do you break into the field? With author of Inference Engineering Philip Kiely and head of training at Baseten Charlie O'Neill 0:00: What is inference? 2:47: History of inference 4:59: Downstream effects of AI research on inference 13:54: What you'll learn from Inference Engineering 16:14: Advice for engineers transitioning into AI 19:00: Open source models driving inference growth 20:55: Specialization vs. frontier closed models 23:51: "Big Token" and the importance of open source AI 27:18: Where to get Inference Engineering

Madison Kanna

119,268 views • 3 months ago

"The exponential growth curve for many years to come is on the inference side." — Mark Lohmeyer, VP/GM, AI and Computing Infrastructure, Google Cloud Training gets the attention. Inference is the actual bet. Patrick Moorhead and Mark Lohmeyer on why Google built a separate chip for it.

"The exponential growth curve for many years to come is on the inference side." — Mark Lohmeyer, VP/GM, AI and Computing Infrastructure, Google Cloud Training gets the attention. Inference is the actual bet. Patrick Moorhead and Mark Lohmeyer on why Google built a separate chip for it.

Six Five Media

58,024 views • 2 months ago

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

Long video generation is a systems problem. Introducing LongLive-2.0 from NVIDIA Research: an end-to-end NVFP4 training and inference system for long video generation. Low-precision deployment often relies on post-training quantization, creating a gap between how models are trained and how they run. LongLive-2.0 aligns NVFP4-aware training, distillation, and W4A4 inference, maintaining strong benchmark quality while improving speed and memory efficiency.

NVIDIA AI

60,547 views • 1 month ago

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.

Open source like DeepSeek doesn’t shrink markets. It grows them. Faster, cheaper training means rising inference demand and a new era of AI hardware.

The Linux Foundation

32,930 views • 1 year ago

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla

Mario Nawfal

225,142 views • 8 months ago

Watch and listen to this video carefully to the end. Imagine I made Dwayne Johnson say absolutely EVERYTHING I wanted in less than 5 minutes. And anyone can do the same. Here's how I did it: OPEN THIS ↓

Watch and listen to this video carefully to the end. Imagine I made Dwayne Johnson say absolutely EVERYTHING I wanted in less than 5 minutes. And anyone can do the same. Here's how I did it: OPEN THIS ↓

FELIX

134,147 views • 1 year ago

Recently, Evan lin designed an open-source TPU (from scratch). With Instachip, YOU can "pip install" Evan's TinyTPU into your own project! How? Check it out 👇

Recently, Evan lin designed an open-source TPU (from scratch). With Instachip, YOU can "pip install" Evan's TinyTPU into your own project! How? Check it out 👇

Sathvik Redrouthu

24,444 views • 1 year ago

Releasing kyutai pocket TTS inference in webgpu today — it’s open-source, here’s a demo from my phone It took a while and a bunch of code to implement streaming inference from scratch!

Releasing kyutai pocket TTS inference in webgpu today — it’s open-source, here’s a demo from my phone It took a while and a bunch of code to implement streaming inference from scratch!

Eric Zhang

22,085 views • 5 months ago

If you're curious how I managed to do over $20,000 in inference on the last 48 hours, here's a video all about it. Spoiler: loops are really powerful

If you're curious how I managed to do over $20,000 in inference on the last 48 hours, here's a video all about it. Spoiler: loops are really powerful

Theo - t3.gg

333,275 views • 29 days ago

Our first open-source release. YOLO26-MLX, native YOLO26 on Apple Silicon. No PyTorch. No external GPU. Up to 2.6x faster inference. Up to 1.7x faster training. Accuracy within 0.2% of official results. It won't be the last.

Our first open-source release. YOLO26-MLX, native YOLO26 on Apple Silicon. No PyTorch. No external GPU. Up to 2.6x faster inference. Up to 1.7x faster training. Accuracy within 0.2% of official results. It won't be the last.

webAI

96,990 views • 3 months ago

I built an open source TPU chip in 4 days 8x8 systolic array, 16 instruction ISA, 64 MACs/cycle all in ~3,400 lines of SystemVerilog I sent it to Tiny Tapeout for fabrication soon here's the full arch

I built an open source TPU chip in 4 days 8x8 systolic array, 16 instruction ISA, 64 MACs/cycle all in ~3,400 lines of SystemVerilog I sent it to Tiny Tapeout for fabrication soon here's the full arch

Jaber

29,079 views • 5 months ago

After five days of training sessions, I FINALLY DID IT. UI Hidden and predicting Sigillum's HP to end with Ultimate finish. #wutheringwaves #wuwabattle

After five days of training sessions, I FINALLY DID IT. UI Hidden and predicting Sigillum's HP to end with Ultimate finish. #wutheringwaves #wuwabattle

Rexlent

169,224 views • 2 months ago

📣 ANNOUNCEMENT DAY AT CEREBRAS 📣 Today, we are thrilled to share some of the biggest announcements in our company’s history. 📢 Cerebras announces CS-3, the world’s fastest AI Chip with a whopping 4 trillion transistors 📢 Cerebras selects Qualcomm to deliver unprecedented performance in AI Inference 📢 Cerebras and G42 break ground on Condor Galaxy 3, an 8 exaFLOPs AI Supercomputer Read all about it! 📰 CS-3 Press Release: 📰 Cerebras + Qualcomm Press Release: 📰 Condor Galaxy 3 Press Release: #AI #Supercomputer #ExaFLOPs #ML #Training #Inference

📣 ANNOUNCEMENT DAY AT CEREBRAS 📣 Today, we are thrilled to share some of the biggest announcements in our company’s history. 📢 Cerebras announces CS-3, the world’s fastest AI Chip with a whopping 4 trillion transistors 📢 Cerebras selects Qualcomm to deliver unprecedented performance in AI Inference 📢 Cerebras and G42 break ground on Condor Galaxy 3, an 8 exaFLOPs AI Supercomputer Read all about it! 📰 CS-3 Press Release: 📰 Cerebras + Qualcomm Press Release: 📰 Condor Galaxy 3 Press Release: #AI #Supercomputer #ExaFLOPs #ML #Training #Inference

Cerebras

128,577 views • 2 years ago

Vibe coded this customizable video game entirely with AI. You can add custom nano banana tiles and 3D models from Tripo. It's open sourced (repo at the end) and here's how it works 👇

Vibe coded this customizable video game entirely with AI. You can add custom nano banana tiles and 3D models from Tripo. It's open sourced (repo at the end) and here's how it works 👇

TechHalla

96,354 views • 4 months ago

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵

Google Magenta Project

112,452 views • 1 month ago

I will never forget Roki’s contributions to the 2025 chip and how electric it was. Hope my man figures it out

I will never forget Roki’s contributions to the 2025 chip and how electric it was. Hope my man figures it out

Nate

49,949 views • 3 months ago

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

AI inference is already live on UOMI Router. Access frontier open-source models through a decentralized inference network with: • Low-cost inference • OpenAI-compatible APIs • Verifiable compute • On-chain settlement Start building: And if you have idle GPUs, there's now a second way to participate. The UOMI Provider whitelist is officially open. Instead of letting your hardware sit unused, you can contribute GPU power to the UOMI Inference Network and earn from real AI demand. Join the Provider whitelist:

Uomi

18,816 views • 27 days ago