Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

◈@Exla_ai speeds up AI on edge devices (e.g. NVIDIA Jetsons) with advanced quantization. Reduce memory use by up to 80% and boost inference speed by 3x–20x— all with just a few lines of code. Congrats on the launch, Viraat & Pranav!

Y Combinator

1,650,673 subscribers

20,168 views • 1 year ago •via X (Twitter)

Education Science & Technology News & Politics

Anya Rossi• Live Now

Private livecam show

5 Comments

Adhityaa1 year ago

@exla_ai @therealviraat @pranawnayr the pure text landing page goes hard the underlying tech? even harder

ksminnovation1 year ago

Dr. Tal Patalon explores how AI mega initiatives like the $500B Stargate project & AI-powered genomic analysis by Illumina & NVIDIA are transforming healthcare. @TalPatalon @forbes @edengallery_ #AIinHealthcare #PrecisionMedicine #Multiomics #HealthTech

Joshua Martinez1 year ago

@exla_ai @therealviraat @pranawnayr this is insane, you guys make it look like black magic

Rebis AI | GoldRock AI🏆 | AI Dev Services🇺🇸1 year ago

@exla_ai @therealviraat @pranawnayr Me gusta - how does it work?

Zhongjie Chen1 year ago

@exla_ai @therealviraat @pranawnayr SF little poser @Rippling Jonathan Nasca calbar #321176 moves to my Chinese neighborhood in oakland putting up a BLM sign. Tells me how to pick up dog shit, tries to fight me on the street. SWAT me and tells my parents he had the DA put arrest warrant on their son during lockdown

Related Videos

Sygaldry (Sygaldry) is addressing the rising compute costs and energy requirements for AI by building quantum-accelerated AI servers to exponentially speed up training and inference. Congrats to @chadrigetti and @idaliafriedson on the launch!

Sygaldry (Sygaldry) is addressing the rising compute costs and energy requirements for AI by building quantum-accelerated AI servers to exponentially speed up training and inference. Congrats to @chadrigetti and @idaliafriedson on the launch!

Y Combinator

38,150 views • 1 year ago

Whisper 🤝 Torch Compile Speed-up inference by a factor of 4x with just 2 additional lines of code No accuracy degradation. No additional training. Just kernels 😍 Code example in thread 👇

Whisper 🤝 Torch Compile Speed-up inference by a factor of 4x with just 2 additional lines of code No accuracy degradation. No additional training. Just kernels 😍 Code example in thread 👇

Sanchit Gandhi

12,875 views • 2 years ago

.Piris Labs delivers low-latency, low-cost AI inference by eliminating the memory wall using full-stack optical interconnects and a purpose-built software stack. Congrats on the launch Ali & Keyvan!

.Piris Labs delivers low-latency, low-cost AI inference by eliminating the memory wall using full-stack optical interconnects and a purpose-built software stack. Congrats on the launch Ali & Keyvan!

Y Combinator

26,889 views • 7 months ago

.RunAnywhere makes it easy for app developers to run AI models directly on phones and edge devices — fast, private, offline — with a single SDK and a dashboard to manage everything across user devices. Congrats on the launch Sanchit monga and Shubham Malhotra!

.RunAnywhere makes it easy for app developers to run AI models directly on phones and edge devices — fast, private, offline — with a single SDK and a dashboard to manage everything across user devices. Congrats on the launch Sanchit monga and Shubham Malhotra!

Y Combinator

69,732 views • 8 months ago

.NVIDIA AI kindly sent me a DGX Spark! As soon as I got back from SF, I unboxed it and wanted to share a first look with you. It measures just 15 × 15 × 5.05 cm and weighs 1.2 kg. Inside: • NVIDIA GB10 Grace Blackwell Superchip • 128 GB of unified memory shared by the CPU and GPU • Up to 1 petaflop of theoretical AI compute at FP4 precision with sparsity According to NVIDIA, this little machine can run models with up to 200 billion parameters locally with suitable quantization, and fine-tune models with up to 70 billion parameters using memory-efficient methods such as QLoRA. You can use it to fine-tune models on your own data, customize image generation, and build AI assistants that run locally. I’m looking forward to trying some of those workflows myself. Thanks again to the NVIDIA team for sending this over (NVIDIA RTX Spark)! Once I’m back home, I’ll record some hands-on demos of interesting use cases and share what I learn with you.

.NVIDIA AI kindly sent me a DGX Spark! As soon as I got back from SF, I unboxed it and wanted to share a first look with you. It measures just 15 × 15 × 5.05 cm and weighs 1.2 kg. Inside: • NVIDIA GB10 Grace Blackwell Superchip • 128 GB of unified memory shared by the CPU and GPU • Up to 1 petaflop of theoretical AI compute at FP4 precision with sparsity According to NVIDIA, this little machine can run models with up to 200 billion parameters locally with suitable quantization, and fine-tune models with up to 70 billion parameters using memory-efficient methods such as QLoRA. You can use it to fine-tune models on your own data, customize image generation, and build AI assistants that run locally. I’m looking forward to trying some of those workflows myself. Thanks again to the NVIDIA team for sending this over (NVIDIA RTX Spark)! Once I’m back home, I’ll record some hands-on demos of interesting use cases and share what I learn with you.

Chubby♨️

129,604 views • 5 days ago

Fastshot (Fastshot) lets you build and ship beautiful mobile apps in minutes, just by chatting with AI. No code required. Congrats on the launch, Elvira D and Dmitry Fatkhi!

Fastshot (Fastshot) lets you build and ship beautiful mobile apps in minutes, just by chatting with AI. No code required. Congrats on the launch, Elvira D and Dmitry Fatkhi!

Y Combinator

29,193 views • 10 months ago

NVIDIA sent us 2 DGX Sparks. For a while we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.

NVIDIA sent us 2 DGX Sparks. For a while we wondered what we would do with them. The memory bandwidth is 273GB/s making it 3x slower than an M3 Ultra (819GB/s) for batch_size=1 inference. But it has 4x more FLOPS (100 TFLOPS compared to 26 TFLOPS). So we thought, what if we could combine the DGX Spark & M3 Ultra, and make use of both the massive compute on the DGX Spark and the massive memory-bandwidth on the M3 Ultra. We came up with a way to split inference across both devices and achieve a speedup of up to 4x for long prompts compared to the M3 Ultra on its own. Full details in the blog post linked below.

Alex Cheema

281,225 views • 11 months ago

Introducing Shred API Build apps at the speed of RISE. Achieve real-time UX with just a few lines of code. Only on RISE

Introducing Shred API Build apps at the speed of RISE. Achieve real-time UX with just a few lines of code. Only on RISE

RISE

38,296 views • 1 year ago

.MochaCare (YC W26) unlocks home care agency growth with human virtual assistants supercharged by AI. They handle hiring, scheduling, or both—while surfacing growth insights. Congrats on the launch, Nick Walker and Pranav!

.MochaCare (YC W26) unlocks home care agency growth with human virtual assistants supercharged by AI. They handle hiring, scheduling, or both—while surfacing growth insights. Congrats on the launch, Nick Walker and Pranav!

Y Combinator

22,361 views • 6 months ago

First hands-on with Microsoft’s new Surface Laptop Ultra. Microsoft is clearly positioning this as a new class of creator and AI laptop, powered by new NVIDIA silicon with an RTX GPU built for local AI, creative workflows, and gaming. A few standout specs: -New NVIDIA chip with RTX GPU -Up to 1 petaflop of AI compute -Up to 128GB unified memory -15-inch mini-LED PixelSense Ultra touchscreen -3:2 aspect ratio -262 PPI -Up to 2,000 nits peak HDR brightness -Less than 18mm thick

First hands-on with Microsoft’s new Surface Laptop Ultra. Microsoft is clearly positioning this as a new class of creator and AI laptop, powered by new NVIDIA silicon with an RTX GPU built for local AI, creative workflows, and gaming. A few standout specs: -New NVIDIA chip with RTX GPU -Up to 1 petaflop of AI compute -Up to 128GB unified memory -15-inch mini-LED PixelSense Ultra touchscreen -3:2 aspect ratio -262 PPI -Up to 2,000 nits peak HDR brightness -Less than 18mm thick

Chubby♨️

65,993 views • 3 months ago

Hideo Kojima on AI "A lot of people use AI in creative work to come up with ideas, but I think of AI as more of a friend ... I would lead the creative part and use AI to boost efficiency"

Hideo Kojima on AI "A lot of people use AI in creative work to come up with ideas, but I think of AI as more of a friend ... I would lead the creative part and use AI to boost efficiency"

Pirat_Nation 🔴

173,356 views • 11 months ago

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 views • 2 years ago

STS-133 launch anniversary. Flagship Discovery on her 39th and final launch, with added drama by picking up the count with just two seconds before a scrub would have been called. Epic NASA (JSC SE&I) video. Still one of the best launch videos of all time:

STS-133 launch anniversary. Flagship Discovery on her 39th and final launch, with added drama by picking up the count with just two seconds before a scrub would have been called. Epic NASA (JSC SE&I) video. Still one of the best launch videos of all time:

NSF - NASASpaceflight.com

41,799 views • 1 year ago

Polymarket just launched Bitcoin and crypto perpetuals with up to 20x leverage. It has up to $1 million of liquidity on both sides with just a few dollars slippage. While still in limited beta. 👀👀👀👀

Polymarket just launched Bitcoin and crypto perpetuals with up to 20x leverage. It has up to $1 million of liquidity on both sides with just a few dollars slippage. While still in limited beta. 👀👀👀👀

Car

30,846 views • 3 months ago

We are getting 2 new devices from NVIDIA and Microsoft: 1. The DGX Station, with GB300 superchip and up to 748GB of memory. 2. The RTX Spark laptop, with 1 petaflop of AI performance and 128GB of unified memory. The hardware here is pretty bonkers! Think about it this way:

We are getting 2 new devices from NVIDIA and Microsoft: 1. The DGX Station, with GB300 superchip and up to 748GB of memory. 2. The RTX Spark laptop, with 1 petaflop of AI performance and 128GB of unified memory. The hardware here is pretty bonkers! Think about it this way:

Santiago

15,307 views • 3 months ago

The Context Company (The Context Company) is redefining observability for AI agents. Catch silent failures like bad tool calls, infinite loops, and hallucinations in just 10 lines of code. Congrats on the launch, Rohil Agarwal & arman!

The Context Company (The Context Company) is redefining observability for AI agents. Catch silent failures like bad tool calls, infinite loops, and hallucinations in just 10 lines of code. Congrats on the launch, Rohil Agarwal & arman!

Y Combinator

49,670 views • 11 months ago

🚨The launch of the day!! Xiaomi just unveiled the Xring O100 (玄戒O100) - first dedicated high-bandwidth AI accelerator for edge large models - Vertically stacks a 6nm NPU compute wafer with dual high-speed DRAM memory wafers (Wafer-on-Wafer 3D stacking). - Uses advanced Hybrid Bonding instead of conventional micro-bumps. •- Bonding pitch shrinks from ~50 μm all the way to 1.4 μm. - Creates 2.58 million high-speed physical pathways (TSV diameter just 0.7 μm). - Face-to-Face metal-layer direct connections open ultra-short “data wells” straight into the compute cores. - Paired with Xiaomi’s self-designed high-bandwidth matrix bus (HB-Matrix) and 14-core NPU, data flows between the NPUs and memory at extreme speed with dynamic topology switching (ring for prefill, broadcast for decode). - 1.22 TB/s near-memory bandwidth — 16× a flagship phone’s LPDDR5X - Up to 330 tokens/s edge inference (3B model in Xiaomi lab tests) - Designed for on-device LLMs for following Applications: - Smartphones - PCs - Robots - Xiaomi cars / intelligent driving systems - AI Cube prototype (paired with Xring O3 + D100), for models up to 120B parameters locally at up to 150W sustained 🗓️ The chip has completed silicon validation and is scheduled for commercial use in 2027. #Xiaomi #XRing #DRAM

🚨The launch of the day!! Xiaomi just unveiled the Xring O100 (玄戒O100) - first dedicated high-bandwidth AI accelerator for edge large models - Vertically stacks a 6nm NPU compute wafer with dual high-speed DRAM memory wafers (Wafer-on-Wafer 3D stacking). - Uses advanced Hybrid Bonding instead of conventional micro-bumps. •- Bonding pitch shrinks from ~50 μm all the way to 1.4 μm. - Creates 2.58 million high-speed physical pathways (TSV diameter just 0.7 μm). - Face-to-Face metal-layer direct connections open ultra-short “data wells” straight into the compute cores. - Paired with Xiaomi’s self-designed high-bandwidth matrix bus (HB-Matrix) and 14-core NPU, data flows between the NPUs and memory at extreme speed with dynamic topology switching (ring for prefill, broadcast for decode). - 1.22 TB/s near-memory bandwidth — 16× a flagship phone’s LPDDR5X - Up to 330 tokens/s edge inference (3B model in Xiaomi lab tests) - Designed for on-device LLMs for following Applications: - Smartphones - PCs - Robots - Xiaomi cars / intelligent driving systems - AI Cube prototype (paired with Xring O3 + D100), for models up to 120B parameters locally at up to 150W sustained 🗓️ The chip has completed silicon validation and is scheduled for commercial use in 2027. #Xiaomi #XRing #DRAM

Neil Shah

69,363 views • 1 month ago

Halluminate (Halluminate) provides data and environments to train computer-use AI. Model labs and enterprises partner with Halluminate to accelerate the development of frontier computer/browser use agents. Congrats on the launch, Jerry Wu & wyatt marshall!

Halluminate (Halluminate) provides data and environments to train computer-use AI. Model labs and enterprises partner with Halluminate to accelerate the development of frontier computer/browser use agents. Congrats on the launch, Jerry Wu & wyatt marshall!

Y Combinator

29,216 views • 1 year ago