Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

LLMs have a limited context window. When conversations grow too long this affects output quality, performance and cost. New blog post from Earendil engineer Vegard Stikbakke on how compaction addresses this and how we’ve implemented it in Pi. Read the full post below

Pi

34,743 subscribers

225,445 views • 3 days ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

LongWriter Unleashing 10,000+ Word Generation from Long Context LLMs discuss: Current long context large language models (LLMs) can process inputs up to 100,000 tokens, yet struggle to generate outputs exceeding even a modest length of 2,000 words. Through controlled experiments, we find that the model's effective generation length is inherently bounded by the sample it has seen during supervised fine-tuning (SFT). In other words, their output limitation is due to the scarcity of long-output examples in existing SFT datasets. To address this, we introduce AgentWrite, an agent-based pipeline that decomposes ultra-long generation tasks into subtasks, enabling off-the-shelf LLMs to generate coherent outputs exceeding 20,000 words. Leveraging AgentWrite, we construct LongWriter-6k, a dataset containing 6,000 SFT data with output lengths ranging from 2k to 32k words. By incorporating this dataset into model training, we successfully scale the output length of existing models to over 10,000 words while maintaining output quality. We also develop LongBench-Write, a comprehensive benchmark for evaluating ultra-long generation capabilities. Our 9B parameter model, further improved through DPO, achieves state-of-the-art performance on this benchmark, surpassing even much larger proprietary models. In general, our work demonstrates that existing long context LLM already possesses the potential for a larger output window--all you need is data with extended output during model alignment to unlock this capability.

AK

50,995 views • 2 years ago

This is how easy you add tenting to your keyboard (Go60), while being also sleek, small and portable. A have a few more photos and videos about the tenting in my soon to published blog post.

This is how easy you add tenting to your keyboard (Go60), while being also sleek, small and portable. A have a few more photos and videos about the tenting in my soon to published blog post.

Fatih Arslan

39,267 views • 2 months ago

As promised! New blog with Josh Newton combining our talks from AI Engineer's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

As promised! New blog with Josh Newton combining our talks from AI Engineer's first design track. In addition, we wanted to actually show a demonstration of how you can iterate and improve upon that first design output from AI. We had a ton of fun building the interactions for this post (especially the horse animator hahaha). Would love to hear your thoughts!

Sabina

12,300 views • 13 days ago

I found a CVE in my studio lights💡 ✨【𝒞𝒱𝐸－𝟤𝟢𝟤𝟧－𝟩𝟤𝟢𝟤】✨ I discovered a CSRF vulnerability in Elgato’s Key Lights that allows any website make your Elgato lights flicker aggressively. Want to learn more about how I did this? Read the full blog post below 👇

I found a CVE in my studio lights💡 ✨【𝒞𝒱𝐸－𝟤𝟢𝟤𝟧－𝟩𝟤𝟢𝟤】✨ I discovered a CSRF vulnerability in Elgato’s Key Lights that allows any website make your Elgato lights flicker aggressively. Want to learn more about how I did this? Read the full blog post below 👇

PinkDraconian

51,231 views • 1 year ago

Aave Glass, our take on Apple's Liquid Glass, now works across mobile and web. We've published a post that breaks down how we built Aave Glass and the family of components it now powers. Read it below.

Aave Glass, our take on Apple's Liquid Glass, now works across mobile and web. We've published a post that breaks down how we built Aave Glass and the family of components it now powers. Read it below.

Aave

92,070 views • 2 months ago

In the files of “don’t be this guy” Perfect example of the problem with so many lifters… Ego lifting and caught up in how much weight they have on the bar or machine These are all shitty half ass reps. How about dropping the weight in half and actually performing a full rep below parallel? This needs to be addressed. See this way too often. And don’t comment that I shouldn’t post this. I really don’t care. Same guy who screams and grunts super load on every rep.

In the files of “don’t be this guy” Perfect example of the problem with so many lifters… Ego lifting and caught up in how much weight they have on the bar or machine These are all shitty half ass reps. How about dropping the weight in half and actually performing a full rep below parallel? This needs to be addressed. See this way too often. And don’t comment that I shouldn’t post this. I really don’t care. Same guy who screams and grunts super load on every rep.

BowTiedMeatHead 🥩💪

28,271 views • 4 months ago

🍇 This limited-edition Grape Juicebox case could be yours!🍇 It’s hands-down one of our absolute favorite Juicebox designs to date, and this particular one needs a brand-new home.💜 Want to take it home? Here’s how to enter: - Follow us - Like this post - Tag a friend in the comments below Best of luck to everyone entering!✨️

Ironside Computers

113,571 views • 27 days ago

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Wow. This is one of the best interactive sites I’ve seen for learning how LLMs work! 🔥 It starts w/ a clear intro and guides you through every core component: from Embedding, Layer Norm, and Self-Attention to MLPs, Transformer blocks, Softmax, and Output layers. link in 🧵↓

Charly Wargnier

52,092 views • 1 year ago

The bold part of this variable font was generated with my custom AI harness, img2bez, and a small neural network I trained from scratch. I will have a blog post out Monday explaining the full workflow.

The bold part of this variable font was generated with my custom AI harness, img2bez, and a small neural network I trained from scratch. I will have a blog post out Monday explaining the full workflow.

Eli Heuer

28,626 views • 1 month ago

I like how post Astartes and especially post Secret Level that the collective idea of how Space Marines fight has changed. I just really like this style of it. Precise calculated sudden movements, almost like a machine in how efficient every action is.

I like how post Astartes and especially post Secret Level that the collective idea of how Space Marines fight has changed. I just really like this style of it. Precise calculated sudden movements, almost like a machine in how efficient every action is.

Rylanor, The Ancient of Rites

84,324 views • 8 months ago

this is the kind of video modern day robin buckley would post on social media. nancy comments on how cool it looks, how smart she thinks robin is and then has a gay meltdown in the privacy of her room.

this is the kind of video modern day robin buckley would post on social media. nancy comments on how cool it looks, how smart she thinks robin is and then has a gay meltdown in the privacy of her room.

joey ★ is seeing djo

89,431 views • 5 months ago

So many of you asking me what your first post should be ???? … how about we start the “ tongue 👅 challenge” .. everyone just go post this .. say your name and what you do and then let your tongue 😜 goooooo” everyone is overthinking shit 💩 .. listen to the second video on this post to understand and understand the 80/20 rule of content …too many fear a post not doing well, or the judgement in the comments .. we have to stop that ! #firstpost #inspiration #garyvee

So many of you asking me what your first post should be ???? … how about we start the “ tongue 👅 challenge” .. everyone just go post this .. say your name and what you do and then let your tongue 😜 goooooo” everyone is overthinking shit 💩 .. listen to the second video on this post to understand and understand the 80/20 rule of content …too many fear a post not doing well, or the judgement in the comments .. we have to stop that ! #firstpost #inspiration #garyvee

Gary Vaynerchuk

15,775 views • 1 year ago

Today, two of us who have built billion dollar companies are launching the ULTIMATE guide for founders on how to start a lean, AI-native startup in 2025. This covers every single tool Founders should use when building their startup from the ground up, from legal, finance, payroll, sales, tech and more. My coauthor Henry has built Super and I have built Glean into both $100M+ businesses and we've invested in 100+ startups between us in the last few years, and this is a constant question we get. And it's completely FREE. The article drops midnight tonight. Bookmark this post. Link below:

Today, two of us who have built billion dollar companies are launching the ULTIMATE guide for founders on how to start a lean, AI-native startup in 2025. This covers every single tool Founders should use when building their startup from the ground up, from legal, finance, payroll, sales, tech and more. My coauthor Henry has built Super and I have built Glean into both $100M+ businesses and we've invested in 100+ startups between us in the last few years, and this is a constant question we get. And it's completely FREE. The article drops midnight tonight. Bookmark this post. Link below:

Deedy

323,554 views • 1 year ago

You know how when youre scrolling, aching for a relapse and then there's that one post that sends you over the edge? Well.. this ones it Why? Because pxxxphxxxs like you dont need to be baited twice 🤤 And when youre done dont forget to rue how one little post fucked you up 🤭

Sensitive content

You know how when youre scrolling, aching for a relapse and then there's that one post that sends you over the edge? Well.. this ones it Why? Because pxxxphxxxs like you dont need to be baited twice 🤤 And when youre done dont forget to rue how one little post fucked you up 🤭

Bopper Peddler 🍼

25,322 views • 10 months ago

Just watched a reel of Indian girls partying at a club in NYC. Read the comments, and it's full of racism. Indian girls always post and comment everywhere about how India is the worst and spread racism against Indian men. They don’t realize they will face racism too.

Just watched a reel of Indian girls partying at a club in NYC. Read the comments, and it's full of racism. Indian girls always post and comment everywhere about how India is the worst and spread racism against Indian men. They don’t realize they will face racism too.

︎ ︎venom

649,172 views • 1 year ago

This clip below (quoted post) in the tweet being used by Christian Nationalists or their bots is a lie. They claim this Muslim imam said the constitution should be replaced by the Quran, which is based on false edits and a lie. He didn’t say that — here’s the full context. Watch.

This clip below (quoted post) in the tweet being used by Christian Nationalists or their bots is a lie. They claim this Muslim imam said the constitution should be replaced by the Quran, which is based on false edits and a lie. He didn’t say that — here’s the full context. Watch.

HalalFlow

42,203 views • 8 months ago

👟💪🏻 I’m partnering with TYR Sport to give away a full training bundle — a TYR top, bottom, socks, and a pair of the new CXT-3 trainers! This giveaway is for both men and women! How to enter: ✅ Go to this post on INSTAGRAM 1️⃣ Like the post 2️⃣ Follow me and TYR Sport 3️⃣ Tag your training partner in the comments (each tag = 1 entry) 4️⃣ Share this post to your story for a bonus entry Winner will be picked by TYR Sport on July 10th! Good luck!! #sport #fitness

👟💪🏻 I’m partnering with TYR Sport to give away a full training bundle — a TYR top, bottom, socks, and a pair of the new CXT-3 trainers! This giveaway is for both men and women! How to enter: ✅ Go to this post on INSTAGRAM 1️⃣ Like the post 2️⃣ Follow me and TYR Sport 3️⃣ Tag your training partner in the comments (each tag = 1 entry) 4️⃣ Share this post to your story for a bonus entry Winner will be picked by TYR Sport on July 10th! Good luck!! #sport #fitness

Janna Breslin

19,383 views • 1 month ago

The resistance is mobilizing. We’re meeting in a secret hub, guarded by a mysterious figure named Clyde. We’ve received a transmission: 185209141 This code decrypts into a word - enter it at the hub gates and interact with this post to gain access. Functions for a limited time.

The resistance is mobilizing. We’re meeting in a secret hub, guarded by a mysterious figure named Clyde. We’ve received a transmission: 185209141 This code decrypts into a word - enter it at the hub gates and interact with this post to gain access. Functions for a limited time.

PROJECT NOVA

48,907 views • 3 years ago

Can somebody at NYC DOT please explain to me how this is even remotely acceptable? With the new parking lane, cars can't pull out of the way to let this ambulance pass. When every second counts in an emergency, this could literally mean death for someone in a crisis. I have heard from so many people about how this Court Street redesign was a terrible idea, and this really seals the deal. Someone could be dead because of this. Local Law 6 of 2025 - which I introduced, and was passed - mandates that local firehouses MUST be consulted before changes like this go into effect. Did the local firehouse sign off on this? New York Post Metro New York Post Elle McLogan CBS New York Eyewitness News FOX 5 NY PIX11 News Eyewitness News

Can somebody at NYC DOT please explain to me how this is even remotely acceptable? With the new parking lane, cars can't pull out of the way to let this ambulance pass. When every second counts in an emergency, this could literally mean death for someone in a crisis. I have heard from so many people about how this Court Street redesign was a terrible idea, and this really seals the deal. Someone could be dead because of this. Local Law 6 of 2025 - which I introduced, and was passed - mandates that local firehouses MUST be consulted before changes like this go into effect. Did the local firehouse sign off on this? New York Post Metro New York Post Elle McLogan CBS New York Eyewitness News FOX 5 NY PIX11 News Eyewitness News

Joann Ariola NYC Council District 32

130,520 views • 10 months ago

we've rolled out a new experimental feature to bankr club members! airdrops right in the social feed! this is a great way to engage with your audience. so how does it work? step 1: create a post for people to reply to step 2: quote post your original post with your airdrop instructions. oh and by the way .. comment "ok bankr" below — first 100 people get $5 of $BNKR airdropped.

we've rolled out a new experimental feature to bankr club members! airdrops right in the social feed! this is a great way to engage with your audience. so how does it work? step 1: create a post for people to reply to step 2: quote post your original post with your airdrop instructions. oh and by the way .. comment "ok bankr" below — first 100 people get $5 of $BNKR airdropped.

Bankr

39,001 views • 1 year ago