正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

🖥️ Fixed horizontal screen tearing on NVIDIA! Lead Graphics Developer kd-11 fixed a bug in texture cleanup that led to screen tearing on slow GPU drivers, impacting NVIDIA users. RPCS3 now handles driver management on a separate thread, processing driver stalls asynchronously.

RPCS3

96,735 subscribers

138,447 次观看 • 1 个月前 •via X (Twitter)

科学技术游戏

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

We improved the accuracy of z-culling emulation, thanks to kd-11, who fixed lighting issues in many games such as: Resistance 3, Killzone 2 & 3, and Ryu ga Gotoku Ishin, which previously rendered light through objects Here’s a comparison showing the fix in action on Yakuza Ishin

We improved the accuracy of z-culling emulation, thanks to kd-11, who fixed lighting issues in many games such as: Resistance 3, Killzone 2 & 3, and Ryu ga Gotoku Ishin, which previously rendered light through objects Here’s a comparison showing the fix in action on Yakuza Ishin

RPCS3

216,708 次观看 • 7 个月前

When Tenna switches through channels rapidly at the end of his fight, on release there was a bug for PC caused by using certain GPUs that would display texture data on the screen instead of the intended static. This has since been fixed.

When Tenna switches through channels rapidly at the end of his fight, on release there was a bug for PC caused by using certain GPUs that would display texture data on the screen instead of the intended static. This has since been fixed.

Daily Toby Trivia (Chapter 6 2027)

395,454 次观看 • 10 个月前

This video is a perfect example of why Ontario has a law compelling drivers to move over for emergency vehicles and tow trucks. On January 2nd, shortly before 11:30am, #OttawaOPP were on scene of a collision on #Hwy417 near Bayshore when a vehicle collided with a tow truck! The vehicle drove past two cruisers, on the shoulder, and struck the tow truck. Thankfully, nobody was hurt. The driver now faces a $490 fine and three demerit points. If you see an emergency vehicle or tow truck with their lights on, slow down and move over. #OttTraffic ^mf

This video is a perfect example of why Ontario has a law compelling drivers to move over for emergency vehicles and tow trucks. On January 2nd, shortly before 11:30am, #OttawaOPP were on scene of a collision on #Hwy417 near Bayshore when a vehicle collided with a tow truck! The vehicle drove past two cruisers, on the shoulder, and struck the tow truck. Thankfully, nobody was hurt. The driver now faces a $490 fine and three demerit points. If you see an emergency vehicle or tow truck with their lights on, slow down and move over. #OttTraffic ^mf

OPP East Region

72,131 次观看 • 8 个月前

This Nvidia GPU farm sits in a spare room and prints $18,000 a month Ten cards running 24 hours a day, seven days a week. The setup cost $120,000 to build but it paid for itself in seven months. It does not mine crypto. It rents compute to AI companies that need processing power right now. Companies pay by the hour and the demand never stops. At full capacity the farm pulls $18,000 a month after electricity costs. The owner does not touch it. It just runs. Nvidia GPUs are the most in-demand piece of hardware on the planet right now. The companies that figured this out two years ago are already sitting on serious passive income. The barrier to entry is high but the people inside are not leaving. Follow if you want to understand where the real AI money is actually going.

This Nvidia GPU farm sits in a spare room and prints $18,000 a month Ten cards running 24 hours a day, seven days a week. The setup cost $120,000 to build but it paid for itself in seven months. It does not mine crypto. It rents compute to AI companies that need processing power right now. Companies pay by the hour and the demand never stops. At full capacity the farm pulls $18,000 a month after electricity costs. The owner does not touch it. It just runs. Nvidia GPUs are the most in-demand piece of hardware on the planet right now. The companies that figured this out two years ago are already sitting on serious passive income. The barrier to entry is high but the people inside are not leaving. Follow if you want to understand where the real AI money is actually going.

winkle.

22,626 次观看 • 3 个月前

Every Amazon, Etsy, Shopify, and eBay seller faces the same pain: You have product photos, but making videos is expensive and slow. I fixed that with Pollo AI's "Photo to Video Ads" on the mobile app. One photo of a beach dress → a professional video ad in minutes. No editing. No team. Pollo Agent is my team now. Seriously impressive. 🤯

Every Amazon, Etsy, Shopify, and eBay seller faces the same pain: You have product photos, but making videos is expensive and slow. I fixed that with Pollo AI's "Photo to Video Ads" on the mobile app. One photo of a beach dress → a professional video ad in minutes. No editing. No team. Pollo Agent is my team now. Seriously impressive. 🤯

Zahray Ai

23,575 次观看 • 3 个月前

Suppose two cars are traveling on an expressway at 100 km/h with a 50-meter gap. If the front car slams on its brakes, it takes about 3.5 seconds and nearly 50 meters to stop. If the driver behind does nothing, a collision occurs in exactly 3.5 seconds. This gives the rear driver a 3.5-second window to avoid a crash. For an alert driver, 1 second is reaction time, leaving 2.5 seconds to brake and stop safely. Now, consider the same 50-meter gap, but the front car is reversing at 10 km/h. For the car approaching from behind at 100 km/h, the closing speed skyrockets to 110 km/h, slashing the time before collision to just 1.6 seconds. In reality, it is even worse: because drivers never expect a vehicle to reverse on an expressway, their brains freeze. Reaction time jumps from 1 second to 2, 3, or more. At high speeds, drivers often crash before even realizing they need to brake. That's why reversing on an expressway is hundred times more dangerous than a sudden braking. A maneuver that feels slow to the reversing driver creates an unexpectedly violent closing speed for those behind, leaving virtually no time to react. In the video, a family of seven missed their exit on the Dehradun expressway to Haridwar. They reversed, were struck from behind, and four of them died.

Suppose two cars are traveling on an expressway at 100 km/h with a 50-meter gap. If the front car slams on its brakes, it takes about 3.5 seconds and nearly 50 meters to stop. If the driver behind does nothing, a collision occurs in exactly 3.5 seconds. This gives the rear driver a 3.5-second window to avoid a crash. For an alert driver, 1 second is reaction time, leaving 2.5 seconds to brake and stop safely. Now, consider the same 50-meter gap, but the front car is reversing at 10 km/h. For the car approaching from behind at 100 km/h, the closing speed skyrockets to 110 km/h, slashing the time before collision to just 1.6 seconds. In reality, it is even worse: because drivers never expect a vehicle to reverse on an expressway, their brains freeze. Reaction time jumps from 1 second to 2, 3, or more. At high speeds, drivers often crash before even realizing they need to brake. That's why reversing on an expressway is hundred times more dangerous than a sudden braking. A maneuver that feels slow to the reversing driver creates an unexpectedly violent closing speed for those behind, leaving virtually no time to react. In the video, a family of seven missed their exit on the Dehradun expressway to Haridwar. They reversed, were struck from behind, and four of them died.

THE SKIN DOCTOR

547,307 次观看 • 2 个月前

I explored a further possibility with local models: Qwen3.6 35B A3B + NVIDIA LocateAnything-3B as a local Computer Use agent (proof of concept). In the demo, I asked it to switch my Mac to light mode. It did. Then back to dark. Did that too — finding the right toggle in System Settings, clicking it, and verifying the change itself. It's fully screenshot-based, so no Accessibility API needed. If it's on screen, the agent can see it and act on it. This runs entirely on your own hardware — private, local, built from two small open models.

I explored a further possibility with local models: Qwen3.6 35B A3B + NVIDIA LocateAnything-3B as a local Computer Use agent (proof of concept). In the demo, I asked it to switch my Mac to light mode. It did. Then back to dark. Did that too — finding the right toggle in System Settings, clicking it, and verifying the change itself. It's fully screenshot-based, so no Accessibility API needed. If it's on screen, the agent can see it and act on it. This runs entirely on your own hardware — private, local, built from two small open models.

stevibe

44,242 次观看 • 3 个月前

As a graphics engine coder I think when you look at a flickering bug like this one in the video below it’s not immediately obvious what is going on. The key here is observation - to study this flickering/bugged render carefully - what do we see? Firstly for me it was very obvious that nearly all of the scene shadows were flashing on and off - but (but!) there was a secondary issue where some buildings and parts of the sky were also flashing purple. Hmmmm. Interesting. I initially thought then this might be two separate bugs - but because the sky purple element could only based on full screen post fx and not 3D rendering I looked at this first with a few GPU captures to step through all our post processing to find the rendering stage which made these pixels turn purple: When I did this I found the colour 3D texture LUT grading that makes our different biomes have unique colour palettes was going very wrong - colours near 0 or 1 were wrapping and making the purple elements that we see in the said sky and base parts. The only way this could happen was if the texture was corrupt (which it was not) or if the 3D texture sampling was wrapping and not clamped as intended. That was the Eureka moment - because if the post fx had the wrong texture sampler then the disappearing shadows which also require an exact texture sampler for comparing depth might be also wrong because of the same kind of texture sampling issue! So with this idea that the engine was using the wrong texture samplers, but only in very high draw call scenes like the big base here I the looked at some engine limits and found the bug very quickly - a circular dx12 descriptor buffer for samplers running out over multiple frames, reusing the wrong data for new scenes inflight. Hence the flickering, as the GPU randomly got wrong samplers for some post textures or shadow depth. Easy to fix with triple limits for future expansion and also adding an assert/debug spam in case this limit is ever reached again - QA testers would see this message and report if they ever saw a flicker with this style of bug. My bug and my bad from 2017 porting NMS to DX12 without foreseeing how massively complex bases and our game would grow.

As a graphics engine coder I think when you look at a flickering bug like this one in the video below it’s not immediately obvious what is going on. The key here is observation - to study this flickering/bugged render carefully - what do we see? Firstly for me it was very obvious that nearly all of the scene shadows were flashing on and off - but (but!) there was a secondary issue where some buildings and parts of the sky were also flashing purple. Hmmmm. Interesting. I initially thought then this might be two separate bugs - but because the sky purple element could only based on full screen post fx and not 3D rendering I looked at this first with a few GPU captures to step through all our post processing to find the rendering stage which made these pixels turn purple: When I did this I found the colour 3D texture LUT grading that makes our different biomes have unique colour palettes was going very wrong - colours near 0 or 1 were wrapping and making the purple elements that we see in the said sky and base parts. The only way this could happen was if the texture was corrupt (which it was not) or if the 3D texture sampling was wrapping and not clamped as intended. That was the Eureka moment - because if the post fx had the wrong texture sampler then the disappearing shadows which also require an exact texture sampler for comparing depth might be also wrong because of the same kind of texture sampling issue! So with this idea that the engine was using the wrong texture samplers, but only in very high draw call scenes like the big base here I the looked at some engine limits and found the bug very quickly - a circular dx12 descriptor buffer for samplers running out over multiple frames, reusing the wrong data for new scenes inflight. Hence the flickering, as the GPU randomly got wrong samplers for some post textures or shadow depth. Easy to fix with triple limits for future expansion and also adding an assert/debug spam in case this limit is ever reached again - QA testers would see this message and report if they ever saw a flicker with this style of bug. My bug and my bad from 2017 porting NMS to DX12 without foreseeing how massively complex bases and our game would grow.

Martin Griffiths

72,855 次观看 • 1 年前

A bus belonging to Stallion Cruise (No. 79), travelling towards Victoria Falls, overtook four Painted Dog Conservation vehicles displaying hazard lights shortly after Gwayi Bridge. The area is clearly marked with multiple wildlife warning signs, yet the driver overtook on a blind rise. This kind of driving puts passengers, other road users and wildlife at unnecessary risk. We urge all drivers travelling through wildlife areas to slow down, obey road rules and remember that arriving safely is more important than arriving quickly. Zimbabwe Republic Police

A bus belonging to Stallion Cruise (No. 79), travelling towards Victoria Falls, overtook four Painted Dog Conservation vehicles displaying hazard lights shortly after Gwayi Bridge. The area is clearly marked with multiple wildlife warning signs, yet the driver overtook on a blind rise. This kind of driving puts passengers, other road users and wildlife at unnecessary risk. We urge all drivers travelling through wildlife areas to slow down, obey road rules and remember that arriving safely is more important than arriving quickly. Zimbabwe Republic Police

EXPLOREREYES

68,017 次观看 • 2 个月前

No fewer than one driver has been confirmed dead following a multiple articulated vehicle collision in the early hours of Friday at Otedola Bridge, inward Secretariat, along the Lagos–Ibadan Motorway. The deceased, yet to be identified, was the driver of a sand-laden trailer involved in a collision with another heavy-duty truck conveying granite. The crash reportedly occurred after both drivers struggled for right of way. Officials said the sand truck driver was trapped in the wreckage and sustained fatal injuries. Operatives of the Lagos State Traffic Management Authority (LASTMA) Rescue Team extricated the victim and handed him over to personnel of the Lagos State Ambulance Service (LASAMBUS), who conveyed him to the General Hospital, Ikeja, where he was pronounced dead. The driver of the granite-laden trailer was rescued unhurt. A separate incident about 50 metres away, involving a trailer carrying flour that overturned, further compounded traffic congestion in the area, though no additional fatalities were recorded. The Special Adviser to the Governor on Transportation, Sola Giwa, described the fatal crash as “deeply regrettable and largely preventable,” attributing it to impatience, indiscipline, and refusal to yield by drivers on the motorway. He extended condolences to the family of the deceased. Traffic along the corridor was severely disrupted, with nearly 80 per cent of the carriageway obstructed, forcing authorities to divert motorists heading into Lagos.

No fewer than one driver has been confirmed dead following a multiple articulated vehicle collision in the early hours of Friday at Otedola Bridge, inward Secretariat, along the Lagos–Ibadan Motorway. The deceased, yet to be identified, was the driver of a sand-laden trailer involved in a collision with another heavy-duty truck conveying granite. The crash reportedly occurred after both drivers struggled for right of way. Officials said the sand truck driver was trapped in the wreckage and sustained fatal injuries. Operatives of the Lagos State Traffic Management Authority (LASTMA) Rescue Team extricated the victim and handed him over to personnel of the Lagos State Ambulance Service (LASAMBUS), who conveyed him to the General Hospital, Ikeja, where he was pronounced dead. The driver of the granite-laden trailer was rescued unhurt. A separate incident about 50 metres away, involving a trailer carrying flour that overturned, further compounded traffic congestion in the area, though no additional fatalities were recorded. The Special Adviser to the Governor on Transportation, Sola Giwa, described the fatal crash as “deeply regrettable and largely preventable,” attributing it to impatience, indiscipline, and refusal to yield by drivers on the motorway. He extended condolences to the family of the deceased. Traffic along the corridor was severely disrupted, with nearly 80 per cent of the carriageway obstructed, forcing authorities to divert motorists heading into Lagos.

Neusroom

16,298 次观看 • 8 个月前

hey if you have a 3060, or any GPU with 8GB or more sitting in a drawer right now, that thing can run 9 billion parameters of intelligence autonomously. and you don't know it yet. 2 hours ago i posted that 9B hit a ceiling. 2,699 lines across 11 files. blank screen. said the limit for autonomous multifile coding on 9 billion parameters is real. then i audited every file. found 11 bugs. exact file, exact line, exact fix. duplicate variable declarations killing the script loader. a canvas reference never connected to the DOM. enemies with no movement logic. particle systems called on the class instead of the instance. fed that list as a single prompt to the same Qwen 3.5 9B on the same RTX 3060 through Hermes Agent. it fixed all 11. surgically. patch level edits across 4 files. no rewrites. no hallucinated changes. game boots. enemies spawn, move, collide. background renders. particles fire. and here's what nobody is talking about. this is a 9 billion parameter model running a full agentic framework. Hermes Agent with 31 tools. file operations, terminal, browser, code execution. not a single tool call failed. the agent chain never broke. most people think you need 70B+ for reliable tool use. this is 9B on 12 gigs doing it clean. the model didn't fail. my prompting strategy did. the ceiling is not the parameter count. the ceiling is how you prompt it. this is not done. bullets don't fire yet. boss fights need wiring. but the screen that was black 2 hours ago now has a full game rendering in real time. iterating right now. anyone with a GPU from the last 5 years should be paying attention to what is happening right now.

hey if you have a 3060, or any GPU with 8GB or more sitting in a drawer right now, that thing can run 9 billion parameters of intelligence autonomously. and you don't know it yet. 2 hours ago i posted that 9B hit a ceiling. 2,699 lines across 11 files. blank screen. said the limit for autonomous multifile coding on 9 billion parameters is real. then i audited every file. found 11 bugs. exact file, exact line, exact fix. duplicate variable declarations killing the script loader. a canvas reference never connected to the DOM. enemies with no movement logic. particle systems called on the class instead of the instance. fed that list as a single prompt to the same Qwen 3.5 9B on the same RTX 3060 through Hermes Agent. it fixed all 11. surgically. patch level edits across 4 files. no rewrites. no hallucinated changes. game boots. enemies spawn, move, collide. background renders. particles fire. and here's what nobody is talking about. this is a 9 billion parameter model running a full agentic framework. Hermes Agent with 31 tools. file operations, terminal, browser, code execution. not a single tool call failed. the agent chain never broke. most people think you need 70B+ for reliable tool use. this is 9B on 12 gigs doing it clean. the model didn't fail. my prompting strategy did. the ceiling is not the parameter count. the ceiling is how you prompt it. this is not done. bullets don't fire yet. boss fights need wiring. but the screen that was black 2 hours ago now has a full game rendering in real time. iterating right now. anyone with a GPU from the last 5 years should be paying attention to what is happening right now.

Sudo su

684,336 次观看 • 6 个月前

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

ZEUS⚡️

85,803 次观看 • 3 个月前

I found an old screen recording from my iPhone 11 Pro Max with iOS 17 Decided to compare the animations to my iPhone 17 with iOS 27, and I have so many thoughts 😭 Overall, iOS 17 is INFINITELY smoother than iOS 27, purely due to its animation speed. Opening and closing apps is slow and elegant, accompanied by a beautiful wallpaper zoom. 27 is rough, fast, and lacks fluidity. For some reason, they got rid of the wallpaper zoom when opening/closing apps, making everything feel flat. All of that is paired with the atrocious performance of the iOS screen recorder that started with iOS 18 and still hasn’t been fixed. Notice how there are zero jitters on iOS 17, but both jitters and artifact on 27. This is not a hate post. I just wanted to share my thoughts. And no, this isn’t the problem of a “beta”. Stable iOS 26 has the exact same issues. Thanks 💔

I found an old screen recording from my iPhone 11 Pro Max with iOS 17 Decided to compare the animations to my iPhone 17 with iOS 27, and I have so many thoughts 😭 Overall, iOS 17 is INFINITELY smoother than iOS 27, purely due to its animation speed. Opening and closing apps is slow and elegant, accompanied by a beautiful wallpaper zoom. 27 is rough, fast, and lacks fluidity. For some reason, they got rid of the wallpaper zoom when opening/closing apps, making everything feel flat. All of that is paired with the atrocious performance of the iOS screen recorder that started with iOS 18 and still hasn’t been fixed. Notice how there are zero jitters on iOS 17, but both jitters and artifact on 27. This is not a hate post. I just wanted to share my thoughts. And no, this isn’t the problem of a “beta”. Stable iOS 26 has the exact same issues. Thanks 💔

Out of Galaxy

184,020 次观看 • 1 个月前

Fine-tune DeepSeek-OCR on your own language! (100% local) DeepSeek-OCR is a 3B-parameter vision model that achieves 97% precision while using 10× fewer vision tokens than text-based LLMs. It handles tables, papers, and handwriting without killing your GPU or budget. Why it matters: Most vision models treat documents as massive sequences of tokens, making long-context processing expensive and slow. DeepSeek-OCR uses context optical compression to convert 2D layouts into vision tokens, enabling efficient processing of complex documents. The best part? You can easily fine-tune it for your specific use case on a single GPU. I used Unsloth to run this experiment on Persian text and saw an 88.26% improvement in character error rate. ↳ Base model: 149% character error rate (CER) ↳ Fine-tuned model: 60% CER (57% more accurate) ↳ Training time: 60 steps on a single GPU Persian was just the test case. You can swap in your own dataset for any language, document type, or specific domain you're working with. I've shared the complete guide in the next tweet - all the code, notebooks, and environment setup ready to run with a single click. Everything is 100% open-source!

Fine-tune DeepSeek-OCR on your own language! (100% local) DeepSeek-OCR is a 3B-parameter vision model that achieves 97% precision while using 10× fewer vision tokens than text-based LLMs. It handles tables, papers, and handwriting without killing your GPU or budget. Why it matters: Most vision models treat documents as massive sequences of tokens, making long-context processing expensive and slow. DeepSeek-OCR uses context optical compression to convert 2D layouts into vision tokens, enabling efficient processing of complex documents. The best part? You can easily fine-tune it for your specific use case on a single GPU. I used Unsloth to run this experiment on Persian text and saw an 88.26% improvement in character error rate. ↳ Base model: 149% character error rate (CER) ↳ Fine-tuned model: 60% CER (57% more accurate) ↳ Training time: 60 steps on a single GPU Persian was just the test case. You can swap in your own dataset for any language, document type, or specific domain you're working with. I've shared the complete guide in the next tweet - all the code, notebooks, and environment setup ready to run with a single click. Everything is 100% open-source!

Akshay 🚀

126,250 次观看 • 10 个月前

Every website you’ve ever used is broken in a way you never noticed and it’s been this way for 30 years... A Midjourney engineer finally just fixed it. It’s called Pretext: A tiny library that lets websites lay out text the way magazines and newspapers do, with text flowing around images, wrapping into columns, and fitting perfectly into any shape, all at 120fps. This has been basically impossible on the web for 30 years. Every website you’ve ever used relies on the same clunky system from the 90s to figure out where text goes on screen. Pretext bypasses it entirely. 500x faster. The demos look like they shouldn’t be possible in a browser. Go look.

Every website you’ve ever used is broken in a way you never noticed and it’s been this way for 30 years... A Midjourney engineer finally just fixed it. It’s called Pretext: A tiny library that lets websites lay out text the way magazines and newspapers do, with text flowing around images, wrapping into columns, and fitting perfectly into any shape, all at 120fps. This has been basically impossible on the web for 30 years. Every website you’ve ever used relies on the same clunky system from the 90s to figure out where text goes on screen. Pretext bypasses it entirely. 500x faster. The demos look like they shouldn’t be possible in a browser. Go look.

Josh Kale

1,021,525 次观看 • 5 个月前

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

max fu

29,556 次观看 • 2 个月前

The Emergency Response Team of the Uganda Red Cross Society responded to a road traffic accident that occurred on 18 July 2026 at 11:30 PM in Opiro. Road traffic accident involved a Kakise bus traveling from Soroti to Kampala. The driver reportedly lost control after knocking a boda boda cyclist, causing the bus to overturn. 16 casualities were recorded and supported by Uganda Red Cross Society, Ministry of Health- Uganda, & Uganda Police Force All casualties were evacuated to Soroti Regional Referral Hospital (Soroti RRH) for further medical management. #EMS Government of Uganda

The Emergency Response Team of the Uganda Red Cross Society responded to a road traffic accident that occurred on 18 July 2026 at 11:30 PM in Opiro. Road traffic accident involved a Kakise bus traveling from Soroti to Kampala. The driver reportedly lost control after knocking a boda boda cyclist, causing the bus to overturn. 16 casualities were recorded and supported by Uganda Red Cross Society, Ministry of Health- Uganda, & Uganda Police Force All casualties were evacuated to Soroti Regional Referral Hospital (Soroti RRH) for further medical management. #EMS Government of Uganda

Uganda Red Cross Society

44,631 次观看 • 2 个月前

Last night I drove on Jinja road & I must say it’s by the grace of God that you reach your destination when you drive on this road - especially at night! Besides maneuvering an unmarked road amidst potholes, you have to be ware of broken down trucks stuck on the road with no warning signs. As if that’s not enough, the road is full of slow moving sugarcane trucks - most with no tail lights & you barely see them in time, unless you are driving in full beam. And then there’s the high speeding, ever impatient taxi drivers always overtaking in dangerous spots, risking the lives of their passengers, and other road users, Lord! If you survive all that, you’re likely to crash straight into one of those Uganda Police Force road blocks that appear without warning. Kwegamba, that road is a ticking time bomb. PS: My co-driver recorded the video.

Last night I drove on Jinja road & I must say it’s by the grace of God that you reach your destination when you drive on this road - especially at night! Besides maneuvering an unmarked road amidst potholes, you have to be ware of broken down trucks stuck on the road with no warning signs. As if that’s not enough, the road is full of slow moving sugarcane trucks - most with no tail lights & you barely see them in time, unless you are driving in full beam. And then there’s the high speeding, ever impatient taxi drivers always overtaking in dangerous spots, risking the lives of their passengers, and other road users, Lord! If you survive all that, you’re likely to crash straight into one of those Uganda Police Force road blocks that appear without warning. Kwegamba, that road is a ticking time bomb. PS: My co-driver recorded the video.

Kuteesa Tumwiine🇺🇬

107,962 次观看 • 2 年前

HE BUILT A $10,000-TIER ANIMATED SITE WITH CLAUDE CODE - FOR THE COST OF A SUBSCRIPTION What's on screen isn't a landing page with a parallax background It's a fully interactive, scroll-driven site with real-time 3D rendered in the browser What's actually on the page: > 3D product models rotating and reacting to scroll in real time via WebGL > Smooth hover interactions and transitions - no hand-coded keyframes > Cinematic minimal aesthetic that got featured on Awwwards > Typography layering, editorial layouts, everything assembled in one session What it normally takes: > A 3D artist, a motion designer, and a frontend developer > Weeks of handoffs - modelling, exporting, wiring animations, layout, copy > Six separate systems integrated by hand That pipeline was the moat. It's what justified the invoice The price gap: > Studio build at this level: $5,000-10,000+ > Your cost: a Claude subscription Timeline: weeks of production -> a single session Full walkthrough in the article below

HE BUILT A $10,000-TIER ANIMATED SITE WITH CLAUDE CODE - FOR THE COST OF A SUBSCRIPTION What's on screen isn't a landing page with a parallax background It's a fully interactive, scroll-driven site with real-time 3D rendered in the browser What's actually on the page: > 3D product models rotating and reacting to scroll in real time via WebGL > Smooth hover interactions and transitions - no hand-coded keyframes > Cinematic minimal aesthetic that got featured on Awwwards > Typography layering, editorial layouts, everything assembled in one session What it normally takes: > A 3D artist, a motion designer, and a frontend developer > Weeks of handoffs - modelling, exporting, wiring animations, layout, copy > Six separate systems integrated by hand That pipeline was the moat. It's what justified the invoice The price gap: > Studio build at this level: $5,000-10,000+ > Your cost: a Claude subscription Timeline: weeks of production -> a single session Full walkthrough in the article below

monokern

703,487 次观看 • 2 个月前