正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Anthropic turned one GitHub request into a living 3D model that answers every new issue with a rebuilt version of itself Opus 5.5 builds the whole system from that first request, from geometry and rig to the viewer and the pipeline behind it. > REQUEST - one issue describes... the object in plain words, nothing else needed > BUILD - mesh, rig and materials generated and committed as editable source files > VIEW - a browser viewer added to the repo so anyone can spin the model instantly > LISTEN - every new issue is read as a change request against the current model > PATCH - only what changed gets rebuilt, then a pull request opens with a fresh render A 3D asset is normally a dead file. Someone exports it once, and every later change means opening the software again. Here the model stays alive inside the repo. Ask for a longer handle or a softer material, and it answers in a pull request. Each pull request carries a new render, so the change gets reviewed by eye before anyone merges it. Because the source is code and not a binary blob, every version is diffable. You see exactly which part moved and why. Rejected changes cost nothing. Close the pull request and the model stays exactly as it was. Teams already run software this way. Running a 3D model like a codebase is the part nobody had wired up. The first request, the repo layout and one full issue-to-render cycle are pinned in the article.show more

wast3

7,217 subscribers

16,350 次观看 • 13 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

18 条评论

Dekos 的头像

Dekos13 天前

I wonder when the development of SHIT will stop.

Insomnia 的头像

Insomnia13 天前

bro I love to see it now

Danylo Fedorenko 的头像

Danylo Fedorenko12 天前

this is such a cool idea. does it rebuild from scratch every time or just patch the old one?

0xbobaa 的头像

0xbobaa12 天前

What kind of repository is this?

Zero 的头像

Zero13 天前

damn, that’s insanely detailed!

shmidt 的头像

shmidt13 天前

cool dot

beamnxw ./ 的头像

beamnxw ./13 天前

insane

Celebs Hub 的头像

Celebs Hub12 天前

The idea that a 3D asset is dead until reopened in software is doing a lot of heavy lifting here; git already did this for code, you just pointed it at a mesh.

kemal buger 的头像

kemal buger13 天前

Evren resmen davetiye göndermiş, FungoLabs daha neyi bekliyor?

magsimich 的头像

magsimich12 天前

Wow this looks good to me

Biscuit 的头像

Biscuit12 天前

client revisions would be so much easier

monokern 的头像

monokern13 天前

new render in every pr is smart

Ridark 的头像

Ridark13 天前

You have a unique terminal with a really interesting approach for real for sure honestly for real, how did you build it out?

Antony Claude 的头像

Antony Claude13 天前

this is pretty cool and very useful for 3d devs

kiosa 的头像

kiosa13 天前

Cool Fam

Lummox 的头像

Lummox13 天前

It’s really pretty

Gipp 🦅 的头像

Gipp 🦅13 天前

plain text requests in opus 5.5 always flag my mesh changes cleaner than my old modeling workflow

Codio 的头像

Codio13 天前

share that github repo with me via dm

相关视频

Anthropic built a model that turns any object into a 3D schematic and then runs experiments on it before anyone touches the real thing. The geometry was never the hard part. The hard part is a model that knows which parts of the object are allowed to move. > SHAPE - the object rebuilt as parts with real dimensions, nothing smoothed over > JOINT - every connection marked with what it permits and what it forbids > LOAD - forces applied the way the object actually meets them in use > BREAK - the weakest path traced until something gives, over and over > SWAP - a single part changed and the whole run repeated from scratch > RANK - failure modes sorted by how little force it takes to trigger them A render shows what something looks like. A schematic with constraints shows what it cannot survive. Simulation tools already do this, and they need weeks of setup by someone who knows the software cold. Here the setup is the description. The constraints come from what the object is, not from a form someone fills in. The swap step is where it earns its place. You change one part and get the new failure order in minutes, not days. Most prototypes exist to find the first thing that breaks. This finds it, then the next four behind it. Every failure path and the force that caused it sits in the breakdown below.

Anthropic built a model that turns any object into a 3D schematic and then runs experiments on it before anyone touches the real thing. The geometry was never the hard part. The hard part is a model that knows which parts of the object are allowed to move. > SHAPE - the object rebuilt as parts with real dimensions, nothing smoothed over > JOINT - every connection marked with what it permits and what it forbids > LOAD - forces applied the way the object actually meets them in use > BREAK - the weakest path traced until something gives, over and over > SWAP - a single part changed and the whole run repeated from scratch > RANK - failure modes sorted by how little force it takes to trigger them A render shows what something looks like. A schematic with constraints shows what it cannot survive. Simulation tools already do this, and they need weeks of setup by someone who knows the software cold. Here the setup is the description. The constraints come from what the object is, not from a form someone fills in. The swap step is where it earns its place. You change one part and get the new failure order in minutes, not days. Most prototypes exist to find the first thing that breaks. This finds it, then the next four behind it. Every failure path and the force that caused it sits in the breakdown below.

wast3

101,950 次观看 • 15 天前

Fusion Models by robinhood:0xa92768863a55d8a0591709f7f5e594a249d36ea3, explained properly. Why a panel beats one model: Ask three models the same hard question and they disagree. The disagreement is the information, where they contradict each other is where one of them is wrong, and a judge can see it. OpenRouter shipped exactly this in June and their panels beat any single frontier model. Credit to them and their $10B sale to stripe. Why the same panel for everyone is the wrong shape: You don't have one best model. You have a best model per job. The one you trust for code is not the one you trust for writing, and neither is the one you'd pick for a five-line reply. A panel chosen per request, identical for every user, can't know that. A panel arranged around you can. The number that matters isn't price: It's cost per answer that works. Model A costs $4 per million tokens and gets your job right one time in four. Model B costs $10 and gets it right three times in four. Per answer you actually keep, A costs $16 and B costs $13. The cheap model was the expensive one. Everyone comparing price lists is measuring the wrong thing. You never rate anything: Stopping an answer halfway is a rating. Regenerating is a rating. Switching model and asking again is the loudest rating there is. Every one of those already writes a row we can learn from. The router learns which models earn your credits, per job, and tries a cheaper one now and then to check. Fusion only when it pays: Cost mode tries the cheap model first and escalates only if a judge says it failed. Performance mode fans out to your panel and merges. Most simple prompts never fuse. That's why it costs less, not more. Put it together and you have a personal model: Not trained. Assembled. Your profile from a two-minute questionnaire and everything you've imported from Claude or ChatGPT. Your memory, shared across every model so switching never starts from zero. Your skills, workflows you build once and run on any model. Your scorecard, which models earn your credits per job. One per person. Nobody else has yours. Every model is someone else's; the arrangement is you, and it leaves with you. People aren't loyal to models. They're loyal to results. A moat can't be a commodity, but it can be our own creation.

askr

13,900 次观看 • 1 个月前

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Production traffic is not uniform. You get a few requests that need your best model, but most are simple questions and lookups you can solve with cheaper, faster models. The most expensive mistake you can make today is sending every request to your strongest model. You need routing. Period. This is the simplest trick to improve the architecture of whatever you are building. Please, don't implement routing yourself. You don't have to. I'm currently working with TrueFoundry's Auto Routing. It reads each request, classifies it as simple, medium, or complex, and sends it to the model assigned to that tier. You have two choices: 1. Send every request to the free heuristic classifier to score signals such as technical vocabulary, code, prompt length, and multi-step reasoning. 2. Send the request to an LLM classifier when its difficulty requires a more nuanced judgment. The beauty of using routing is that nothing changes in your code. You still call a single endpoint model, but routing works behind the scenes to pair every request with the best possible model. TrueFoundry ran several experiments with two different setups: 1. Send every request to Claude Opus 2. Send every request to a router with Haiku, Sonnet, and Opus The first experiment ran 550 deterministically graded academic prompts through every setup. Auto Routing was 69% cheaper while retaining 98% of the baseline quality. The second experiment ran three production-shaped workloads through every setup, using user chats, developer chats, and long agent tasks. Auto Routing was 80% cheaper. Thanks to the TrueFoundry team for partnering with me on this post.

Santiago

15,543 次观看 • 1 个月前

the docs said the caching is automatic, so you did nothing automatic is where it happens, not whether it happens the cache lives on the machine that built it, and your next request lands wherever the fleet sends it, which is usually not there, so it pays full price for a prefix it already paid to build there is a key that pins a conversation to one machine and it is a single header cached input is fifty cents a million against two dollars you never needed a smaller prompt you needed the request to land in the same place twice

the docs said the caching is automatic, so you did nothing automatic is where it happens, not whether it happens the cache lives on the machine that built it, and your next request lands wherever the fleet sends it, which is usually not there, so it pays full price for a prefix it already paid to build there is a key that pins a conversation to one machine and it is a single header cached input is fifty cents a million against two dollars you never needed a smaller prompt you needed the request to land in the same place twice

Lagosaur

16,892 次观看 • 1 个月前

Do you understand what just got open sourced? 5,800+ quant trading strategies for FREE along with its CODEBASE i gave these trading strategies to opus 5.5 + jev and it built me a PROFITABLE 24/7 trading agent here is how to use the repo and build AI trading bot: 1. clone the vault and get 5,806 trading strategies (spot, perps, grid, stat arb and market making) 2. open any strategy file, the math, the parameters and the source are already inside 3. point AgenKit at that file inside Claude Code with model as opus 5.5, one command and it reads the spec as ground truth 4. it will run a full engineering team and ships a production bot with risk controls 5. Jev is the live brain, it answers buy sell or hold in 81 milliseconds, calibrated, no hallucination 6. your engine sizes the trade from that confidence and fires it before the next block 7. a hard risk layer holds every limit, nothing goes live until you approve it 8. every night Opus 5.5 reviews the fills and ships a better version, so the agent improves while you sleep one model THINKS, the other REACTS and a strategy that sat locked in a vault now trades itself 24/7 the COMPLETE codebase and the EXACT Opus 5.5 + Jev trading system which I am running is in the article below:

Do you understand what just got open sourced? 5,800+ quant trading strategies for FREE along with its CODEBASE i gave these trading strategies to opus 5.5 + jev and it built me a PROFITABLE 24/7 trading agent here is how to use the repo and build AI trading bot: 1. clone the vault and get 5,806 trading strategies (spot, perps, grid, stat arb and market making) 2. open any strategy file, the math, the parameters and the source are already inside 3. point AgenKit at that file inside Claude Code with model as opus 5.5, one command and it reads the spec as ground truth 4. it will run a full engineering team and ships a production bot with risk controls 5. Jev is the live brain, it answers buy sell or hold in 81 milliseconds, calibrated, no hallucination 6. your engine sizes the trade from that confidence and fires it before the next block 7. a hard risk layer holds every limit, nothing goes live until you approve it 8. every night Opus 5.5 reviews the fills and ships a better version, so the agent improves while you sleep one model THINKS, the other REACTS and a strategy that sat locked in a vault now trades itself 24/7 the COMPLETE codebase and the EXACT Opus 5.5 + Jev trading system which I am running is in the article below:

Roan

96,820 次观看 • 25 天前

THIS GUY REBUILT A CHILDHOOD PS2 GAME AS A FREE BROWSER VERSION WITH CLAUDE CODE he grew up obsessed with downhill domination, the ps2 downhill mountain biking game that never got a sequel so he rebuilt the entire thing from scratch > its a full downhill biking game you play right in your browser, nothing to install > the whole thing is one single html file with no separate image, model, or sound files > everything, the track, the bike, the physics, is generated in code, not loaded from assets > built with fable, opus, vanilla js and three.js over multiple sessions of iterating someone even sent a pull request to add controller support, he merged it, and people helped him squash bugs on the fly a dead ps2 game came back to life as an open source community project this is the best use of this stuff

THIS GUY REBUILT A CHILDHOOD PS2 GAME AS A FREE BROWSER VERSION WITH CLAUDE CODE he grew up obsessed with downhill domination, the ps2 downhill mountain biking game that never got a sequel so he rebuilt the entire thing from scratch > its a full downhill biking game you play right in your browser, nothing to install > the whole thing is one single html file with no separate image, model, or sound files > everything, the track, the bike, the physics, is generated in code, not loaded from assets > built with fable, opus, vanilla js and three.js over multiple sessions of iterating someone even sent a pull request to add controller support, he merged it, and people helped him squash bugs on the fly a dead ps2 game came back to life as an open source community project this is the best use of this stuff

Om Patel

22,230 次观看 • 3 个月前

⚡️NEW A Colonel in the Iranian Army made a Request to the Supreme Leader: "‘I request that the Supreme Leader provide me with a Speedboat loaded with Bombs and a Suicide Vest so that I can ram it into an American warship and Blow It up"

⚡️NEW A Colonel in the Iranian Army made a Request to the Supreme Leader: "‘I request that the Supreme Leader provide me with a Speedboat loaded with Bombs and a Suicide Vest so that I can ram it into an American warship and Blow It up"

Iran Observer

468,968 次观看 • 1 个月前

📍🇮🇷 NEW: A Colonel in the Iranian Army made a Request to the Supreme Leader: "‘I request that the Supreme Leader provide me with a Speedboat loaded with Bombs and a Suicide Vest so that I can ram it into an American warship and Blow It up"

📍🇮🇷 NEW: A Colonel in the Iranian Army made a Request to the Supreme Leader: "‘I request that the Supreme Leader provide me with a Speedboat loaded with Bombs and a Suicide Vest so that I can ram it into an American warship and Blow It up"

Jackson Hinkle 🇺🇸

127,808 次观看 • 1 个月前

Yesterday, Van Jones called me a “demagogue” for discussing the Great Replacement Theory. Well, here are Van’s words in 2021: “The request from the racial justice left: we want the white majority to go from being a majority to being a minority and like it. That’s a tough request, and change is hard.”

Yesterday, Van Jones called me a “demagogue” for discussing the Great Replacement Theory. Well, here are Van’s words in 2021: “The request from the racial justice left: we want the white majority to go from being a majority to being a minority and like it. That’s a tough request, and change is hard.”

Vivek Ramaswamy

3,605,934 次观看 • 2 年前

A Jev engineer hasn’t waited on a CI pipeline in three months, and his changes still reach production faster and safer than the rest of the team’s He never sped the pipeline up. He routed around it, letting every change prove itself on live traffic users never see. > SHADOW - each change runs beside production on a mirrored copy of real traffic > JUDGE - the Jev system compares both outputs request by request, never by averages > SWAP - a change that matches or beats live behaviour goes in, with a 200 ms rollback armed CI was built for a world where real traffic was too risky to touch, so it tests against guesses about what users do. Shadow traffic removes the guess. The new code sees exactly what users send, and its answers simply never reach them. The comparison is the hard part. Two outputs can differ and both be right, so Jev scores intent instead of exact matches. Most changes clear in minutes. The ones that fail get caught on the exact request that broke them, input attached. The armed rollback is what makes it safe. One bad response and the old path is back before the next request lands. His team still writes tests. They just stopped treating a green checkmark as proof that real users will be fine. His full setup, from the traffic mirror to the rollback trigger, is written up in the article attached here.

A Jev engineer hasn’t waited on a CI pipeline in three months, and his changes still reach production faster and safer than the rest of the team’s He never sped the pipeline up. He routed around it, letting every change prove itself on live traffic users never see. > SHADOW - each change runs beside production on a mirrored copy of real traffic > JUDGE - the Jev system compares both outputs request by request, never by averages > SWAP - a change that matches or beats live behaviour goes in, with a 200 ms rollback armed CI was built for a world where real traffic was too risky to touch, so it tests against guesses about what users do. Shadow traffic removes the guess. The new code sees exactly what users send, and its answers simply never reach them. The comparison is the hard part. Two outputs can differ and both be right, so Jev scores intent instead of exact matches. Most changes clear in minutes. The ones that fail get caught on the exact request that broke them, input attached. The armed rollback is what makes it safe. One bad response and the old path is back before the next request lands. His team still writes tests. They just stopped treating a green checkmark as proof that real users will be fine. His full setup, from the traffic mirror to the rollback trigger, is written up in the article attached here.

Beaver

21,131 次观看 • 11 天前

A Jev engineer hasn’t waited on a CI pipeline in three months, and his changes still reach production faster and safer than the rest of the team’s He never sped the pipeline up. He routed around it, letting every change prove itself on live traffic users never see. > SHADOW - each change runs beside production on a mirrored copy of real traffic > JUDGE - the Jev system compares both outputs request by request, never by averages > SWAP - a change that matches or beats live behaviour goes in, with a 200 ms rollback armed CI was built for a world where real traffic was too risky to touch, so it tests against guesses about what users do. Shadow traffic removes the guess. The new code sees exactly what users send, and its answers simply never reach them. The comparison is the hard part. Two outputs can differ and both be right, so Jev scores intent instead of exact matches. Most changes clear in minutes. The ones that fail get caught on the exact request that broke them, input attached. The armed rollback is what makes it safe. One bad response and the old path is back before the next request lands. His team still writes tests. They just stopped treating a green checkmark as proof that real users will be fine. His full setup, from the traffic mirror to the rollback trigger, is written up in the article attached here.

A Jev engineer hasn’t waited on a CI pipeline in three months, and his changes still reach production faster and safer than the rest of the team’s He never sped the pipeline up. He routed around it, letting every change prove itself on live traffic users never see. > SHADOW - each change runs beside production on a mirrored copy of real traffic > JUDGE - the Jev system compares both outputs request by request, never by averages > SWAP - a change that matches or beats live behaviour goes in, with a 200 ms rollback armed CI was built for a world where real traffic was too risky to touch, so it tests against guesses about what users do. Shadow traffic removes the guess. The new code sees exactly what users send, and its answers simply never reach them. The comparison is the hard part. Two outputs can differ and both be right, so Jev scores intent instead of exact matches. Most changes clear in minutes. The ones that fail get caught on the exact request that broke them, input attached. The armed rollback is what makes it safe. One bad response and the old path is back before the next request lands. His team still writes tests. They just stopped treating a green checkmark as proof that real users will be fine. His full setup, from the traffic mirror to the rollback trigger, is written up in the article attached here.

wast3

114,539 次观看 • 12 天前

OpenAI’s studio integration lands in Blender this month, and the interesting part is that it refuses to generate anything twice Most tools race to produce assets. This one spends its first pass deciding what already exists and should never be made again. > INDEX - the studio’s entire asset library read and described, not just filenamed > MATCH - every new request checked against that index before a single polygon is made > REUSE - a close enough match returned as a link to the original, not as a copy > ADAPT - near misses modified from the existing asset instead of built from nothing > MAKE - only genuinely new requests sent to generation > FILE - whatever is produced indexed immediately, so the next request can find it Studios already own most of what they keep rebuilding. They just cannot find it. A library with ten thousand assets and no semantic index is the same as an empty one. Nobody searches it, so everyone remodels. Generative tools made that worse. Producing a new chair became faster than locating the three chairs already on disk. The adapt step is the quiet win. A modified existing asset inherits its materials, its scale and its pipeline history. Every reuse also keeps the library coherent, instead of filling it with near-duplicates nobody can tell apart. The index format, the match threshold and a full reuse log are laid out underneath.

OpenAI’s studio integration lands in Blender this month, and the interesting part is that it refuses to generate anything twice Most tools race to produce assets. This one spends its first pass deciding what already exists and should never be made again. > INDEX - the studio’s entire asset library read and described, not just filenamed > MATCH - every new request checked against that index before a single polygon is made > REUSE - a close enough match returned as a link to the original, not as a copy > ADAPT - near misses modified from the existing asset instead of built from nothing > MAKE - only genuinely new requests sent to generation > FILE - whatever is produced indexed immediately, so the next request can find it Studios already own most of what they keep rebuilding. They just cannot find it. A library with ten thousand assets and no semantic index is the same as an empty one. Nobody searches it, so everyone remodels. Generative tools made that worse. Producing a new chair became faster than locating the three chairs already on disk. The adapt step is the quiet win. A modified existing asset inherits its materials, its scale and its pipeline history. Every reuse also keeps the library coherent, instead of filling it with near-duplicates nobody can tell apart. The index format, the match threshold and a full reuse log are laid out underneath.

wast3

16,937 次观看 • 4 天前

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

DeepLearning.AI

56,896 次观看 • 2 个月前

nobody reads pull request descriptions, so we made videos when Polylane submits a PR to you, we attach a video walkthrough explaining exactly what the issue is, and how it fixed it Vi Tran wrote down how we built it here:

nobody reads pull request descriptions, so we made videos when Polylane submits a PR to you, we attach a video walkthrough explaining exactly what the issue is, and how it fixed it Vi Tran wrote down how we built it here:

boris

11,409 次观看 • 8 天前

Security Swarm sends a swarm of parallel Devins through your codebase to build a threat model, hunt down real, exploitable vulnerabilities (even ones chained across files), prove them in a sandbox, and hand you the fix as a pull request. Here's how it works:

Security Swarm sends a swarm of parallel Devins through your codebase to build a threat model, hunt down real, exploitable vulnerabilities (even ones chained across files), prove them in a sandbox, and hand you the fix as a pull request. Here's how it works:

Devin Developers

15,490 次观看 • 14 天前

Continuous batching in LLMs, clearly explained: (a popular LLM interview question; bookmark this) In traditional ML inference, a batch is a matrix. Every input is padded to the same length, one forward pass runs, and every row finishes at the same moment. LLM decoding does not work that way. One forward pass produces one token per sequence, so a request needs as many passes as it has output tokens, and nobody knows that count until the model emits a stop token. Under static batching, membership is fixed when the batch starts. A request that finishes in 30 tokens holds its slot until the slowest request in the same batch finishes at 400. The GPU keeps paying the full weight read for a batch that is mostly empty. Loading model weights out of HBM costs the same whether four slots are producing tokens or one. Continuous batching moves the decision boundary. Instead of scheduling once per batch, the scheduler runs a single forward pass, gets control back, and decides again. A finished request leaves at the next iteration boundary, and a queued request takes its slot right there. No slot stays reserved for work that is already done. Anyscale benchmarked both OPT-13B on a single A100. With uniform generation lengths, the two policies came out about level (as expected), and as output length variance rose, static batching fell to around 81 tokens per second while vLLM reached 23x the throughput of naive Hugging Face serving. Variance drives the entire gap. Production traffic mixes 30-token replies with 400-token ones, which is exactly the condition static batching handles worst. None of this alters the model. vLLM, SGLang, TGI, and TensorRT-LLM all run it by default, and NVIDIA ships the same mechanism under the name in-flight batching. The animation below runs both policies on the same 16 requests and the same 4 slots, stepping in lockstep. The only difference is when a new request is allowed in. To dive deeper into continuous batching specifically, I wrote a full breakdown of the scheduler underneath it. It covers what happens between two forward passes, how tokens get handed out against a fixed budget, why the scheduler needs no separate path for prefill and decode, and what preemption costs you when the KV cache fills up mid-generation. Read it below.

Continuous batching in LLMs, clearly explained: (a popular LLM interview question; bookmark this) In traditional ML inference, a batch is a matrix. Every input is padded to the same length, one forward pass runs, and every row finishes at the same moment. LLM decoding does not work that way. One forward pass produces one token per sequence, so a request needs as many passes as it has output tokens, and nobody knows that count until the model emits a stop token. Under static batching, membership is fixed when the batch starts. A request that finishes in 30 tokens holds its slot until the slowest request in the same batch finishes at 400. The GPU keeps paying the full weight read for a batch that is mostly empty. Loading model weights out of HBM costs the same whether four slots are producing tokens or one. Continuous batching moves the decision boundary. Instead of scheduling once per batch, the scheduler runs a single forward pass, gets control back, and decides again. A finished request leaves at the next iteration boundary, and a queued request takes its slot right there. No slot stays reserved for work that is already done. Anyscale benchmarked both OPT-13B on a single A100. With uniform generation lengths, the two policies came out about level (as expected), and as output length variance rose, static batching fell to around 81 tokens per second while vLLM reached 23x the throughput of naive Hugging Face serving. Variance drives the entire gap. Production traffic mixes 30-token replies with 400-token ones, which is exactly the condition static batching handles worst. None of this alters the model. vLLM, SGLang, TGI, and TensorRT-LLM all run it by default, and NVIDIA ships the same mechanism under the name in-flight batching. The animation below runs both policies on the same 16 requests and the same 4 slots, stepping in lockstep. The only difference is when a new request is allowed in. To dive deeper into continuous batching specifically, I wrote a full breakdown of the scheduler underneath it. It covers what happens between two forward passes, how tokens get handed out against a fixed budget, why the scheduler needs no separate path for prefill and decode, and what preemption costs you when the KV cache fills up mid-generation. Read it below.

Avi Chawla

16,140 次观看 • 2 个月前

I just found a way to use Claude Code completely free. Claude Code is Anthropic’s terminal coding agent that usually needs a paid API key to run. But a viral GitHub project called Free Claude Code, now sitting at 37k stars, lets you skip the billing entirely. It works as a tiny proxy that sits between Claude Code and free model providers like NVIDIA NIM, OpenRouter, DeepSeek, and Gemini. Every request Claude Code would send to Anthropic gets quietly rerouted to a free model on the other side. The interface stays the same. What’s actually thinking behind it changes. Setup takes a couple of quick commands and a free API key from NVIDIA NIM or OpenRouter. Paste it into the local dashboard, pick a model, and you’re building. The catch is you’re not really using Claude anymore. But when the code ships, does the model badge matter? -- I have shared the Github Link in my free Whatsapp Community ↓

I just found a way to use Claude Code completely free. Claude Code is Anthropic’s terminal coding agent that usually needs a paid API key to run. But a viral GitHub project called Free Claude Code, now sitting at 37k stars, lets you skip the billing entirely. It works as a tiny proxy that sits between Claude Code and free model providers like NVIDIA NIM, OpenRouter, DeepSeek, and Gemini. Every request Claude Code would send to Anthropic gets quietly rerouted to a free model on the other side. The interface stays the same. What’s actually thinking behind it changes. Setup takes a couple of quick commands and a free API key from NVIDIA NIM or OpenRouter. Paste it into the local dashboard, pick a model, and you’re building. The catch is you’re not really using Claude anymore. But when the code ships, does the model badge matter? -- I have shared the Github Link in my free Whatsapp Community ↓

Vaibhav Sisinty

37,255 次观看 • 3 个月前

A DEVELOPER WHO HELPS MAINTAIN GIT ITSELF PROVED THAT USING GIT AND CONTRIBUTING WITH GIT ARE TWO DIFFERENT SKILLS AND ALMOST NO ONE LEARNS THE SECOND 55 minutes from Johannes Schindelin, one of git's most prolific contributors, on the workflow that gets your code accepted into a project you don't own. -> The moment it lands, committing to your own repo turns out to be the easy 10%. Landing a change in someone else's codebase is the part nobody taught you. Most people can save their own work. Far fewer can revise a patch five times, keep the history clean, and merge it into a project moving under them. Saving your code was never the skill -> getting it merged into someone else's is. And when an AI agent fires off a pull request nobody asked for, the person who knows how to really contribute is the one who gets it accepted instead of closed. Anyone can commit to their own repo. The person who can land a change in a project they don't own is the one who actually ships. Bookmark & Watch it today ↓

A DEVELOPER WHO HELPS MAINTAIN GIT ITSELF PROVED THAT USING GIT AND CONTRIBUTING WITH GIT ARE TWO DIFFERENT SKILLS AND ALMOST NO ONE LEARNS THE SECOND 55 minutes from Johannes Schindelin, one of git's most prolific contributors, on the workflow that gets your code accepted into a project you don't own. -> The moment it lands, committing to your own repo turns out to be the easy 10%. Landing a change in someone else's codebase is the part nobody taught you. Most people can save their own work. Far fewer can revise a patch five times, keep the history clean, and merge it into a project moving under them. Saving your code was never the skill -> getting it merged into someone else's is. And when an AI agent fires off a pull request nobody asked for, the person who knows how to really contribute is the one who gets it accepted instead of closed. Anyone can commit to their own repo. The person who can land a change in a project they don't own is the one who actually ships. Bookmark & Watch it today ↓

slash1s

12,797 次观看 • 4 个月前

install these before your next long Claude Code session if your Opus 5.5 bill hurts and you don’t know where to start, copy this: claude-code-router: sits in front of Claude Code and sends each request to the cheapest model that can do it. Haiku does the grep, Opus only gets the hard part. 37k stars rtk: shrinks test, build and git output before the model sees it. one binary, up to 90% less bash output in your context. 83k stars claude-mem: remembers what happened in your past sessions and feeds back only the relevant part, so you stop paying to re-explain your project every morning. 100k stars context-mode: a huge tool output goes into a local database instead of your context window, and Claude pulls out only the lines it needs. 18k stars caveman: makes Claude answer in short telegraphic sentences. code stays identical, and output is the expensive side on Opus 5.5 ($20 per million). 51k stars token-optimizer: a local dashboard that shows where your tokens actually went, and keeps your work alive across compactions. 1.9k stars token-savior: replaces whole-file reads with symbol lookups, so Claude reads the function and not the 2,000 lines around it code-review-graph: maps your repo as a graph and gives Claude only the code connected to the change you’re reviewing jev-ultrafast: a browser agent where Jev picks every click, and a text model wakes up only when something has to be typed. 16,758 stars none of them makes Opus smarter. every one stops it from paying for work it never needed to do the full setup with scripts is in the article. do the measuring step first, so you know what you actually saved

install these before your next long Claude Code session if your Opus 5.5 bill hurts and you don’t know where to start, copy this: claude-code-router: sits in front of Claude Code and sends each request to the cheapest model that can do it. Haiku does the grep, Opus only gets the hard part. 37k stars rtk: shrinks test, build and git output before the model sees it. one binary, up to 90% less bash output in your context. 83k stars claude-mem: remembers what happened in your past sessions and feeds back only the relevant part, so you stop paying to re-explain your project every morning. 100k stars context-mode: a huge tool output goes into a local database instead of your context window, and Claude pulls out only the lines it needs. 18k stars caveman: makes Claude answer in short telegraphic sentences. code stays identical, and output is the expensive side on Opus 5.5 ($20 per million). 51k stars token-optimizer: a local dashboard that shows where your tokens actually went, and keeps your work alive across compactions. 1.9k stars token-savior: replaces whole-file reads with symbol lookups, so Claude reads the function and not the 2,000 lines around it code-review-graph: maps your repo as a graph and gives Claude only the code connected to the change you’re reviewing jev-ultrafast: a browser agent where Jev picks every click, and a text model wakes up only when something has to be typed. 16,758 stars none of them makes Opus smarter. every one stops it from paying for work it never needed to do the full setup with scripts is in the article. do the measuring step first, so you know what you actually saved

rvaniaaa

23,217 次观看 • 9 天前