I finally landed on a memory-optimal way to render... audio waveforms in the browser. The full audio is decoded once, then chunked into 2s bins. Each bin stores peaks at 800 peaks/sec in a Uint8Array. Bins are persisted individually in IndexedDB, which keeps the data as raw Uint8 arrays and is faster to read than OPFS. Local or session storage wouldn’t work here. A 2h video ends up as ~5.76 MB cached on disk. While scrolling or zooming the timeline, I load only the visible bins and downsample them to the required resolution. Everything runs async and I never keep more than a single bin in memory at once, ~800 bytes. Rendering supports mixed resolutions in the same pass since bins arrive async. By caching a read index and sorting bins ascending, each peak lookup stays O(1). On refresh, I wipe the IndexedDB so cache size never gets out of hand.show more

🚨🇬🇧 Protesters have started RAMMING and THROWING BINS at... the police. After ramming the bins into police, they chant “We want our bin back!”show more

See that red tape line on the floor? You... better not cross it before they tell you “GO!” or you’ll not only get yelled at, you’ll make about 100 people instantly dislike you. My first experience at the Goodwill bins… this operation is so much crazier than I was expecting. This sweet woman spotted me sticking out like a sore thumb. “You a first timer?” She proceeded to give me the rundown: “The college kid resellers are aggressive. See those bins along that wall? Be careful if you go there. They’ll snatch things right out of your hands.” “Every 30-45 minutes they swap out the bins. You have to stay behind the tape line.” “Don’t bring your cart on the floor. Go to the register and get a piece of paper to tape your name on the cart and leave it along the wall.” I have never seen anything like it. And of course, I snuck by the warehouse doors to peek inside what’s happening behind the scenes. Massive pallet racking FULL of clothes bales. I’m talking hundreds of thousands of pieces. Conveyer belt sorting systems with everything you can imagine. Pallets of bicycles. Filthy vacuums. And rolling bins. A bazillion rolling bins full of who knows what, but apparently some treasure. Before you get triggered by “people making money off donations that are ‘supposed’ to support people who are impoverished,” I invite you to go see an operation like this. The amount of excess materialism is astounding, and this is a microscopic fraction of it. Resellers duking it out over labels and prized goods all day everyday at places like this are playing an extremely important role in a circular economy that is making a difference in the appalling about of waste we have created as a society. You won’t find me doing this to source inventory, but of course I snagged a small stack of books because who would I be if I didn’t 😜 And I have a lot of respect for the number of young adults I saw hustling their way through this chaos, leaving with garbage bags full of items that are putting food on their tables. Wild.show more

Public garbage bins in Helsingborg use a foot pedal... and different interactive audio clips to encourage people to use them.show more

The ‘Trash Revolution’ is here, and the rats are... FURIOUS! Order an official NYC bin at — for half as much as you’d pay retail — and help get 70% of NYC’s trash off our streets and into bins by November 12.show more

🇫🇷 Rioters burn trash bins on the steps of... a Caen high school in Normandy, Franceshow more

Day 12/90 of Inference Engineering What is chunked prefill... within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~show more

Supervising sunshine levels, guarding the bins, and soaking up... a well-earned glow… multitasking at its finest. Treato breaks will be taken as required.show more

🚨Breaking News The Irish are putting the bins out... in Belfast.show more

Protesters - chanting 'Henry, Henry' - have thrown bottles,... bricks and wheelie bins in clashes with riot police, after a row broke out over the murder of Henry Nowak Read more about the clashes taking place in Southampton here ⤵️show more

Memory in AI Agent Harness. LLMs are stateless, so... every bit of memory in an AI agent is something the harness saves to disk and loads back in when it needs it. most good agent harnesses split that memory into a few layers, let's look at each one of them: • context: what the model can see in the current session. when it gets close to full, the harness summarizes the older messages to make room. • notes: a progress file the agent keeps updating while it works, so if a session ends halfway the next one can read it and continue from there. • semantic: facts about you, your team and your project, like your preferences or which repo you work in. Hermes keeps these in MEMORY.md and USER.md. • procedural: how to do a specific job. when the agent figures out something hard, it writes it down as a skill and uses that skill the next time. • episodic: a log of every past session that the agent can search when it needs to remember what happened before. First two only last as long as the session or the task. Other three are persistent, so they're still there the next time you start the agent. save this for the next time someone asks how agents remember. follow Shubham Saboo for more on Agent Harness.show more

NEW: Revere Robotics is building a robot that takes... your trash bin to the curb and brings it back for you. Curbie attaches to your existing trash can and is designed to handle the weekly curb run without needing a special bin. -Attaches directly to standard residential trash bins -Drives the bin to the curb on collection day -Brings it back after pickup -Designed to move bins weighing 100+ lbs -Being built for rain, snow, and icy sidewalks -Stops and alerts you if it detects an unsafe tilt Pricing has not been announced yet.show more

Cleaning up trash... and gods. Both are in his... job description! Butler of the Bins Danzaburou is available now in the Tidy tails Wandering Market Collection!show more

A RUSSIAN MATHEMATICIAN BUILT A SYSTEM WHERE MEMORY AND... EVAL WORK AS ONE PIPELINE NOT TWO SEPARATE TOOLS Most setups run memory and evaluation as separate systems that never actually talk to each other. He wired them into one loop instead, memory stores every past output, eval scores each one before it gets kept. Anything that scores low never enters memory at all, so nothing weak gets carried into future decisions. High scorers get tagged with the exact eval criteria they passed, not just a raw pass or fail. The next task pulls only memories that passed the same bar it's about to be judged on. See how the two systems feed each other below👇show more

Researchers made KMeans 200x faster. And the new technique... also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.show more

Did you know you can create the Maxwell-Boltzmann distribution... with just some balls and a motor? I built this simple device to illustrate it. This is how it works: A spinning agitator pumps energy into ~400 balls, so they bounce around and collide like gas particles in a box. They escape one at a time through a tiny hole, then fall a fixed gap into bins below. Since the hole is so small, every ball leaves with no vertical speed and takes the same time to fall. Constant fall time means how far a ball flies sideways is set purely by its speed. A faster ball goes into a further bin. So each bin is really a speed. Tally all 400 and you get the Maxwell-Boltzmann distribution.show more

Can't believe this is happening in the UK in... 2024 🥴 ➡️Hull: thug shouts to police saying you're as bad as the paki who killed those three girls (the full clip has more obscenities) ➡️ Manchester: group of thugs surround then attack a black man ➡️ A group of thugs push bins into the police ➡️ A group of thugs knock over a policeman on his motorbike, then attack him There are the thugs who have turned up causing disruption across the country, and then those who are motivating and encouraging the thugs to cause disruption across the country - this behaviour must never be normalised Thank you to the police for trying to keep the peace 🙏show more

Claude Monet painted the same stretch of cliff more... than ninety times. The place is Étretat, a small fishing village on the coast of Normandy, where the chalk cliffs fall into the sea in great arches and a single spire of rock, the Aiguille, stands alone in the water. Monet had known the place since childhood. He grew up in Normandy, and these cliffs were among the first landscapes he ever saw... He returned to paint them again and again. He worked through the 1880s in front of the same rock formations, and across that time he produced more than ninety canvases of them: the cliffs at dawn, at sunset, under storm, under calm, in winter light and in the gold of a clear evening. In his letters to Alice, the woman he would later marry, he described the agony of it: the weather turning, the tide rising, the sun moving, the colour he had begun to capture vanishing before he could finish. He often worked on several canvases at once, switching between them as the conditions changed, racing each one against the hour. In a letter to his friend Frédéric Bazille he wrote: "It is beautiful here in Etretat. Every day I discover even more beautiful things. It is intoxicating me, and I want to paint it all, my head is bursting. I want to fight, scratch it off, start again, because I start to see and understand. It seems to me as if I can see nature and I can catch it all." The cliffs of Étretat had stood for millions of years and would look, to most people, the same on any given day. Monet saw that they were never the same even for two minutes. He stood on that shore and tried to hold, on canvas, something that exists only for an instant and then is gone forever. And that's exactly what those paintings really are: 90 attempts to keep a single, vanishing moment of light from disappearing. As Dylan Thomas once wrote: "Do not go gentle into that good night. Rage, rage against the dying of the light." -- -- -- If you want a deeper dive into the craft of painting, I recently wrote a piece exploring it in detail. You can read it here: And if you'd like to support my work, a paid subscription is what makes it possible:show more