正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog... on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~show more

max fu

3,185 subscribers

28,891 次观看 • 7 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

69,270 次观看 • 8 天前

While working on a new video with solutions to the previous one, I found ChatGPT's new UI struggles even more with concurrent updates: entries lose state and stick around for too long (see video). If this was a LiveView app, we would be getting so much flak.😅 --- I believe part of the problem here is having separate mutate and fetch requests on every deletion. The first fetch is cancelled when the second one comes up, causing items to stick around for longer. Many said yesterday that you could do the mutation and fetch as a single request, but that leads to other problems, such zombie entries. For example, imagine you delete link1 and link2 within a brief period of time. There is no guarantee the deletion order in the database will match the order the client receives the response, so you may end up with this: 1. (client) request to delete link1 sent 2. (client) request to delete link2 sent 3. (server) deletes link1 and loads a new list (includes link2) 4. (server) deletes link2 and loads a new list (no link1 or link2) 5. (client) receives link2 response 6. (client) receives link1 response So if you choose to use the latest response (link1), you brought link2 back to life. If you say you will use the response from the last request, events 3-4 can be swapped, and now you bring link1 back to life. Another way to solve this is by basically not allowing concurrent requests at all but that can affect the user experience drastically in other ways. Next week I should publish a video explaining how LiveView tackles this. Stay tuned!

While working on a new video with solutions to the previous one, I found ChatGPT's new UI struggles even more with concurrent updates: entries lose state and stick around for too long (see video). If this was a LiveView app, we would be getting so much flak.😅 --- I believe part of the problem here is having separate mutate and fetch requests on every deletion. The first fetch is cancelled when the second one comes up, causing items to stick around for longer. Many said yesterday that you could do the mutation and fetch as a single request, but that leads to other problems, such zombie entries. For example, imagine you delete link1 and link2 within a brief period of time. There is no guarantee the deletion order in the database will match the order the client receives the response, so you may end up with this: 1. (client) request to delete link1 sent 2. (client) request to delete link2 sent 3. (server) deletes link1 and loads a new list (includes link2) 4. (server) deletes link2 and loads a new list (no link1 or link2) 5. (client) receives link2 response 6. (client) receives link1 response So if you choose to use the latest response (link1), you brought link2 back to life. If you say you will use the response from the last request, events 3-4 can be swapped, and now you bring link1 back to life. Another way to solve this is by basically not allowing concurrent requests at all but that can affect the user experience drastically in other ways. Next week I should publish a video explaining how LiveView tackles this. Stay tuned!

José Valim

22,976 次观看 • 1 年前

Hugo Gonzalez on the Celtics: “What can I say? This is the [winningest] franchise in the whole NBA. It’s a privilege to be part of a franchise like this. It’s more or less like where I used to be in Real Madrid, they are the most winningest club in Europe, too. So having that type of transition and similarities and everything is something that is going to be easier.”

Hugo Gonzalez on the Celtics: “What can I say? This is the [winningest] franchise in the whole NBA. It’s a privilege to be part of a franchise like this. It’s more or less like where I used to be in Real Madrid, they are the most winningest club in Europe, too. So having that type of transition and similarities and everything is something that is going to be easier.”

Justin Turpin

60,435 次观看 • 1 年前

The animated scene of the new background + logo is so lovely. They didn't have to make it but they did and I think that reflects the level of extra care that is going into the game now. For the first time in a long time, the future of the game is bright.

Wazzy

17,643 次观看 • 6 个月前

A very good morning. Welcome to The Council Benji This marks the third Skull in a little run. The first went to a fund I've never met. The second: through Eli Scheinman to a new collector/foundation who has been quietly entering the space in a very significant way across a number of collections whom I’ve never spoken to. Their new entrance enabled a wedding and start of a new married life for Conviction. In my very first conversation with him, we spoke about curses and commitments to the people we love. Since meeting got to talk through each step on that path, from letting go, what is imbued in the ring and ceremony of it all, a proposal, and on the way to the most important of the steps in pursuit of a blessed life. It is easy to get a little cynical on the over-leveraged exit stories that spring up from time to time, so it is a treat to watch one go towards a celebration that’s been building up in his life since the Skull was first acquired. And now: this. The third Skull and the first I can really write about as a shared story across both source and destination. An exit and an entrance. The exit: The Skulls of Luci were awarded as gifts 4 years ago. But before I'd minted Birth of Luci or painted the other 49, the first person in this space I showed the sketch of The Blueprint Skull to was actually Casey💎, when he was working at SuperRare . Casey was the very first person who onboarded me to NFTs, helping me navigate the early days of whatever it meant to even mint something. I explained the idea of gifting one to each person who bid in my first auctions. Though most of the Skulls went to the bidders, Casey's didn't. He didn't ask for one. I didn't tell him I'd give him one. But he helped me take my first steps here, and it's hard to imagine any of this making sense, or unfolding the way it has, without him. Since then, we've broken bread across continents, seen quite a lot of chortling margarita consumption, watched the rise and fall of a lot around us, weathered inter-Council dramas. He brought Laura El into The Monument Game, played as a Player, wore a Mask. Most of the vibe that started all of this, the wild west of it, feels faded in the broader space at times. But every Skull has a story and a person who helped us get here. Casey will always be the one who was there before any metric muddled the reason to care. The entrance: Last fall, Benji came over for a studio visit. We walked through Luci, the works, structure, and dream, as anyone who visits does. But we mostly talked about being a father and having a father. We discussed the very idea of "collection" stripped of accumulation, value, or signal, located more in the act or ceremony of it. What it was to grow up with a curious father who studied the edges of each thing he saw to know the next layer beneath why anyone might look or ignore it. That to pass this on is to pass on questioning, more than it is to pass on any kind of answer. The process of collecting can be perceived as an individual act of hoarding. For some it is maybe. But at its best, it's a way to bind through shared questioning, to bond in cooperation and competition with friends and family, it is the swapped story and meme of it all, and each object gathered along the way carries some shared memory that can, often does, and with intent: should; drift out of the object entirely. All in the psalm, always has been. The studio visit came and went. Soon after, a package arrived in the mail with two of the softest stuffed animals added to my daughter's own collection, now among her favorites. The Skull is a bonus to that, in the scheme of shared memory. For Rachel and I, while we are heads down making a body of work that unsettles us and excites us but demands unknown time to accomplish, it means a great deal to have this kind of support from long term people in the quiet process of making work we want to leave behind ourselves. Enormously grateful to Casey for the many years of support and friendship, to Benny for being a true patron, and to Benji for entering the arena for what I'm working on next. Welcome.

A very good morning. Welcome to The Council Benji This marks the third Skull in a little run. The first went to a fund I've never met. The second: through Eli Scheinman to a new collector/foundation who has been quietly entering the space in a very significant way across a number of collections whom I’ve never spoken to. Their new entrance enabled a wedding and start of a new married life for Conviction. In my very first conversation with him, we spoke about curses and commitments to the people we love. Since meeting got to talk through each step on that path, from letting go, what is imbued in the ring and ceremony of it all, a proposal, and on the way to the most important of the steps in pursuit of a blessed life. It is easy to get a little cynical on the over-leveraged exit stories that spring up from time to time, so it is a treat to watch one go towards a celebration that’s been building up in his life since the Skull was first acquired. And now: this. The third Skull and the first I can really write about as a shared story across both source and destination. An exit and an entrance. The exit: The Skulls of Luci were awarded as gifts 4 years ago. But before I'd minted Birth of Luci or painted the other 49, the first person in this space I showed the sketch of The Blueprint Skull to was actually Casey💎, when he was working at SuperRare . Casey was the very first person who onboarded me to NFTs, helping me navigate the early days of whatever it meant to even mint something. I explained the idea of gifting one to each person who bid in my first auctions. Though most of the Skulls went to the bidders, Casey's didn't. He didn't ask for one. I didn't tell him I'd give him one. But he helped me take my first steps here, and it's hard to imagine any of this making sense, or unfolding the way it has, without him. Since then, we've broken bread across continents, seen quite a lot of chortling margarita consumption, watched the rise and fall of a lot around us, weathered inter-Council dramas. He brought Laura El into The Monument Game, played as a Player, wore a Mask. Most of the vibe that started all of this, the wild west of it, feels faded in the broader space at times. But every Skull has a story and a person who helped us get here. Casey will always be the one who was there before any metric muddled the reason to care. The entrance: Last fall, Benji came over for a studio visit. We walked through Luci, the works, structure, and dream, as anyone who visits does. But we mostly talked about being a father and having a father. We discussed the very idea of "collection" stripped of accumulation, value, or signal, located more in the act or ceremony of it. What it was to grow up with a curious father who studied the edges of each thing he saw to know the next layer beneath why anyone might look or ignore it. That to pass this on is to pass on questioning, more than it is to pass on any kind of answer. The process of collecting can be perceived as an individual act of hoarding. For some it is maybe. But at its best, it's a way to bind through shared questioning, to bond in cooperation and competition with friends and family, it is the swapped story and meme of it all, and each object gathered along the way carries some shared memory that can, often does, and with intent: should; drift out of the object entirely. All in the psalm, always has been. The studio visit came and went. Soon after, a package arrived in the mail with two of the softest stuffed animals added to my daughter's own collection, now among her favorites. The Skull is a bonus to that, in the scheme of shared memory. For Rachel and I, while we are heads down making a body of work that unsettles us and excites us but demands unknown time to accomplish, it means a great deal to have this kind of support from long term people in the quiet process of making work we want to leave behind ourselves. Enormously grateful to Casey for the many years of support and friendship, to Benny for being a true patron, and to Benji for entering the arena for what I'm working on next. Welcome.

Sam Spratt

20,786 次观看 • 3 个月前

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

89,234 次观看 • 1 个月前

I finally landed on a memory-optimal way to render audio waveforms in the browser. The full audio is decoded once, then chunked into 2s bins. Each bin stores peaks at 800 peaks/sec in a Uint8Array. Bins are persisted individually in IndexedDB, which keeps the data as raw Uint8 arrays and is faster to read than OPFS. Local or session storage wouldn’t work here. A 2h video ends up as ~5.76 MB cached on disk. While scrolling or zooming the timeline, I load only the visible bins and downsample them to the required resolution. Everything runs async and I never keep more than a single bin in memory at once, ~800 bytes. Rendering supports mixed resolutions in the same pass since bins arrive async. By caching a read index and sorting bins ascending, each peak lookup stays O(1). On refresh, I wipe the IndexedDB so cache size never gets out of hand.

I finally landed on a memory-optimal way to render audio waveforms in the browser. The full audio is decoded once, then chunked into 2s bins. Each bin stores peaks at 800 peaks/sec in a Uint8Array. Bins are persisted individually in IndexedDB, which keeps the data as raw Uint8 arrays and is faster to read than OPFS. Local or session storage wouldn’t work here. A 2h video ends up as ~5.76 MB cached on disk. While scrolling or zooming the timeline, I load only the visible bins and downsample them to the required resolution. Everything runs async and I never keep more than a single bin in memory at once, ~800 bytes. Rendering supports mixed resolutions in the same pass since bins arrive async. By caching a read index and sorting bins ascending, each peak lookup stays O(1). On refresh, I wipe the IndexedDB so cache size never gets out of hand.

konstantinpaulus

32,463 次观看 • 5 个月前

We just released a new marketing site and I wanted to talk through some of the changes that we made and why. Design thinking is the other side of the feed, it's usually more nuanced than the flashier parts and typically doesn't hold up as well to a quick scroll. Doesn't bode well for this thread! But, this will be my attempt to talk through the changes made and why. First up, this is our new hero. At Loops.so, our goal right now is to be the only platform you need to email your users The H1 matches that intention and the subheader reinforces it. We also wanted it to be clear we ship fast and often, so we added a pill showing the latest featured changelog. Agents are part of how people build software today, so we wanted to give that the second most important placement on the page, right after signup. Finally, the logo strip should not distract too much, but it should still help people understand these are users, not just logos. So we added the date of first payment from Stripe next to each logo. Next up, more things!

Chris Frantz

20,382 次观看 • 3 个月前

Heroes never die—they remain in our hearts. 😔🕯️I recently received a request to honor the memory of a fallen soldier – Danylo Unfortunately, it is impossible to bring him back to life, but we can save the lives of other soldiers. To do this, I ask you to support the Armed Forces of Ukraine with your donations. One such option is the charity project “Revenge,” where, for a donation, we write your message on artillery, tank, mortar, and other shells. All funds go directly to the soldiers for their immediate needs. So don't pass by, write to us in private messages and place your order, thereby supporting the Armed Forces of Ukraine!

Heroes never die—they remain in our hearts. 😔🕯️I recently received a request to honor the memory of a fallen soldier – Danylo Unfortunately, it is impossible to bring him back to life, but we can save the lives of other soldiers. To do this, I ask you to support the Armed Forces of Ukraine with your donations. One such option is the charity project “Revenge,” where, for a donation, we write your message on artillery, tank, mortar, and other shells. All funds go directly to the soldiers for their immediate needs. So don't pass by, write to us in private messages and place your order, thereby supporting the Armed Forces of Ukraine!

Cloooud |🇺🇦

23,122 次观看 • 8 个月前

There's been an unfortunate incident in LA with a Uhaul plowing into a crowd of anti-Khameini protestors This man should never have been able to get near the crowd with a uhaul but some info about the situation seem to be - signage on the truck is anti both the shah and the current ayatollah. is this his actual position or is this camouflage to have gotten into the protest to perpetrate an attack? "no Shah, No regime, No Mullah" Mullah is a religious leader so possibly referring to the current leader and not a king like Pahlavi Timeline appears to be - anti-Khameini protestors try to rip signs off his vehicle and are bashing on the windows and eventually his passenger side window is broken - guy in uhaul then stutterstops forward into the crowd, eventually accelerating further, then stuttering again, then full stopping down the road - there is a man surfing on top of the uhaul in the 3rd video, below I have posted another video showing the man on top of the uhaul trying to take the posters off the side, so he is likely part of the anti-Khameini protestors - uhaul driver is taken into custody by police is this a case of police not having the street sufficiently blocked off and so a guy was able to get a uhaul in here? He should not have been able to drive a uhaul this close to a massive protest crowd There are a lot of people saying this is a terrorist attack, it is possible it could be one but I don't think there's enough information to accurately assert that at this time The chronology of events also shows it is possible that the driver was in fear of his life since protestors were banging on the uhaul, windows, and removing signs+ eventually breaking his window Whatever turns out to be the actual case, it is an unfortunate event and as of right now a seeming silver lining is that no deaths have been reported

There's been an unfortunate incident in LA with a Uhaul plowing into a crowd of anti-Khameini protestors This man should never have been able to get near the crowd with a uhaul but some info about the situation seem to be - signage on the truck is anti both the shah and the current ayatollah. is this his actual position or is this camouflage to have gotten into the protest to perpetrate an attack? "no Shah, No regime, No Mullah" Mullah is a religious leader so possibly referring to the current leader and not a king like Pahlavi Timeline appears to be - anti-Khameini protestors try to rip signs off his vehicle and are bashing on the windows and eventually his passenger side window is broken - guy in uhaul then stutterstops forward into the crowd, eventually accelerating further, then stuttering again, then full stopping down the road - there is a man surfing on top of the uhaul in the 3rd video, below I have posted another video showing the man on top of the uhaul trying to take the posters off the side, so he is likely part of the anti-Khameini protestors - uhaul driver is taken into custody by police is this a case of police not having the street sufficiently blocked off and so a guy was able to get a uhaul in here? He should not have been able to drive a uhaul this close to a massive protest crowd There are a lot of people saying this is a terrorist attack, it is possible it could be one but I don't think there's enough information to accurately assert that at this time The chronology of events also shows it is possible that the driver was in fear of his life since protestors were banging on the uhaul, windows, and removing signs+ eventually breaking his window Whatever turns out to be the actual case, it is an unfortunate event and as of right now a seeming silver lining is that no deaths have been reported

Kirsche 🥥 🧁

41,247 次观看 • 6 个月前

Adiyogi represents the absolute unity of the existence. To be able to experience this unity within oneself is Yoga. For too long human beings have turned their differences into discriminations and in turn conflict. It is time everyone embraces all there is as life, and be in Yoga or union with the existence. In this brief life, this possibility is the only way forward. Adiyogi and the technologies of wellbeing that he has offered to us in the form of Yogic sciences, belong not to the past, but to the future and the future wellbeing of humanity on this planet. -Sg #HarGharTiranga #InnerEngineering Ministry of Culture

Adiyogi represents the absolute unity of the existence. To be able to experience this unity within oneself is Yoga. For too long human beings have turned their differences into discriminations and in turn conflict. It is time everyone embraces all there is as life, and be in Yoga or union with the existence. In this brief life, this possibility is the only way forward. Adiyogi and the technologies of wellbeing that he has offered to us in the form of Yogic sciences, belong not to the past, but to the future and the future wellbeing of humanity on this planet. -Sg #HarGharTiranga #InnerEngineering Ministry of Culture

Sadhguru

270,094 次观看 • 2 年前

Dear The Right When you think of the DIRECTION you want our country to go in, how brilliant it is that so many are now facing that way. If we turn on each other as angry individuals, we are fighting the wrong enemy Let us keep moving forwards, together, in the DIRECTION we need the country to go. I believe good people wish to maintain their integrity when they vote. A unified party will reveal itself. And that is a wonderful thing.

Dear The Right When you think of the DIRECTION you want our country to go in, how brilliant it is that so many are now facing that way. If we turn on each other as angry individuals, we are fighting the wrong enemy Let us keep moving forwards, together, in the DIRECTION we need the country to go. I believe good people wish to maintain their integrity when they vote. A unified party will reveal itself. And that is a wonderful thing.

Katie Hopkins

132,248 次观看 • 5 个月前

The part of PixVerse that stood out to me was not just the AI fan host, but the chance to turn a football cheer into something personal. Users can interact with Reina, create personalized cheer videos, and enter a lucky draw for PixVerse service credits. The credits can only be used on PixVerse. They are non transferable and have no cash value. It is a playful early experiment, but I like that it gives users a simple way to make something of their own and share it with others. That mix of creating a cheer moment and joining the credit draw is what I would want to try 👉

The part of PixVerse that stood out to me was not just the AI fan host, but the chance to turn a football cheer into something personal. Users can interact with Reina, create personalized cheer videos, and enter a lucky draw for PixVerse service credits. The credits can only be used on PixVerse. They are non transferable and have no cash value. It is a playful early experiment, but I like that it gives users a simple way to make something of their own and share it with others. That mix of creating a cheer moment and joining the credit draw is what I would want to try 👉

Amit

46,845 次观看 • 10 天前

✅Explanation of Meaning (by parts): 1. “Life consists of a sum of moments in the present” : The author begins by defining life as an accumulation of individual moments experienced in the present. This perspective emphasizes that existence is not a distant future or past but a continuous series of nows, each contributing to the whole of one’s life. It underscores the transient nature of time, suggesting that the present is the fundamental building block of human experience. 2. “Therefore, it’s crucial to always focus on the present” : Building on this, the author argues that prioritizing the present moment is essential. By concentrating on the here and now, individuals can fully engage with their immediate experiences, harnessing the potential of each moment to shape their lives meaningfully. This focus counters distractions from past regrets or future anxieties, grounding one’s attention in what is tangible and actionable. 3. “and live fully here and now” : The author advocates for an immersive approach to living, where one embraces each present moment with intensity and authenticity. Living fully implies investing emotional, intellectual, and physical energy into current experiences, whether they involve joy, work, or reflection, to maximize their value and impact. 4. “and the entire life will be productive and worthy” : The author concludes that consistently focusing on and fully living in the present leads to a life that is both productive and meaningful. A life built on intentional, present-focused moments accumulates into a legacy of purpose and worth, suggesting that mindfulness in the now shapes a fulfilling existence over time. 🗝️Main Idea (refined version): The author asserts that life is composed of present moments, making it essential to focus on and fully engage with the here and now to ensure a productive and worthy existence. By prioritizing mindfulness in each moment, individuals can avoid the pitfalls of dwelling on the past or worrying about the future, instead channeling their energy into meaningful actions and experiences. This approach fosters a life of intentionality, where each present moment contributes to a cumulative sense of purpose and fulfillment. Embracing the present not only enhances personal well-being but also cultivates a legacy of value, as the sum of these moments defines a life well-lived, rich with authenticity and achievement.

✅Explanation of Meaning (by parts): 1. “Life consists of a sum of moments in the present” : The author begins by defining life as an accumulation of individual moments experienced in the present. This perspective emphasizes that existence is not a distant future or past but a continuous series of nows, each contributing to the whole of one’s life. It underscores the transient nature of time, suggesting that the present is the fundamental building block of human experience. 2. “Therefore, it’s crucial to always focus on the present” : Building on this, the author argues that prioritizing the present moment is essential. By concentrating on the here and now, individuals can fully engage with their immediate experiences, harnessing the potential of each moment to shape their lives meaningfully. This focus counters distractions from past regrets or future anxieties, grounding one’s attention in what is tangible and actionable. 3. “and live fully here and now” : The author advocates for an immersive approach to living, where one embraces each present moment with intensity and authenticity. Living fully implies investing emotional, intellectual, and physical energy into current experiences, whether they involve joy, work, or reflection, to maximize their value and impact. 4. “and the entire life will be productive and worthy” : The author concludes that consistently focusing on and fully living in the present leads to a life that is both productive and meaningful. A life built on intentional, present-focused moments accumulates into a legacy of purpose and worth, suggesting that mindfulness in the now shapes a fulfilling existence over time. 🗝️Main Idea (refined version): The author asserts that life is composed of present moments, making it essential to focus on and fully engage with the here and now to ensure a productive and worthy existence. By prioritizing mindfulness in each moment, individuals can avoid the pitfalls of dwelling on the past or worrying about the future, instead channeling their energy into meaningful actions and experiences. This approach fosters a life of intentionality, where each present moment contributes to a cumulative sense of purpose and fulfillment. Embracing the present not only enhances personal well-being but also cultivates a legacy of value, as the sum of these moments defines a life well-lived, rich with authenticity and achievement.

Zafar Mirzo | Quotes

2,173,612 次观看 • 1 年前

A single E. coli cell, placed on a dish, will become 70 billion cells in just 12 hours. That’s exponential growth. But a new preprint shows that it's possible to engineer E. coli to grow linearly instead, where only one daughter cell continues dividing and the other stops. First, some context. In nature, there is a bacterium called Mycobacterium smegmatis (initially discovered in 1884 in ulcers scraped from syphilis patients.) M. smegmatis is weird because it divides asymmetrically. These cells grow only from one end, and all their cell wall biosynthesis machinery is located on that one end. So when the cell divides, one daughter gets this machinery and the other gets nothing. The daughter that gets the machinery can keep dividing immediately, but the other daughter has to remake all that machinery from scratch, so its growth is delayed. E. coli doesn’t grow like this. When it divides, it pinches in the middle and splits everything evenly. Enzymes, metabolites, and proteins get partitioned more or less randomly between the two daughters. For the new preprint, though, researchers engineered E. coli to behave more like M. smegmatis. Here is how they did it: First, they deleted a gene called cyaA, which encodes an enzyme (adenylate cyclase) that makes a molecule called cAMP. cAMP is SUPER IMPORTANT! It is a nutrient sensor that instructs E. coli to switch on genes that help it digest non-glucose carbon sources when glucose is scarce. Without cAMP, E. coli cells growing on alternative carbon sources will starve; they won’t know how to eat the food. Next, they added back a “split” version of the cyaA gene into the cells. In other words, they split the gene in two so that each half of the enzyme is made separately. Cells can only make cAMP, and thus eat non-glucose carbon sources, if these two halves come together. To facilitate that “coming together,” the researchers also fused the split cyaA proteins to sticky proteins that clump together, and to a fluorescent protein (to make it easy to track these molecules in the cell.) So now some interesting things start to happen if you grow E. coli on a growth medium lacking glucose. As the cell grows, its cyaA “halves” start clumping together into a giant ball. Inside the aggregate, the two enzyme halves come together and make cAMP. And when the cell gets big enough and divides, the clump of cyaA RANDOMLY goes to either daughter cell #1 or #2. The daughter that gets the aggregate (called PA+ in this paper) can keep dividing. The daughter that doesn’t (PA–) cannot. It still grows a few times — about four divisions — because it inherits some leftover cAMP from its mother. But after that, the metabolite is diluted away, and the cell stops growing. PA+ cells went through about 23 divisions on average before their aggregate decayed. And the population of cells, as a whole, grew linearly. This paper is cool because there are many applications where exponential growth is too unpredictable and, perhaps, unsafe. If you want to engineer bacteria to deliver drugs, clean up waste, or live in the gut, you don’t want them to double uncontrollably. This paper shows you can make them expand in a controlled, linear way. Alas, mutations could break this whole engineered system. A mutation that restores cyaA, for example, would give cells a new way to make cAMP. Mutations that make the aggregates split between daughters would break the asymmetry, too. But still, I really enjoy proof-of-concept engineering papers like this.

A single E. coli cell, placed on a dish, will become 70 billion cells in just 12 hours. That’s exponential growth. But a new preprint shows that it's possible to engineer E. coli to grow linearly instead, where only one daughter cell continues dividing and the other stops. First, some context. In nature, there is a bacterium called Mycobacterium smegmatis (initially discovered in 1884 in ulcers scraped from syphilis patients.) M. smegmatis is weird because it divides asymmetrically. These cells grow only from one end, and all their cell wall biosynthesis machinery is located on that one end. So when the cell divides, one daughter gets this machinery and the other gets nothing. The daughter that gets the machinery can keep dividing immediately, but the other daughter has to remake all that machinery from scratch, so its growth is delayed. E. coli doesn’t grow like this. When it divides, it pinches in the middle and splits everything evenly. Enzymes, metabolites, and proteins get partitioned more or less randomly between the two daughters. For the new preprint, though, researchers engineered E. coli to behave more like M. smegmatis. Here is how they did it: First, they deleted a gene called cyaA, which encodes an enzyme (adenylate cyclase) that makes a molecule called cAMP. cAMP is SUPER IMPORTANT! It is a nutrient sensor that instructs E. coli to switch on genes that help it digest non-glucose carbon sources when glucose is scarce. Without cAMP, E. coli cells growing on alternative carbon sources will starve; they won’t know how to eat the food. Next, they added back a “split” version of the cyaA gene into the cells. In other words, they split the gene in two so that each half of the enzyme is made separately. Cells can only make cAMP, and thus eat non-glucose carbon sources, if these two halves come together. To facilitate that “coming together,” the researchers also fused the split cyaA proteins to sticky proteins that clump together, and to a fluorescent protein (to make it easy to track these molecules in the cell.) So now some interesting things start to happen if you grow E. coli on a growth medium lacking glucose. As the cell grows, its cyaA “halves” start clumping together into a giant ball. Inside the aggregate, the two enzyme halves come together and make cAMP. And when the cell gets big enough and divides, the clump of cyaA RANDOMLY goes to either daughter cell #1 or #2. The daughter that gets the aggregate (called PA+ in this paper) can keep dividing. The daughter that doesn’t (PA–) cannot. It still grows a few times — about four divisions — because it inherits some leftover cAMP from its mother. But after that, the metabolite is diluted away, and the cell stops growing. PA+ cells went through about 23 divisions on average before their aggregate decayed. And the population of cells, as a whole, grew linearly. This paper is cool because there are many applications where exponential growth is too unpredictable and, perhaps, unsafe. If you want to engineer bacteria to deliver drugs, clean up waste, or live in the gut, you don’t want them to double uncontrollably. This paper shows you can make them expand in a controlled, linear way. Alas, mutations could break this whole engineered system. A mutation that restores cyaA, for example, would give cells a new way to make cAMP. Mutations that make the aggregates split between daughters would break the asymmetry, too. But still, I really enjoy proof-of-concept engineering papers like this.

Niko McCarty.

58,028 次观看 • 11 个月前

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,839,572 次观看 • 2 个月前

Hello! I logged in for the first time in a long time today, and the reason is that since last year, there have been a lot of AI-generated pictures based on my drawings on Reddit and many other sites, so I'm here to announce this...

Hello! I logged in for the first time in a long time today, and the reason is that since last year, there have been a lot of AI-generated pictures based on my drawings on Reddit and many other sites, so I'm here to announce this...

윤

92,368 次观看 • 5 个月前

$IREN "we haven't disclosed the specific amount of GPUs" 1. 🤮 reminds me of $NBIS 2. Setting a terrible precedent here for future deals 3. Making it purposely difficult, to not let analysts properly value your 2027 revenue 4. Increasing the polarized view on IREN by the market However: "approximately 60MW of air-cooled Blackwells" 1. You typically don't talk about gross capacity in a deployment like this 2. If it would be gross capacity, the GPU hour rate at IT level would be crazy high (at PUE 1.2, $680m / 50 = 13.6m/MW) 3. At 60MW IT load, and ~14kW draw at DGX server level, we can get to ~4,286 DGX systems with 8 GPUs per. 4. Based on this we can conclude that 60MW of IT load can run approximately 34k DGX B300. 5. 34k DGX B300 at $680m/yr, would represent a GPU hour price of $2.28 Now this is the problem with not disclosing your GPU quantity. You purposely make your business model look bad, because by approach, you get to a GPU hour price that would imply a payback period of 4 years, where only the last year of the contract is 100% margin. But of course, we can also take "the glass is half full" approach. IREN has ordered 50K B300s from Dell. They have 2 purchase orders for this, 1 between Dell Canada and IE CA Leasing Ltd for 4 phases, and 1 between Dell USA and IE US Hardware 1 Inc (amended from IE US Hardware 4 Inc on April 27, 2026). The order for Canada is divided in 4 phases, and are going to Mackenzie for 80MW of gross capacity, which happens to be 4 buildings of 20MW. The order for Childress is divided in 2 phases, and are going to DC35 and DC36, (as depicted in the earnings presentation) and those are 50MW gross. The purchase price of the order for Childress was $1.2B, and for Canada it was $2.3B If we go with 50,000 B300s for a total of $3.5B then $1.2 would represent 34.285% of the 50,000 GPUs, or 17,140 B300s rounded down. For this calculation I will consider that $IREN will deploy 17,140 GPUs in 50MW gross capacity in DC35 and DC36 of block 3 in Childress.. That would imply at 1.2 PUE, IREN can run 17,140 B300s in 41.67MW IT load. Now by that ratio, they can run 24,680 GPUs in 60MW IT load — a massive difference with 34k units through the Nvidia DGX reference calculation. If common sense is applied, you can still get to 2 completely different outcomes, that show a difference of more than 9k GPUs. The GPU hour rate at 24.68k GPUs would be $3.145 per B300, as MASSIVE difference from the earlier calculated $2.28. Sure, the DGX system may be a factor here. And I'm sure that the reality is somewhere in the middle. But I personally hate this as an investor, to be unable to calculate profitability on unit economic basis. After all, contracts are signed on a $/GPU hour basis. Why hide this from your investors? Not being able to calculate payback periods, unable to calculate ROIC. And most importantly, we cannot properly assess the $NVDA deal on a contract basis. I really hope the payback period of this contract is not 4 years. I want the glass to be half full, but by starting to censor the purchases, IREN is taking a step in the wrong direction. Not a fan of this.

$IREN "we haven't disclosed the specific amount of GPUs" 1. 🤮 reminds me of $NBIS 2. Setting a terrible precedent here for future deals 3. Making it purposely difficult, to not let analysts properly value your 2027 revenue 4. Increasing the polarized view on IREN by the market However: "approximately 60MW of air-cooled Blackwells" 1. You typically don't talk about gross capacity in a deployment like this 2. If it would be gross capacity, the GPU hour rate at IT level would be crazy high (at PUE 1.2, $680m / 50 = 13.6m/MW) 3. At 60MW IT load, and ~14kW draw at DGX server level, we can get to ~4,286 DGX systems with 8 GPUs per. 4. Based on this we can conclude that 60MW of IT load can run approximately 34k DGX B300. 5. 34k DGX B300 at $680m/yr, would represent a GPU hour price of $2.28 Now this is the problem with not disclosing your GPU quantity. You purposely make your business model look bad, because by approach, you get to a GPU hour price that would imply a payback period of 4 years, where only the last year of the contract is 100% margin. But of course, we can also take "the glass is half full" approach. IREN has ordered 50K B300s from Dell. They have 2 purchase orders for this, 1 between Dell Canada and IE CA Leasing Ltd for 4 phases, and 1 between Dell USA and IE US Hardware 1 Inc (amended from IE US Hardware 4 Inc on April 27, 2026). The order for Canada is divided in 4 phases, and are going to Mackenzie for 80MW of gross capacity, which happens to be 4 buildings of 20MW. The order for Childress is divided in 2 phases, and are going to DC35 and DC36, (as depicted in the earnings presentation) and those are 50MW gross. The purchase price of the order for Childress was $1.2B, and for Canada it was $2.3B If we go with 50,000 B300s for a total of $3.5B then $1.2 would represent 34.285% of the 50,000 GPUs, or 17,140 B300s rounded down. For this calculation I will consider that $IREN will deploy 17,140 GPUs in 50MW gross capacity in DC35 and DC36 of block 3 in Childress.. That would imply at 1.2 PUE, IREN can run 17,140 B300s in 41.67MW IT load. Now by that ratio, they can run 24,680 GPUs in 60MW IT load — a massive difference with 34k units through the Nvidia DGX reference calculation. If common sense is applied, you can still get to 2 completely different outcomes, that show a difference of more than 9k GPUs. The GPU hour rate at 24.68k GPUs would be $3.145 per B300, as MASSIVE difference from the earlier calculated $2.28. Sure, the DGX system may be a factor here. And I'm sure that the reality is somewhere in the middle. But I personally hate this as an investor, to be unable to calculate profitability on unit economic basis. After all, contracts are signed on a $/GPU hour basis. Why hide this from your investors? Not being able to calculate payback periods, unable to calculate ROIC. And most importantly, we cannot properly assess the $NVDA deal on a contract basis. I really hope the payback period of this contract is not 4 years. I want the glass to be half full, but by starting to censor the purchases, IREN is taking a step in the wrong direction. Not a fan of this.

Frans Bakker

146,717 次观看 • 2 个月前