Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Transformer and Mixture of Experts, explained visually! Mixture of Experts (MoE) is a popular architecture that uses different experts to improve Transformer models. Transformer and MoE differ in the decoder block: - Transformer uses a feed-forward network. - MoE uses experts, which are feed-forward networks but smaller compared to...

DailyDoseOfDS_'s profile picture

Daily Dose of Data Science

51,286 subscribers

53,112 görüntüleme • 3 gün önce •via X (Twitter)

Haberler & PolitikaEğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.
0:17

this is the worst local AI will ever be. tomorrow it gets faster. next month the models get smarter. next year your GPU runs what a data center runs today. Qwen3.5-35B-A3B on a single 3090. told it to visualize its own expert routing. 256 experts, 8 active per token, rendered in 3D on the same GPU running inference. no API key. no subscription. no permission needed. closed AI isn't losing ground. it's losing the argument.

Sudo su

106,822 görüntüleme • 6 ay önce

The Toxic Algae Bloom is decimating California sea life: Just last week, more than 50 dead and dying dolphins washed up along the L.A. County coast. 16 in San Diego County . This is the fourth year in a row and experts say this is the worst yet. Experts also say the toxic algae bloom is feeding off of the billions of gallons of raw sewage flowing into our oceans from Mexico, but we’re not supposed to talk about that.
0:13

The Toxic Algae Bloom is decimating California sea life: Just last week, more than 50 dead and dying dolphins washed up along the L.A. County coast. 16 in San Diego County . This is the fourth year in a row and experts say this is the worst yet. Experts also say the toxic algae bloom is feeding off of the billions of gallons of raw sewage flowing into our oceans from Mexico, but we’re not supposed to talk about that.

Kevin Dalton

68,246 görüntüleme • 1 yıl önce

Vintage Camera Experts 🎥 We appreciate all the camera operator experts and enthusiast that have been reaching out. Ideally, we would love a robust army of imagers helping us gather as much imagery as possible - from different angles, distances, and altitudes with a variety of equipment. Two of the primary challenges for our camera operators right now are digital interference and the extreme speed and movement of the objects. For the former, there seems to be something about the phenomenon that interferes with digital camera technology. Because of this we are looking to also add a vintage camera suite. There’s lots of evidence that suggests analog imagery of the phenomenon produces better results in some circumstances. In regards to the speed and movement challenge, please see the below video clip from episode two. This is what we are dealing with. Note the speed and movement of the Tic Tacs at 100% vs. 15% . If you feel you’re up to the challenge and have something of value to add, please continue to reach out and offer solutions. 🙏🇺🇸
0:10

Vintage Camera Experts 🎥 We appreciate all the camera operator experts and enthusiast that have been reaching out. Ideally, we would love a robust army of imagers helping us gather as much imagery as possible - from different angles, distances, and altitudes with a variety of equipment. Two of the primary challenges for our camera operators right now are digital interference and the extreme speed and movement of the objects. For the former, there seems to be something about the phenomenon that interferes with digital camera technology. Because of this we are looking to also add a vintage camera suite. There’s lots of evidence that suggests analog imagery of the phenomenon produces better results in some circumstances. In regards to the speed and movement challenge, please see the below video clip from episode two. This is what we are dealing with. Note the speed and movement of the Tic Tacs at 100% vs. 15% . If you feel you’re up to the challenge and have something of value to add, please continue to reach out and offer solutions. 🙏🇺🇸

jakebarber

76,706 görüntüleme • 1 yıl önce

There are so many models that it can get confusing so today we launched an auto-router - you get the best answers from any of the top models. At the same time, we have AI experts and model enthusiasts who will enjoy access to 10 new models we just added including: ✅ GPT-4.1 ✅ o3 & o4 Mini ✅ Grok 3 ✅ Llama 4 Maverick & Scout We're also sunsetting some older models that just can't keep up anymore.
0:10

There are so many models that it can get confusing so today we launched an auto-router - you get the best answers from any of the top models. At the same time, we have AI experts and model enthusiasts who will enjoy access to 10 new models we just added including: ✅ GPT-4.1 ✅ o3 & o4 Mini ✅ Grok 3 ✅ Llama 4 Maverick & Scout We're also sunsetting some older models that just can't keep up anymore.

Richard Socher

24,731 görüntüleme • 1 yıl önce

In a new study from MIT and the University of Queensland, 272 international experts were asked to judge the biggest risks from AI between now and 2030. They concluded that 18 of the 24 risks they looked at each carry at least a 1 in 10 chance of a catastrophic outcome, which they defined as something that kills more than a million people or wipes out more than 100 billion dollars. The risks they worried about most were not science fiction robots. They were AI being used to build weapons, launch large scale cyberattacks, and concentrate power and money in very few hands. None of this is a prediction that it will happen. It is a measure of how seriously the field's own experts take the danger. And the numbers are not small.
0:17

In a new study from MIT and the University of Queensland, 272 international experts were asked to judge the biggest risks from AI between now and 2030. They concluded that 18 of the 24 risks they looked at each carry at least a 1 in 10 chance of a catastrophic outcome, which they defined as something that kills more than a million people or wipes out more than 100 billion dollars. The risks they worried about most were not science fiction robots. They were AI being used to build weapons, launch large scale cyberattacks, and concentrate power and money in very few hands. None of this is a prediction that it will happen. It is a measure of how seriously the field's own experts take the danger. And the numbers are not small.

Anonymous

56,850 görüntüleme • 1 ay önce

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.
0:13

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

nathan (in sf)

40,440 görüntüleme • 8 ay önce

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.
0:14

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,797 görüntüleme • 2 ay önce

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~
0:18

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

max fu

29,556 görüntüleme • 2 ay önce

The issue with Karen Read's expert is he went against the Cardinal rule for experts! Instead of doing his own work and seeing what his conclusions would be, he merely looked at the CW's reports and relied on their work and calculations. Experts do their own experiments and calculations and reach their own conclusions. They don't rely on the work of the opposing expert. They aren't in the business of debunking a theory without doing their own work. Also, it seems clear he had never done this type of work: comparing timing car data with cell timing data. Nice fellow and well-accomplished, just not in his lane. The defense didn't want to pay to have this work done right. Cutting corners is never a good plan. P.S. On a side note, this is not a tennis singles match. Such an odd comment. #KarenRead #KarenReadTrial2 #KarenReadRetrial
0:18

The issue with Karen Read's expert is he went against the Cardinal rule for experts! Instead of doing his own work and seeing what his conclusions would be, he merely looked at the CW's reports and relied on their work and calculations. Experts do their own experiments and calculations and reach their own conclusions. They don't rely on the work of the opposing expert. They aren't in the business of debunking a theory without doing their own work. Also, it seems clear he had never done this type of work: comparing timing car data with cell timing data. Nice fellow and well-accomplished, just not in his lane. The defense didn't want to pay to have this work done right. Cutting corners is never a good plan. P.S. On a side note, this is not a tennis singles match. Such an odd comment. #KarenRead #KarenReadTrial2 #KarenReadRetrial

Jennifer Coffindaffer

47,979 görüntüleme • 1 yıl önce

In flow matching, a coupling determines how noise and data samples are paired during training. The choice of coupling is important because it influences the geometry of trajectories at inference time. The simplest choice is the independent coupling, where noise and data points are paired arbitrarily. This can lead to curved trajectories as the model averages over many conflicting pairings. However, if we use optimal transport on batches of pairs, this leads to fewer ambiguous intersections that the model must resolve, leading to straighter trajectories at inference time.
0:18

In flow matching, a coupling determines how noise and data samples are paired during training. The choice of coupling is important because it influences the geometry of trajectories at inference time. The simplest choice is the independent coupling, where noise and data points are paired arbitrarily. This can lead to curved trajectories as the model averages over many conflicting pairings. However, if we use optimal transport on batches of pairs, this leads to fewer ambiguous intersections that the model must resolve, leading to straighter trajectories at inference time.

Alec Helbling

65,484 görüntüleme • 4 ay önce

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning
0:12

Transformer by hand ✍️ ~ 6 steps walkthrough below Open the hood of a transformer and the parts list is overwhelming: embeddings, positional encoding, attention weighting, self-attention, cross-attention, multi-head attention, layer norm, skip connections, softmax, linear, Nx, shifted right, query, key, value, masking. Which of those actually make the car run? Two of them. Attention weighting and the feed-forward network. Everything else is an enhancement to make it run faster and longer, which is how we got from a car to a truck, and to the word "large" in large language model. So I drew and calculated those two parts entirely by hand. Goal: push five features through one transformer block, filling in every cell yourself. 1. Given Five positions of input features, arriving from the previous block. 2. Attention matrix Let us feed all five features to a query-key module (QK) and read back an attention weight matrix, A. The details of that module are a post of their own. 3. Attention weighting We multiply the input features by A to get the attention weighted features, Z. Still five positions. The effect is to combine features *across positions*, horizontally: X1 becomes X1 + X2, X2 becomes X2 + X3, and so on. 4. First layer Let us feed all five weighted features into the first layer of the FFN. Multiply by the weights and biases. This time the combining happens *across feature dimensions*, vertically, and each feature grows from 3 numbers to 4. Note that every position goes through the same weight matrix. That is what "position-wise" means. 5. ReLU We cross out the negatives. They become zeros. 6. Second layer Let us bring it back down: 4 dimensions to 3. The output feeds the next block, which has a completely separate set of parameters, and the whole thing runs again. You have just calculated a transformer block by hand. ✍️ The takeaway: the two parts are doing two different jobs, and neither one alone is enough. Attention mixes *across positions*, so a feature can see its neighbours. The FFN mixes *across feature dimensions*, so each position can think about itself. Horizontal, then vertical. Then that pattern repeats N times, each block with its own separate set of weights. That is the Nx from the list up top, and that is what makes the transformer run. 💾 Save this post! #AIbyHand #Transformers #DeepLearning

Tom Yeh

26,089 görüntüleme • 2 ay önce

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference
0:14

AI has had exactly two scaling axes that worked so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inference

Sasha Malysheva

14,445 görüntüleme • 1 ay önce

Ukrainians know well what war is and what the sound of a Shahed drone over a peaceful city means. That is why Ukrainians are ready to share defense solutions to help protect civilians in other countries. Ukrainian experts are already working in the Middle East to strengthen air defense systems. Meetings have been held with representatives of the UAE’s security and defense forces. President Volodymyr Zelenskyy also discussed this during a meeting with the President of the United Arab Emirates, Mohamed bin Zayed Al Nahyan. May this cooperation be productive and fruitful. And may peace come.
0:16

Ukrainians know well what war is and what the sound of a Shahed drone over a peaceful city means. That is why Ukrainians are ready to share defense solutions to help protect civilians in other countries. Ukrainian experts are already working in the Middle East to strengthen air defense systems. Meetings have been held with representatives of the UAE’s security and defense forces. President Volodymyr Zelenskyy also discussed this during a meeting with the President of the United Arab Emirates, Mohamed bin Zayed Al Nahyan. May this cooperation be productive and fruitful. And may peace come.

Anton Gerashchenko

52,467 görüntüleme • 5 ay önce

What a day! 🤯 Algorand was mentioned 32 times by Google in its whitepaper on quantum risks, highlighting its leading work in PQC! Not sure everyone realizes how BIG this is, massive credibility for $ALGO and huge recognition for the team! 👏 Algorand has been one of the few blockchains/DLTs taking the quantum threat seriously for years, as it's not a question of if but when. Post-quantum implementation takes time and resources, it needs to be anticipated! The past few months have been incredible for $ALGO on that front, as it became one of the first major chains to execute a post-quantum transaction on mainnet using NIST-selected Falcon signatures! Not to mention the great news from a few days ago, with Chris Peikert officially joining Algorand Foundation from AT after the merge! So exciting, as Chris is literally one of the world's leading experts in post-quantum cryptography! What a chance for Algorand to have him and some of the best experts in PQC on its team! SO BULLISH 🔥🔥
0:15

What a day! 🤯 Algorand was mentioned 32 times by Google in its whitepaper on quantum risks, highlighting its leading work in PQC! Not sure everyone realizes how BIG this is, massive credibility for $ALGO and huge recognition for the team! 👏 Algorand has been one of the few blockchains/DLTs taking the quantum threat seriously for years, as it's not a question of if but when. Post-quantum implementation takes time and resources, it needs to be anticipated! The past few months have been incredible for $ALGO on that front, as it became one of the first major chains to execute a post-quantum transaction on mainnet using NIST-selected Falcon signatures! Not to mention the great news from a few days ago, with Chris Peikert officially joining Algorand Foundation from AT after the merge! So exciting, as Chris is literally one of the world's leading experts in post-quantum cryptography! What a chance for Algorand to have him and some of the best experts in PQC on its team! SO BULLISH 🔥🔥

Ⱥlex | france.algo 🇫🇷

25,162 görüntüleme • 5 ay önce

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies
0:19

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Alok

292,770 görüntüleme • 3 ay önce

A genuine question for urban mobility experts. KR Pura–Mahadevapura–Hoodi & further up to Kadugodi has reasonably wide roads, Metro, Suburban Rail and BMTC connectivity. Yet congestion on this stretch remains intense. We spoke to a few regular commuters and, along the way, some auto/cab drivers who use this stretch every day. Their observation was striking: “Traffic has doubled, not reduced. We know what is happening on the ground.” Is that perception or is there something more to it? If this corridor has multiple public transport options and decent road capacity, what is driving the congestion and more importantly, how can traffic actually be managed on this stretch? Would like to hear from urban mobility experts, traffic engineers and people who regularly use this corridor.
0:19

A genuine question for urban mobility experts. KR Pura–Mahadevapura–Hoodi & further up to Kadugodi has reasonably wide roads, Metro, Suburban Rail and BMTC connectivity. Yet congestion on this stretch remains intense. We spoke to a few regular commuters and, along the way, some auto/cab drivers who use this stretch every day. Their observation was striking: “Traffic has doubled, not reduced. We know what is happening on the ground.” Is that perception or is there something more to it? If this corridor has multiple public transport options and decent road capacity, what is driving the congestion and more importantly, how can traffic actually be managed on this stretch? Would like to hear from urban mobility experts, traffic engineers and people who regularly use this corridor.

Bengaluru Post

24,219 görüntüleme • 13 gün önce

According to "EXPERTS", this object is a traditional antique silver anklet, widely worn by women in regions of the Swahili Coast (Zanzibar, Tanzania) and Oman during the 19th and early 20th centuries. These ethnic jewelry pieces are distinguished by their robust shape, significant weight, and rich relief ornamentation. They typically feature a hinged locking mechanism with a pin to open and close around the ankle. But in your opinion, does it look more like jewelry for the ankle or for the GIANT finger?
0:14

Sensitive content

According to "EXPERTS", this object is a traditional antique silver anklet, widely worn by women in regions of the Swahili Coast (Zanzibar, Tanzania) and Oman during the 19th and early 20th centuries. These ethnic jewelry pieces are distinguished by their robust shape, significant weight, and rich relief ornamentation. They typically feature a hinged locking mechanism with a pin to open and close around the ankle. But in your opinion, does it look more like jewelry for the ankle or for the GIANT finger?

Bronze Giant

45,064 görüntüleme • 8 gün önce