Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25...

browomo's profile picture

Blaze

9,746 subscribers

1,839,572 görüntüleme • 3 ay önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

Cancelled ChatGPT -> Built JARVIS -> Pays $0 -> it works offline + it's smarter than the $20/month version. No WiFi needed, no cloud, no API keys, no rate limits, no queues, no $20/month just to ask a server in Virginia for the weather. Just a local model running directly on the laptop hardware, voice activated, system integrated, controlling apps, answering questions, doing the work. Iron Man had JARVIS embedded in his suit, this guy has it embedded in his MacBook and it works on a plane, in a basement, on a remote cabin with zero signal. OpenAI is burning $700,000 a day on infrastructure to deliver something this guy runs for free. Anthropic charges $200/month for unlimited Claude access, microsoft built Copilot into every product they sell. This guy skipped all of it, downloaded a model and made his laptop the smartest device in the room. No subscription. No login. No internet. No data sent anywhere ever. The most powerful AI assistant on earth is now the one running locally on hardware you already own. ChatGPT charges you to think slower, he pays nothing and thinks alone, he made it himself.
0:15

Cancelled ChatGPT -> Built JARVIS -> Pays $0 -> it works offline + it's smarter than the $20/month version. No WiFi needed, no cloud, no API keys, no rate limits, no queues, no $20/month just to ask a server in Virginia for the weather. Just a local model running directly on the laptop hardware, voice activated, system integrated, controlling apps, answering questions, doing the work. Iron Man had JARVIS embedded in his suit, this guy has it embedded in his MacBook and it works on a plane, in a basement, on a remote cabin with zero signal. OpenAI is burning $700,000 a day on infrastructure to deliver something this guy runs for free. Anthropic charges $200/month for unlimited Claude access, microsoft built Copilot into every product they sell. This guy skipped all of it, downloaded a model and made his laptop the smartest device in the room. No subscription. No login. No internet. No data sent anywhere ever. The most powerful AI assistant on earth is now the one running locally on hardware you already own. ChatGPT charges you to think slower, he pays nothing and thinks alone, he made it himself.

Defileo🔮

154,009 görüntüleme • 3 ay önce

Alright, now that we know *what* an agent is, how does it actually work? When you ask for help on a task, the agent plans a series of steps and executes them directly in the application on your behalf, using the tools it has access to. Say you are booking a local service or trying to organize your inbox (which typically takes multiple steps): the AI model first plans how to achieve the task using its existing knowledge and then interacts with your inbox to execute the task. The agent will continue until it is confident the task has been successfully completed.
0:15

Alright, now that we know *what* an agent is, how does it actually work? When you ask for help on a task, the agent plans a series of steps and executes them directly in the application on your behalf, using the tools it has access to. Say you are booking a local service or trying to organize your inbox (which typically takes multiple steps): the AI model first plans how to achieve the task using its existing knowledge and then interacts with your inbox to execute the task. The agent will continue until it is confident the task has been successfully completed.

Google AI

22,487 görüntüleme • 8 ay önce

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence
0:19

A HARDWARE MODDER BOLTED A BATTERY PACK ONTO A $599 MAC MINI M4 AND TURNED A DESK SERVER INTO A BACKPACK AI RIG THAT RUNS FOR EIGHT HOURS WITHOUT A WALL OUTLET he posts a video of the mac mini with a slim battery module clamped to the side, USB-C plugged into the back. no power brick, no outlet, just the 10 to 30W draw of the M4 chip pulling from a 20,000mAh bank. a full work day on one charge. this is not a portable laptop trick. it is a desk-class AI server that fits in a sling bag the same $599 mac mini i recommended in last week's article as the silent default, now untethered. pair it with a mobile hotspot or a Starlink mini and you have a fully off-grid LLM stack that answers questions in a forest, on a plane, in a power outage. the model lives in the box, the battery powers the box, that is the whole stack this is not a gadget. it is the first time the words "AI server" and "fits in your bag" belong in the same sentence

shmidt

31,619 görüntüleme • 1 ay önce

This guy built a visual scanner that reads 468 points on his face and 42 points on his hands from a regular webcam and turns them into a cloud of thousands of particles right between his palms. Inside, MediaPipe and TouchDesigner are linked: the first captures hands and face from the webcam with high accuracy, the second turns those coordinates into a live plane and feeds it into a POP system that instantly generates a swarm of particles in the shape of a head. No studio, no render farmer, no VR headset. Just a laptop, a webcam, and 1 TouchDesigner session. And traditional VJ studios keep teams of 5 people on a setup with lighting, custom hardware, and commercial plugins, while his expenses are only a TouchDesigner subscription and a regular USB camera. One laptop runs MediaPipe and TouchDesigner simultaneously, holds the camera stream at 60 FPS without drops, and in parallel processes 468 face points + 21 points on each hand. The camera captures frame after frame, MediaPipe in real time sends TouchDesigner the finger coordinates and face geometry, and the POP operator inside the engine translates those numbers into thousands of particle points with colors from bright pink to gold. This setup immediately defines the role of the tool and the limits of its autonomy. It knows where the fingertips are at every moment of the frame. It knows how to read the face geometry at any angle to the camera. It knows how to draw a swarm of particles between them with the right color and contour. → MediaPipe pulls 468 points from the face and 21 points from each hand, 60 times per second → TouchDesigner receives those coordinates, builds a virtual rectangle between the fingertips, and feeds it into the POP system → POP generates thousands of particle points in the shape of a head, coloring them in a gradient from bright pink to gold → The HUD layer adds green corners and a blue neon frame, styling the image like an AR interface → All layers assemble into 1 real-time frame that projects back onto the video in the camera window → The final image is recorded to a file or broadcast to a projector for a live installation And only when the guy spreads his hands wider does the plane between the palms stretch; brings them together, it narrows. Otherwise the system runs on its own. And when he moves from his home room to a concert hall, the same laptop with the same webcam launches the same TouchDesigner session in just 5 minutes, without reconfiguration, without a new team, and without a single line of new code. In his work setup there is no studio of his own and no team for assembly. On the desk sits a laptop with a webcam, on top run MediaPipe and TouchDesigner with POP operators, and the same setup through a USB camera moves to any concert without a new configuration. Out of everything I have seen this year, this is the cleanest Creative Coding setup on 1 laptop: 0 render farms, 0 studio lighting, and between them 3 libraries, thousands of particle points, and 1 webcam.
0:10

This guy built a visual scanner that reads 468 points on his face and 42 points on his hands from a regular webcam and turns them into a cloud of thousands of particles right between his palms. Inside, MediaPipe and TouchDesigner are linked: the first captures hands and face from the webcam with high accuracy, the second turns those coordinates into a live plane and feeds it into a POP system that instantly generates a swarm of particles in the shape of a head. No studio, no render farmer, no VR headset. Just a laptop, a webcam, and 1 TouchDesigner session. And traditional VJ studios keep teams of 5 people on a setup with lighting, custom hardware, and commercial plugins, while his expenses are only a TouchDesigner subscription and a regular USB camera. One laptop runs MediaPipe and TouchDesigner simultaneously, holds the camera stream at 60 FPS without drops, and in parallel processes 468 face points + 21 points on each hand. The camera captures frame after frame, MediaPipe in real time sends TouchDesigner the finger coordinates and face geometry, and the POP operator inside the engine translates those numbers into thousands of particle points with colors from bright pink to gold. This setup immediately defines the role of the tool and the limits of its autonomy. It knows where the fingertips are at every moment of the frame. It knows how to read the face geometry at any angle to the camera. It knows how to draw a swarm of particles between them with the right color and contour. → MediaPipe pulls 468 points from the face and 21 points from each hand, 60 times per second → TouchDesigner receives those coordinates, builds a virtual rectangle between the fingertips, and feeds it into the POP system → POP generates thousands of particle points in the shape of a head, coloring them in a gradient from bright pink to gold → The HUD layer adds green corners and a blue neon frame, styling the image like an AR interface → All layers assemble into 1 real-time frame that projects back onto the video in the camera window → The final image is recorded to a file or broadcast to a projector for a live installation And only when the guy spreads his hands wider does the plane between the palms stretch; brings them together, it narrows. Otherwise the system runs on its own. And when he moves from his home room to a concert hall, the same laptop with the same webcam launches the same TouchDesigner session in just 5 minutes, without reconfiguration, without a new team, and without a single line of new code. In his work setup there is no studio of his own and no team for assembly. On the desk sits a laptop with a webcam, on top run MediaPipe and TouchDesigner with POP operators, and the same setup through a USB camera moves to any concert without a new configuration. Out of everything I have seen this year, this is the cleanest Creative Coding setup on 1 laptop: 0 render farms, 0 studio lighting, and between them 3 libraries, thousands of particle points, and 1 webcam.

Blaze

38,242 görüntüleme • 2 ay önce

Figure 03 just finished an 8-hour work livestream, imperfect, but already good enough to replace a lot of repetitive warehouse labor. 🤖 Brett Adcock put a team of F.03 robots on a factory-style package sorting task for a full shift. The job was simple and brutal: detect the barcode, pick the package, flip it label-side down, place it on the conveyor, repeat. Soft poly bags, rigid boxes, moving belts, messy orientations. That is exactly the kind of boring physical work factories pay humans to do all day. Early in the stream, the system handled 230 packages in 10 minutes. That is roughly 2.6 seconds per item — already in human-speed territory for this narrow workflow. The more important part: it was not one robot pretending to work all day. It was a team of Figure 03 robots keeping the line running. When one robot ran low on battery, it left the station and another robot stepped in. That is the real factory signal: not just autonomy, but shift continuity. F.03 is rated for about 5 hours of runtime, so the 8-hour result depends on fleet orchestration, charging, and handoff. That matters more than a single clean demo. The stream was not perfect. There were pauses, hesitations, missed orientations, and small recovery moments. Good. A perfect short clip hides failure. An 8-hour livestream exposes the parts that actually matter: endurance, recovery, throughput, and whether the robot can stay useful after the novelty wears off. Figure says this was fully autonomous on Helix-02, with zero human intervention. For logistics and manufacturing, that is the threshold worth watching. Not “can it do one impressive task?” Can it keep doing the boring task for an entire shift? Figure is not showing a general human replacement yet. But for structured, repetitive factory work, the gap just got much smaller. The timing is also interesting: Figure says BotQ has already delivered 350+ F.03 units and reached a 1 robot/hour production cadence. And F.04 is now in full design lock, with parts starting to ship. The next test is obvious. 8 hours was the proof of endurance. 24/7 is the proof of labor economics.
0:25

Figure 03 just finished an 8-hour work livestream, imperfect, but already good enough to replace a lot of repetitive warehouse labor. 🤖 Brett Adcock put a team of F.03 robots on a factory-style package sorting task for a full shift. The job was simple and brutal: detect the barcode, pick the package, flip it label-side down, place it on the conveyor, repeat. Soft poly bags, rigid boxes, moving belts, messy orientations. That is exactly the kind of boring physical work factories pay humans to do all day. Early in the stream, the system handled 230 packages in 10 minutes. That is roughly 2.6 seconds per item — already in human-speed territory for this narrow workflow. The more important part: it was not one robot pretending to work all day. It was a team of Figure 03 robots keeping the line running. When one robot ran low on battery, it left the station and another robot stepped in. That is the real factory signal: not just autonomy, but shift continuity. F.03 is rated for about 5 hours of runtime, so the 8-hour result depends on fleet orchestration, charging, and handoff. That matters more than a single clean demo. The stream was not perfect. There were pauses, hesitations, missed orientations, and small recovery moments. Good. A perfect short clip hides failure. An 8-hour livestream exposes the parts that actually matter: endurance, recovery, throughput, and whether the robot can stay useful after the novelty wears off. Figure says this was fully autonomous on Helix-02, with zero human intervention. For logistics and manufacturing, that is the threshold worth watching. Not “can it do one impressive task?” Can it keep doing the boring task for an entire shift? Figure is not showing a general human replacement yet. But for structured, repetitive factory work, the gap just got much smaller. The timing is also interesting: Figure says BotQ has already delivered 350+ F.03 units and reached a 1 robot/hour production cadence. And F.04 is now in full design lock, with parts starting to ship. The next test is obvious. 8 hours was the proof of endurance. 24/7 is the proof of labor economics.

RoboHub🤖

16,818 görüntüleme • 2 ay önce

this is the worst local ai will ever be. it only gets better from here. if you are not expanding your mind with these small models you are missing what's happening right now 99 percent tool call success rate. when steered well with the right skills and a framework like hermes agent the node becomes a cognition layer. not a chatbot. not a toy. an extension of how you think. i was cranking this node at 35 to 50 tok/s all day on personal experiments and now after all the work is done qwen 3.5 9B is iterating on its own code. the game it created. fixing its own bugs autonomously. and the part you should probably not miss is that all of this is happening on a RTX 3060. not an H100. not an A100. the card most of you have sitting in a drawer right now. if you just open that drawer and put that intelligence to work every tensor core on that card should be running for you. your work. your experiments. your thinking. you all have it but because nobody told you what this hardware can actually do in 2026 you never tried. the day it unlocks is the day you test your workload, understand the tradeoffs, debug the loops, and then decide if you need to scale the hardware. there is no point buying 3 mac studios when things done well you can squeeze a similar level of intelligence from 9B compared to 70B. but only when you create the right environment for your model through the right harness. and let me tell you i have tried claude code as a local harness. i have tried opencode. i have tried various others. somehow i landed on hermes agent and never left. there is something magical going on at Nous Research. the tool call parsers, the skills system, the way it handles small models natively. nothing else comes close for local inference. own your cognition. your AI. your agent. your prompts. your experiments. why give them away for free. those are who you are and they don't belong on someone else's servers being monitored. just give it a shot with your existing hardware. you run into a problem the community will help you. and if you are migrating from openclaw to hermes i will personally help you make the switch.
0:13

this is the worst local ai will ever be. it only gets better from here. if you are not expanding your mind with these small models you are missing what's happening right now 99 percent tool call success rate. when steered well with the right skills and a framework like hermes agent the node becomes a cognition layer. not a chatbot. not a toy. an extension of how you think. i was cranking this node at 35 to 50 tok/s all day on personal experiments and now after all the work is done qwen 3.5 9B is iterating on its own code. the game it created. fixing its own bugs autonomously. and the part you should probably not miss is that all of this is happening on a RTX 3060. not an H100. not an A100. the card most of you have sitting in a drawer right now. if you just open that drawer and put that intelligence to work every tensor core on that card should be running for you. your work. your experiments. your thinking. you all have it but because nobody told you what this hardware can actually do in 2026 you never tried. the day it unlocks is the day you test your workload, understand the tradeoffs, debug the loops, and then decide if you need to scale the hardware. there is no point buying 3 mac studios when things done well you can squeeze a similar level of intelligence from 9B compared to 70B. but only when you create the right environment for your model through the right harness. and let me tell you i have tried claude code as a local harness. i have tried opencode. i have tried various others. somehow i landed on hermes agent and never left. there is something magical going on at Nous Research. the tool call parsers, the skills system, the way it handles small models natively. nothing else comes close for local inference. own your cognition. your AI. your agent. your prompts. your experiments. why give them away for free. those are who you are and they don't belong on someone else's servers being monitored. just give it a shot with your existing hardware. you run into a problem the community will help you. and if you are migrating from openclaw to hermes i will personally help you make the switch.

Sudo su

58,717 görüntüleme • 4 ay önce

This lawyer made $150,000 selling portable offline AI. It analyzes docs that can’t legally be shown on the web. The whole setup costs $50 and he sells it for $999. Here's how to make one step-by-step: You need 4 things: → Raspberry Pi 5 (8GB) → PiSugar 3 Plus battery → Whisplay HAT for the screen and mic → 64GB SD card. Total cost on Ali is around $50 to $90 if you wait for the right deals. 1. Write Raspberry Pi OS Lite 64-bit to the SD card using Raspberry Pi Imager. 2. Stack the PiSugar battery underneath the Pi, snap the Whisplay HAT on top, insert the SD card, and boot the device. 3. Open the terminal and install Ollama with one command: curl -fsSL | sh 4. Pull a model that actually runs on the Pi without choking: ollama pull phi3:mini 5. Run the model and start chatting offline: ollama run phi3:mini The whole thing fits in your pocket, lasts 4 hours on battery, and never touches the internet once setup is done. The lawyer wraps his version in a custom case, preloads it with legal document analysis prompts, and sells it to law firms that can't legally process client data in the cloud. You can sell yours to doctors, accountants, government contractors, defense companies, or anyone else who handles data that legally cannot leave the building. Hardware cost: $50 to $90. Selling price: $500 to $1999
0:17

This lawyer made $150,000 selling portable offline AI. It analyzes docs that can’t legally be shown on the web. The whole setup costs $50 and he sells it for $999. Here's how to make one step-by-step: You need 4 things: → Raspberry Pi 5 (8GB) → PiSugar 3 Plus battery → Whisplay HAT for the screen and mic → 64GB SD card. Total cost on Ali is around $50 to $90 if you wait for the right deals. 1. Write Raspberry Pi OS Lite 64-bit to the SD card using Raspberry Pi Imager. 2. Stack the PiSugar battery underneath the Pi, snap the Whisplay HAT on top, insert the SD card, and boot the device. 3. Open the terminal and install Ollama with one command: curl -fsSL | sh 4. Pull a model that actually runs on the Pi without choking: ollama pull phi3:mini 5. Run the model and start chatting offline: ollama run phi3:mini The whole thing fits in your pocket, lasts 4 hours on battery, and never touches the internet once setup is done. The lawyer wraps his version in a custom case, preloads it with legal document analysis prompts, and sells it to law firms that can't legally process client data in the cloud. You can sell yours to doctors, accountants, government contractors, defense companies, or anyone else who handles data that legally cannot leave the building. Hardware cost: $50 to $90. Selling price: $500 to $1999

Coin Shot ☁️

199,586 görüntüleme • 2 ay önce

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.
0:14

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

69,880 görüntüleme • 13 gün önce

here's how the whole thing works. claude code doesn't care what's behind the API. it just sends requests and expects responses. so i pointed it at my own machine instead of anthropic's servers. llama-server runs the model locally. LiteLLM sits in between and translates the API format. claude code thinks it's talking to claude. it's talking to qwen on localhost. the setup: 2x 3090s, 38 layers on GPU, 10 on CPU. 128K context window. generation is only 7 tok/s but the tradeoff is worth it. 128K means the agent can hold an entire project in memory without losing context midtask. claude code alone loads a 17.5K token system prompt on every request. tool definitions, safety rules, agent behavior. that's your baseline before you even say hello. pushed as far as i could tonight. what surprised me most wasn't the speed. it was the iteration quality. first prompt gave me a working particle sim. second prompt, the model read its own 564 lines, understood the architecture, and added trails, explosions, gravity wells, bloom effects. no handholding. 4bit quantized. 45GB on two consumer cards. running a full coding agent autonomously. detailed article coming. full benchmarks, hardware breakdowns, engine debugging, code quality. everything from setup to what broke and why.
0:12

here's how the whole thing works. claude code doesn't care what's behind the API. it just sends requests and expects responses. so i pointed it at my own machine instead of anthropic's servers. llama-server runs the model locally. LiteLLM sits in between and translates the API format. claude code thinks it's talking to claude. it's talking to qwen on localhost. the setup: 2x 3090s, 38 layers on GPU, 10 on CPU. 128K context window. generation is only 7 tok/s but the tradeoff is worth it. 128K means the agent can hold an entire project in memory without losing context midtask. claude code alone loads a 17.5K token system prompt on every request. tool definitions, safety rules, agent behavior. that's your baseline before you even say hello. pushed as far as i could tonight. what surprised me most wasn't the speed. it was the iteration quality. first prompt gave me a working particle sim. second prompt, the model read its own 564 lines, understood the architecture, and added trails, explosions, gravity wells, bloom effects. no handholding. 4bit quantized. 45GB on two consumer cards. running a full coding agent autonomously. detailed article coming. full benchmarks, hardware breakdowns, engine debugging, code quality. everything from setup to what broke and why.

Sudo su

37,623 görüntüleme • 5 ay önce

Yes, this is a National Highway. While heading towards Mumbai, it took us two hours to travel four kilometres between Royal Garden Resort and Fountain Hotel on NH48. On a Tuesday evening. With no religious festivals, no accidents, no extraordinary rains, nothing. This is simply the state of the highway and traffic every day. In the financial capital of the country. The road doesn't have potholes. The potholes have a road. Chandrayaan-3 will have a more stable landing on the moon than cars have on this patch. It is the same thing, every after year, for decades. Anyone who travels through this patch knows the nuisance that it causes everyone. The infamous Fountain Hotel - where you lose 2 hours of your life that you can never get back. Dear Nitin Gadkari ji, I appreciate your work. Once you are done attending conclaves, I request you to travel on this highway with your officers on any day of the week, at any time you choose, and just experience the torture that ordinary people go through on a daily basis. Is this what Mumbaikars deserve?
0:11

Yes, this is a National Highway. While heading towards Mumbai, it took us two hours to travel four kilometres between Royal Garden Resort and Fountain Hotel on NH48. On a Tuesday evening. With no religious festivals, no accidents, no extraordinary rains, nothing. This is simply the state of the highway and traffic every day. In the financial capital of the country. The road doesn't have potholes. The potholes have a road. Chandrayaan-3 will have a more stable landing on the moon than cars have on this patch. It is the same thing, every after year, for decades. Anyone who travels through this patch knows the nuisance that it causes everyone. The infamous Fountain Hotel - where you lose 2 hours of your life that you can never get back. Dear Nitin Gadkari ji, I appreciate your work. Once you are done attending conclaves, I request you to travel on this highway with your officers on any day of the week, at any time you choose, and just experience the torture that ordinary people go through on a daily basis. Is this what Mumbaikars deserve?

Hardik Rajgor

1,864,602 görüntüleme • 3 yıl önce

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~
0:18

Day 12/90 of Inference Engineering What is chunked prefill within vLLM? In continuation of yesterday's post on the high level architecture of vLLM, I want to dive deeper into vLLM core engine starting with the mechanics of chunked prefill. In this post, I will closely follow the original blog on the anatomy of vLLM. To start, let's define chunked prefill. It's a runtime inference optimization technique that splits a long input request so that it doesn’t monopolize the whole GPU. Keep in mind this is all within the context of vLLM. And since vLLM is an inference engine that's meant to serve a model to multiple concurrent users, having a GPU that’s fully monopolized on a single user's request means other users' requests would be in queue waiting to be processed. It isn’t too good to have the whole GPU occupied on a single request when the GPU is meant to be shared! So the key idea behind chunked prefill is to break the long request into smaller chunks, so that each chunk along with other users' requests gets processed and written into the KV cache together. Suppose we split up the long request into chunks and each chunk has 8 tokens. Now each memory block can hold 4 tokens. Therefore, 8 tokens can fit into 2 blocks of memory. After the first forward pass, 2 blocks are occupied, and after the second forward pass, 4 blocks of memory are occupied and so forth. Each forward pass handles a small chunk of the long request so that there's room in the same pass to keep serving other users' requests. Here's a small animation that I made today to fully visualize the idea behind chunked prefill when learning this topic~

max fu

29,111 görüntüleme • 12 gün önce

THIS KITCHEN GADGET DOESN'T EXIST YET AND 10 STORES WILL BE SELLING IT BY NEXT WEEK every week a random product goes viral on TikTok or Twitter someone finds it on AliExpress for $1-3 and lists it on Shopify for $19.99 no inventory. no warehouse. no shipping. just a product page and a checkout link. the numbers don't make sense: • cost on AliExpress: $1.40 • selling price on Shopify: $19.99 • ad cost per sale: $5-8 • profit per unit: $10-15 • first weekend revenue: $2,000-5,000 last time this happened with a honey spoon. 3.3 million views. 14 stores launched in 48 hours. Claude writes the product page, the ad copy, the email sequence, and the scaling strategy. all in one afternoon. one person, one laptop, one afternoon the full system with every prompt is in the article below
0:15

THIS KITCHEN GADGET DOESN'T EXIST YET AND 10 STORES WILL BE SELLING IT BY NEXT WEEK every week a random product goes viral on TikTok or Twitter someone finds it on AliExpress for $1-3 and lists it on Shopify for $19.99 no inventory. no warehouse. no shipping. just a product page and a checkout link. the numbers don't make sense: • cost on AliExpress: $1.40 • selling price on Shopify: $19.99 • ad cost per sale: $5-8 • profit per unit: $10-15 • first weekend revenue: $2,000-5,000 last time this happened with a honey spoon. 3.3 million views. 14 stores launched in 48 hours. Claude writes the product page, the ad copy, the email sequence, and the scaling strategy. all in one afternoon. one person, one laptop, one afternoon the full system with every prompt is in the article below

Chrome

13,518 görüntüleme • 2 ay önce

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:
0:16

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 görüntüleme • 2 ay önce

This Chinese guy built a Second Brain in Obsidian and every morning gets 3 trading ideas that brought him $180,000 in 6 months. Inside he runs a pipeline of 6 workflows on N8N that automatically pulls every read article, listened podcast, and voice note into a shared Obsidian vault, and a neural network analyst every morning at 6:00 finds connections between the fresh and the old and puts the 3 strongest trading ideas for the day into the inbox. No analytics desk, no Bloomberg terminal, no Telegram chats with traders. Just a Mac Mini by the wall, an iPhone in the pocket, and 1 local Obsidian vault. And traditional quant funds keep entire teams of 8 people on salary for the same flow of insights, while his expenses are only subscriptions to Readwise, Whisper API, and N8N hosting. 6 pipelines process about 200 sources a day and close the monthly API bill at about $120. The Mac Mini itself stores the entire vault and keeps the neural network analyst running 24/7, and from the iPhone the owner drops any idea he hears on the go into a Telegram bot, and it lands in the vault inbox in just 30 seconds. The starting instruction that sits in the VAULT.md file at the root of his vault looks like this: "you are the AI analyst of a solo trader. you read his vault every morning at 6:00, find connections between fresh and old notes, and deliver 3 trading ideas he can verify in the hour before the market opens. pipelines: // Reader (pulls every article and highlight from Readwise, Twitter bookmarks, and Kindle into /notes) // Listener (transcribes podcasts through Airr and voice notes through Whisper, puts them in /notes) // Catcher (accepts any message from the Telegram bot and writes it to /inbox with a timestamp) // Connector (every night reads across the entire vault and updates the connection graph between 4,000 notes) // Briefer (at 6:00 AM writes a brief: 3 trading ideas for today plus the emerging thesis of the week, puts it in /inbox) // Mobile (lives in the iPhone, answers any question about the vault by voice, and confirms alerts while the owner is on the go). you wake the owner with a push notification only when a fresh note contradicts his active thesis or when 1 of the 3 morning ideas has a confidence score above 90%." This instruction immediately sets the role for the system and the limits of its autonomy. It knows it is supposed to connect new with old on its own. It knows it is supposed to prepare 3 trading ideas every morning on its own. It knows it connects the live trader only when a thesis is contradicted or an ultra-confident idea appears. → Reader pulls about 80 articles and highlights a day from Readwise, Twitter, and Kindle → Listener transcribes 4 to 6 podcasts a week through Airr and Whisper → Catcher intercepts all voice and text ideas through the Telegram bot, averaging 15 to 20 a day → Connector updates the connection graph between 4,000 notes every night, adding 25 to 30 new edges → Briefer puts a fresh brief with 3 trading ideas and the emerging thesis into the inbox at exactly 6:00 → Mobile answers any question about the vault by voice and confirms alerts right from the iPhone And only when a new note contradicts his active thesis or 1 of the ideas breaks 90% confidence does the orchestrator raise the owner with a push notification. And when the trader at that moment is driving to the gym or eating breakfast, the Mobile agent in his iPhone answers any quick question about the vault by voice: what he wrote about this ticker last week, which 3 sources support the idea of long NVDA, and what counter-thesis already sits in his notes. The trader makes the decision and sends the order before New York opens. The fresh brief from last Monday looks like this: "reader: 78 materials added over the weekend, 11 of them about semiconductors, 4 about energy, 3 about biotech. passing to connector." "connector: 27 new connections found between fresh materials and the vault, the strongest one is that the Goldman report from Wednesday matches the NVDA thesis you wrote 3 weeks ago." "briefer: 3 trading ideas for today: long NVDA (confidence 0.84), short Tesla at the close of the quarterly report (0.71), watch URI (0.62). emerging thesis of the week: the market is underpricing capex on data centers." "alert: your fresh note about long-term risk in semis contradicts the NVDA thesis. sending for review." In his work setup there is no cloud server, no team of analysts, and not even a Bloomberg subscription. At home sits a Mac Mini with a local Obsidian vault, on top run 6 N8N pipelines and a neural network analyst, and the same vault mirrors to a secure terminal on the iPhone. Out of everything I have seen this year, this is the cleanest solo trading setup on a second brain: $120 a month on the API, about $30,000 a month into the account, and between them 6 pipelines, 4,000 connected notes, and 1 iPhone in the pocket.
0:15

This Chinese guy built a Second Brain in Obsidian and every morning gets 3 trading ideas that brought him $180,000 in 6 months. Inside he runs a pipeline of 6 workflows on N8N that automatically pulls every read article, listened podcast, and voice note into a shared Obsidian vault, and a neural network analyst every morning at 6:00 finds connections between the fresh and the old and puts the 3 strongest trading ideas for the day into the inbox. No analytics desk, no Bloomberg terminal, no Telegram chats with traders. Just a Mac Mini by the wall, an iPhone in the pocket, and 1 local Obsidian vault. And traditional quant funds keep entire teams of 8 people on salary for the same flow of insights, while his expenses are only subscriptions to Readwise, Whisper API, and N8N hosting. 6 pipelines process about 200 sources a day and close the monthly API bill at about $120. The Mac Mini itself stores the entire vault and keeps the neural network analyst running 24/7, and from the iPhone the owner drops any idea he hears on the go into a Telegram bot, and it lands in the vault inbox in just 30 seconds. The starting instruction that sits in the VAULT.md file at the root of his vault looks like this: "you are the AI analyst of a solo trader. you read his vault every morning at 6:00, find connections between fresh and old notes, and deliver 3 trading ideas he can verify in the hour before the market opens. pipelines: // Reader (pulls every article and highlight from Readwise, Twitter bookmarks, and Kindle into /notes) // Listener (transcribes podcasts through Airr and voice notes through Whisper, puts them in /notes) // Catcher (accepts any message from the Telegram bot and writes it to /inbox with a timestamp) // Connector (every night reads across the entire vault and updates the connection graph between 4,000 notes) // Briefer (at 6:00 AM writes a brief: 3 trading ideas for today plus the emerging thesis of the week, puts it in /inbox) // Mobile (lives in the iPhone, answers any question about the vault by voice, and confirms alerts while the owner is on the go). you wake the owner with a push notification only when a fresh note contradicts his active thesis or when 1 of the 3 morning ideas has a confidence score above 90%." This instruction immediately sets the role for the system and the limits of its autonomy. It knows it is supposed to connect new with old on its own. It knows it is supposed to prepare 3 trading ideas every morning on its own. It knows it connects the live trader only when a thesis is contradicted or an ultra-confident idea appears. → Reader pulls about 80 articles and highlights a day from Readwise, Twitter, and Kindle → Listener transcribes 4 to 6 podcasts a week through Airr and Whisper → Catcher intercepts all voice and text ideas through the Telegram bot, averaging 15 to 20 a day → Connector updates the connection graph between 4,000 notes every night, adding 25 to 30 new edges → Briefer puts a fresh brief with 3 trading ideas and the emerging thesis into the inbox at exactly 6:00 → Mobile answers any question about the vault by voice and confirms alerts right from the iPhone And only when a new note contradicts his active thesis or 1 of the ideas breaks 90% confidence does the orchestrator raise the owner with a push notification. And when the trader at that moment is driving to the gym or eating breakfast, the Mobile agent in his iPhone answers any quick question about the vault by voice: what he wrote about this ticker last week, which 3 sources support the idea of long NVDA, and what counter-thesis already sits in his notes. The trader makes the decision and sends the order before New York opens. The fresh brief from last Monday looks like this: "reader: 78 materials added over the weekend, 11 of them about semiconductors, 4 about energy, 3 about biotech. passing to connector." "connector: 27 new connections found between fresh materials and the vault, the strongest one is that the Goldman report from Wednesday matches the NVDA thesis you wrote 3 weeks ago." "briefer: 3 trading ideas for today: long NVDA (confidence 0.84), short Tesla at the close of the quarterly report (0.71), watch URI (0.62). emerging thesis of the week: the market is underpricing capex on data centers." "alert: your fresh note about long-term risk in semis contradicts the NVDA thesis. sending for review." In his work setup there is no cloud server, no team of analysts, and not even a Bloomberg subscription. At home sits a Mac Mini with a local Obsidian vault, on top run 6 N8N pipelines and a neural network analyst, and the same vault mirrors to a secure terminal on the iPhone. Out of everything I have seen this year, this is the cleanest solo trading setup on a second brain: $120 a month on the API, about $30,000 a month into the account, and between them 6 pipelines, 4,000 connected notes, and 1 iPhone in the pocket.

Blaze

926,180 görüntüleme • 2 ay önce

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies
0:19

Run Gemma 4 26B MoE on 8GB VRAM with 250k context at 20+ tokens/sec If you own any 8GB VRAM graphics card, stop what you are doing. Local AI just had its absolute "Holy Shit" moment for budget hardware. Yesterday, I benchmarked Unsloth Gemma 4 12B Q4_K_XL on an 8GB card. The community went wild but immediately demanded more: "Can we run a 25B+ model on budget GPUs?" Today, I’m delivering exactly that. I am running a massive 26B parameter Mixture of Experts (MoE) model locally on a standard 8GB VRAM setup with 250k full native context!. If you own an RTX 3060, 3070, 4060, or any budget GPU with 8GB of VRAM, the local AI paradigm has completely changed. The performance metrics are astonishing: - 20 tokens/sec flat decode throughput. - Stable, flat decode speed even with massive prompts. - I threw a 60k token prompt at it, and it still clocked in at 20 TPS without dropping a single frame. # What about prefill? Yes, Time To First Token (TTFT) is slightly high when swallowing massive contexts. But with a solid 200 tokens/sec prefill speed, the wait is barely noticeable and highly usable. And this is running completely without Multi Token Prediction (MTP) active. How is this possible? It’s the magic of Google's new QAT (Quantization Aware Training) quants for Gemma 4. The model weight file (unsloth gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf) is only 13.2 GB, making it the ultimate local powerhouse. # The Test Setup: CPU: Intel Core i7 RAM: 16GB System RAM GPU: NVIDIA GeForce RTX 4060 Laptop GPU (8GB VRAM) # The Secret Sauce (The -cmoe Flag) To make this work properly on any 8GB card, you must use the -cmoe (CPU MoE) flag in llama.cpp. This flag isolates the heavy MoE expert weights directly to system memory (CPU/RAM) while letting your GPU focus strictly on the Attention layers and the KV Cache. It prevents VRAM spillage and holds the throughput rock solid. # The flags: -m "gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf" -cmoe -c 248000 -v Once running, just open the UI on localhost and toggle the new reasoning lightbulb icon in the text input box to watch the model perform multi step thinking. Are you still running smaller models, or are you ready to scale up your budget local setups? Let's discuss in the replies

Alok

292,770 görüntüleme • 1 ay önce

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading speed. Here's how it works: Every other LLM stores weights in 32-bit or 16-bit floats. BitNet uses 1.58 bits. Weights are ternary just -1, 0, or +1. That's it. No floats. No expensive matrix math. Pure integer operations your CPU was already built for. The result: - 100B model runs on a single CPU at 5-7 tokens/second - 2.37x to 6.17x faster than llama.cpp on x86 - 82% lower energy consumption on x86 CPUs - 1.37x to 5.07x speedup on ARM (your MacBook) - Memory drops by 16-32x vs full-precision models The wildest part: Accuracy barely moves. BitNet b1.58 2B4T their flagship model was trained on 4 trillion tokens and benchmarks competitively against full-precision models of the same size. The quantization isn't destroying quality. It's just removing the bloat. What this actually means: - Run AI completely offline. Your data never leaves your machine - Deploy LLMs on phones, IoT devices, edge hardware - No more cloud API bills for inference - AI in regions with no reliable internet The model supports ARM and x86. Works on your MacBook, your Linux box, your Windows machine. 27.4K GitHub stars. 2.2K forks. Built by Microsoft Research. 100% Open Source. MIT License.
0:13

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading speed. Here's how it works: Every other LLM stores weights in 32-bit or 16-bit floats. BitNet uses 1.58 bits. Weights are ternary just -1, 0, or +1. That's it. No floats. No expensive matrix math. Pure integer operations your CPU was already built for. The result: - 100B model runs on a single CPU at 5-7 tokens/second - 2.37x to 6.17x faster than llama.cpp on x86 - 82% lower energy consumption on x86 CPUs - 1.37x to 5.07x speedup on ARM (your MacBook) - Memory drops by 16-32x vs full-precision models The wildest part: Accuracy barely moves. BitNet b1.58 2B4T their flagship model was trained on 4 trillion tokens and benchmarks competitively against full-precision models of the same size. The quantization isn't destroying quality. It's just removing the bloat. What this actually means: - Run AI completely offline. Your data never leaves your machine - Deploy LLMs on phones, IoT devices, edge hardware - No more cloud API bills for inference - AI in regions with no reliable internet The model supports ARM and x86. Works on your MacBook, your Linux box, your Windows machine. 27.4K GitHub stars. 2.2K forks. Built by Microsoft Research. 100% Open Source. MIT License.

Guri Singh

2,180,357 görüntüleme • 4 ay önce

I MADE MY AI AGENT 10X FASTER WITHOUT CHANGING THE MODEL not a smarter model, not a bigger context window, not another clever prompt the same kind of AI that designs vaccines for viruses we have not even met yet was spending two minutes opening the wrong files just to hand me a brief from three months ago the problem was never capability, it was the scaffolding that piled up around my agent by accident, folder by folder an agent does not think in your categories, it searches from scratch every single time, and your tidy human folders are a maze to it the fix was almost stupidly small, one index file at the root of each big folder and a few numbers in front of the folder names slowest task dropped from 2 minutes to 26 seconds, fastest ones hit 10, zero model changes capability is cheap when the scaffolding around it is broken the article breaks down the whole system in 15 minutes ↓
0:11

I MADE MY AI AGENT 10X FASTER WITHOUT CHANGING THE MODEL not a smarter model, not a bigger context window, not another clever prompt the same kind of AI that designs vaccines for viruses we have not even met yet was spending two minutes opening the wrong files just to hand me a brief from three months ago the problem was never capability, it was the scaffolding that piled up around my agent by accident, folder by folder an agent does not think in your categories, it searches from scratch every single time, and your tidy human folders are a maze to it the fix was almost stupidly small, one index file at the root of each big folder and a few numbers in front of the folder names slowest task dropped from 2 minutes to 26 seconds, fastest ones hit 10, zero model changes capability is cheap when the scaffolding around it is broken the article breaks down the whole system in 15 minutes ↓

shmidt

36,446 görüntüleme • 1 ay önce

This wallet did not exist yesterday. At 3 AM, I could not sleep. I was scrolling through the leaderboard and stumbled upon an anomaly that made me rub my eyes. It started with $213. That is the price of dinner for two. One day later, it had $54,871 on its balance. In just 36 hours, it earned enough to buy a car. I clicked on the transaction history, expecting to see a lucky all-in on some piece of news. Instead, I saw a machine gun burst. > 869 positions in a day and a half. > One trade every 2.5 minutes. > No breaks for sleep, food, or the bathroom. At 4:30 AM, while you were deep in your tenth dream, this address closed a $1,899 profit. That was when I realized I was not looking at a human. I was looking at a predator. This bot hunts on 15-minute BTC and ETH markets. Where most people see a casino and flip a coin on up or down, it sees flaws in your thinking. Here is how it works: > When Bitcoin drops 0.5%, the crowd panics. > The price of NO skyrockets to 75 cents. You sell out of fear. > But the bot knows the math. After such a drop, the actual probability of the trend continuing is only around 52%, not 75%. It buys your fear for 25 cents, when it is actually worth 45. It does not need to know the future. It just needs to buy dollars for 50 cents. But you know what scared me the most? There is a single trade in its history with a $3,213 loss. A normal person would freeze. Light a cigarette. Start doubting. This wallet opened its next position 54 seconds later. It has no ego. No tilt. No I need to win it back. Only an algorithm: if the probability is 55%, and the market pays as if it is 40%, press the button. This is the new form of mining. In 2017, you bought GPUs. In 2026, you write code that mines liquidity from other people’s emotions. Before you rush to search for scripts on GitHub, stop. The $213 starting point is an illusion. Behind it are servers next to the exchange, 50-millisecond latency, and a team of engineers. You will not beat them manually. You are not a player at this table. You are the food. While you were reading this post, it made another $50. Just watch.
0:19

This wallet did not exist yesterday. At 3 AM, I could not sleep. I was scrolling through the leaderboard and stumbled upon an anomaly that made me rub my eyes. It started with $213. That is the price of dinner for two. One day later, it had $54,871 on its balance. In just 36 hours, it earned enough to buy a car. I clicked on the transaction history, expecting to see a lucky all-in on some piece of news. Instead, I saw a machine gun burst. > 869 positions in a day and a half. > One trade every 2.5 minutes. > No breaks for sleep, food, or the bathroom. At 4:30 AM, while you were deep in your tenth dream, this address closed a $1,899 profit. That was when I realized I was not looking at a human. I was looking at a predator. This bot hunts on 15-minute BTC and ETH markets. Where most people see a casino and flip a coin on up or down, it sees flaws in your thinking. Here is how it works: > When Bitcoin drops 0.5%, the crowd panics. > The price of NO skyrockets to 75 cents. You sell out of fear. > But the bot knows the math. After such a drop, the actual probability of the trend continuing is only around 52%, not 75%. It buys your fear for 25 cents, when it is actually worth 45. It does not need to know the future. It just needs to buy dollars for 50 cents. But you know what scared me the most? There is a single trade in its history with a $3,213 loss. A normal person would freeze. Light a cigarette. Start doubting. This wallet opened its next position 54 seconds later. It has no ego. No tilt. No I need to win it back. Only an algorithm: if the probability is 55%, and the market pays as if it is 40%, press the button. This is the new form of mining. In 2017, you bought GPUs. In 2026, you write code that mines liquidity from other people’s emotions. Before you rush to search for scripts on GitHub, stop. The $213 starting point is an illusion. Behind it are servers next to the exchange, 50-millisecond latency, and a team of engineers. You will not beat them manually. You are not a player at this table. You are the food. While you were reading this post, it made another $50. Just watch.

Blaze

14,099 görüntüleme • 5 ay önce

The first patient with a Neuralink chip in his brain has told how his life has changed In an interview with Business Insider, Noland Arbaugh, 29, said he uses the chip almost every day from 7am to 11pm. With the implant, Arbaugh uses it to write messages on X, answer emails, play video games, read comic books, learn Japanese, and has already booked a hotel room. The chip connects to a computer via Bluetooth. Arbo described how one day he fell asleep for a few minutes, and while he slept, the cursor kept moving. When Arbo woke up, he found that he had about 10 different applications open on his computer. Now the battery of the device is enough only for a few hours, after which you need to lie down on a special pillow and recharge it. In the future, the battery capacity will be increased.
0:13

The first patient with a Neuralink chip in his brain has told how his life has changed In an interview with Business Insider, Noland Arbaugh, 29, said he uses the chip almost every day from 7am to 11pm. With the implant, Arbaugh uses it to write messages on X, answer emails, play video games, read comic books, learn Japanese, and has already booked a hotel room. The chip connects to a computer via Bluetooth. Arbo described how one day he fell asleep for a few minutes, and while he slept, the cursor kept moving. When Arbo woke up, he found that he had about 10 different applications open on his computer. Now the battery of the device is enough only for a few hours, after which you need to lie down on a special pillow and recharge it. In the future, the battery capacity will be increased.

NEXTA

104,354 görüntüleme • 2 yıl önce

I decided to share part of a prompt you can use to research any protocol in seconds using INFINIT Intelligence by INFINIT. As an example, I used Silo Labs, where I currently farm most of my stablecoin yields. 🔖 Bookmark this + read until the end for a bonus. Just replace [PROTOCOL NAME] with any protocol you want to research. Prompt: "Conduct thorough research on [PROTOCOL NAME] and answer the following questions: - What is the project building? - What problem does it solve, and for whom? - What makes it different from others? - What blockchain is it built on? - Is there a token? What’s its purpose? - How does the protocol work technically? - How does it make money or sustain itself? - What’s the staking model, emissions, burns, and treasury? - Who are the founders, and is the team public and credible? - Who funded it? Who holds most tokens? - Are users, TVL, and volume growing? - How strong is the ecosystem around it? - Who are the main competitors, and how does it compare? - Is the protocol audited? Any past hacks? - What are the best ways to use the protocol (strategies)? - What’s coming next? Key milestones or launches? - What are the biggest risks? - How strong is the community and social traction?" Bonus: Quote this post with a reason why you like INFINIT, and I’ll DM you the full version of the prompt within the next 24 hours.
0:14

I decided to share part of a prompt you can use to research any protocol in seconds using INFINIT Intelligence by INFINIT. As an example, I used Silo Labs, where I currently farm most of my stablecoin yields. 🔖 Bookmark this + read until the end for a bonus. Just replace [PROTOCOL NAME] with any protocol you want to research. Prompt: "Conduct thorough research on [PROTOCOL NAME] and answer the following questions: - What is the project building? - What problem does it solve, and for whom? - What makes it different from others? - What blockchain is it built on? - Is there a token? What’s its purpose? - How does the protocol work technically? - How does it make money or sustain itself? - What’s the staking model, emissions, burns, and treasury? - Who are the founders, and is the team public and credible? - Who funded it? Who holds most tokens? - Are users, TVL, and volume growing? - How strong is the ecosystem around it? - Who are the main competitors, and how does it compare? - Is the protocol audited? Any past hacks? - What are the best ways to use the protocol (strategies)? - What’s coming next? Key milestones or launches? - What are the biggest risks? - How strong is the community and social traction?" Bonus: Quote this post with a reason why you like INFINIT, and I’ll DM you the full version of the prompt within the next 24 hours.

Keno

15,876 görüntüleme • 1 yıl önce