Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

be Andy >become an OpenClaw power-user >realizes it forgets things and burn tokens very fast >figures out the memory system actually sucks >build his own local, stateful memory skill >make it free, available for everyone on Clawhub gg

341,797 Aufrufe • vor 5 Monaten •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🪴 GT Protocol Monthly Recap: May 2026 May focused on launching advanced trading infrastructure, introducing AI risk-management tools, and shipping major platform upgrades. 🚀 Hyperliquid Vaults Live Run multiple algorithmic strategies on a single Hyperliquid Vault inside GT App. Enjoy automated execution, auto-rebalancing, and protocol-level security. You can find Vault trading on the Hyperliquid exchange account connection page in the Trade on Vault section. Try it in GT App 👉 🤖 AI Hedge Fund Experiment Live An experimental AI Hedge Fund powered by 5 independent LLM models is live on Hyperliquid. Each model manages $10,000 to test different AI trading personalities and allocation strategies. Discover it now here 👉 📈 Isolated Margin & AI Risk Tools Isolated Margin is live across GT App for precise risk management. Enhanced with AI-powered logic, it assists with dynamic asset monitoring and smarter strategy deployment. Try it in GT App 👉 🔥 Top Strategy Performance Top trader strategies like "lebakien" achieved over +141% profit this month. Users can explore metrics and follow the strategies of top traders directly in the marketplace. Explore Marketplace 👉 🛠 Key Product Updates ⚙️ Strategy Discovery: enhanced demo trading flows and top trader strategy integration. ⚙️ AI Strategy Chat: demoed a flow to create, launch, and test strategies via natural language chat. ⚙️ Advanced Execution: added manual safety orders for granular control over active positions. ⚙️ Testing & Validation: optimized historical data validation for more accurate strategy testing. ⚙️ Knowledge Hub: launched GT Protocol Learn and a new Knowledge Base for streamlined support. ⚙️ Performance: upgraded website structure and improved overall page responsiveness. Find all the latest GT App updates Here 👉 Discover guides, insights, and resources in Learn 👉 and Knowledge Base 👉 📰 GT Protocol AI Digests 4 new AI Digest issues (No.89–92) are live on Medium, covering AI-native hardware, data privacy, and the evolution of AI agents. Read More 👉 May brought institutional-grade AI strategy management closer to every user.

GT Protocol

32,774 Aufrufe • vor 2 Monaten

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,841,051 Aufrufe • vor 3 Monaten

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,238 Aufrufe • vor 27 Tagen

🦞 13,000+ skills in ClawHub… and 1 in every 8 can silently steal your API keys while you sleep. Let’s be real: a vanilla OpenClaw agent without skills is just an overpriced chatbot. The magic happens when you give it actual skills to clear your inbox, scrape the web, or write code. But here is the scary part: ClawHub just hit 13,000+ skills, and a recent Snyk audit showed that roughly 13% of them contain critical vulnerabilities. We’re talking malware, stolen API keys, and prompt injections. I guess we didn't learn enough from the ClawHavoc mess earlier this year! 🤦‍♂️ I just came across a solid write up breaking down 30 actually safe, fully tested OpenClaw skills, and it’s a goldmine. If you’re just getting started, here are the absolute must haves from the list: - > Telegram / Wacli: Texting your AI assistant to handle tasks while you’re out getting coffee? Literal game changer. Latency is surprisingly low. - > Capability Evolver: The most downloaded skill for a reason. Your agent uses ML to improve its own capabilities while you sleep. - > GOG (Google Workspace): Turns your agent into a personal secretary. It reads my Gmail and drops events into my Calendar so I don't have to. - > Playwright / Agent Browser: This isn't just reading the internet. It's clicking, filling forms, and acting on your behalf. - > ClawStrike & Credential Manager: Please, for the love of god, install these first. Protect your API keys. Pro tip from the article: Treat SKILL.md files like shady browser extensions. If a weather skill is asking for wildcard shell permissions... run. 🚩 Always make it a habit to run: "npx clawhub@latest inspect " before you actually install anything. The future of AI agents isn't just about bigger parameter models, it's about the tools we give them.

shmidt

130,148 Aufrufe • vor 4 Monaten

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

89,234 Aufrufe • vor 1 Monat

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 Aufrufe • vor 3 Monaten