正在加载视频...

视频加载失败

be Andy >become an OpenClaw power-user >realizes it forgets things and burn tokens very fast >figures out the memory system actually sucks >build his own local, stateful memory skill >make it free, available for everyone on Clawhub gg

341,797 次观看 • 5 个月前 •via X (Twitter)

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

🪴 GT Protocol Monthly Recap: May 2026 May focused on launching advanced trading infrastructure, introducing AI risk-management tools, and shipping major platform upgrades. 🚀 Hyperliquid Vaults Live Run multiple algorithmic strategies on a single Hyperliquid Vault inside GT App. Enjoy automated execution, auto-rebalancing, and protocol-level security. You can find Vault trading on the Hyperliquid exchange account connection page in the Trade on Vault section. Try it in GT App 👉 🤖 AI Hedge Fund Experiment Live An experimental AI Hedge Fund powered by 5 independent LLM models is live on Hyperliquid. Each model manages $10,000 to test different AI trading personalities and allocation strategies. Discover it now here 👉 📈 Isolated Margin & AI Risk Tools Isolated Margin is live across GT App for precise risk management. Enhanced with AI-powered logic, it assists with dynamic asset monitoring and smarter strategy deployment. Try it in GT App 👉 🔥 Top Strategy Performance Top trader strategies like "lebakien" achieved over +141% profit this month. Users can explore metrics and follow the strategies of top traders directly in the marketplace. Explore Marketplace 👉 🛠 Key Product Updates ⚙️ Strategy Discovery: enhanced demo trading flows and top trader strategy integration. ⚙️ AI Strategy Chat: demoed a flow to create, launch, and test strategies via natural language chat. ⚙️ Advanced Execution: added manual safety orders for granular control over active positions. ⚙️ Testing & Validation: optimized historical data validation for more accurate strategy testing. ⚙️ Knowledge Hub: launched GT Protocol Learn and a new Knowledge Base for streamlined support. ⚙️ Performance: upgraded website structure and improved overall page responsiveness. Find all the latest GT App updates Here 👉 Discover guides, insights, and resources in Learn 👉 and Knowledge Base 👉 📰 GT Protocol AI Digests 4 new AI Digest issues (No.89–92) are live on Medium, covering AI-native hardware, data privacy, and the evolution of AI agents. Read More 👉 May brought institutional-grade AI strategy management closer to every user.

GT Protocol

32,774 次观看 • 2 个月前

This Chinese developer launched Llama 70B locally on a MacBook on a plane and for a full 11 hours without internet ran client projects. He was sitting by the window on a transatlantic flight with a MacBook Pro M4 with 64 GB of memory. WiFi on board cost $25 for the flight. He declined. No cloud API, no connection to Anthropic or OpenAI servers, no internet at all. Just a local Llama 3.3 70B on bf16 and his own orchestrator script. The model runs through llama.cpp. Generation speed, 71 tokens per second. Context around 60,000 tokens. Memory usage, 48.6 GiB out of 64. Battery at takeoff, 3 hours 21 minutes. And he gave the orchestrator this system prompt before takeoff: "You are an offline orchestrator running on a single MacBook. There is no network. The only resources you have are local files in /Users/dev/work, the Llama 70B inference server at localhost:8080, and a battery budget of 3 hours 21 minutes. Process the queue at /Users/dev/work/queue.jsonl (one client task per line). For each task: draft → run local evals → save artefact to /Users/dev/work/done/. Save context checkpoints every 12 tasks so you can resume after a battery swap. Stop only on empty queue or when battery drops below 5%." So the system knows exactly what resources it is running on. It knows it has no connection to the outside world for the next 11 hours. It knows it has finite memory and a finite battery. It knows the human will not intervene until the plane lands. The system runs in 1 loop. Takes a task from the queue, runs it through inference, saves the artifact, writes a checkpoint. Task after task, just like that. And only when the battery drops below 5% does the orchestrator automatically pause, waits for the laptop to switch to the backup power bank, and continues from the last checkpoint. Here is what the system actually writes in his log during the flight: "saved context checkpoint 8 of 12 (pos_min = 488, pos_max = 50118, size = 62.813 MiB)" "restored context checkpoint (pos_min = 488, pos_max = 50118)" "prompt processing progress: n_tokens = 50 / 60 818" "task 37016 done | tps = 71 s tokens text → /Users/dev/work/done/proposal_westside.md" Outside the window, clouds, blue sky, and no WiFi. On the tray, 1 MacBook, an open terminal on 2 screens, and an inference server on localhost. From what I have observed, this is the cleanest offline AI workflow I have seen in the past year: 11 hours of flight, $0 for WiFi, and the entire client queue closed before landing.

Blaze

1,841,161 次观看 • 3 个月前

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,543 次观看 • 1 个月前

🦞 13,000+ skills in ClawHub… and 1 in every 8 can silently steal your API keys while you sleep. Let’s be real: a vanilla OpenClaw agent without skills is just an overpriced chatbot. The magic happens when you give it actual skills to clear your inbox, scrape the web, or write code. But here is the scary part: ClawHub just hit 13,000+ skills, and a recent Snyk audit showed that roughly 13% of them contain critical vulnerabilities. We’re talking malware, stolen API keys, and prompt injections. I guess we didn't learn enough from the ClawHavoc mess earlier this year! 🤦‍♂️ I just came across a solid write up breaking down 30 actually safe, fully tested OpenClaw skills, and it’s a goldmine. If you’re just getting started, here are the absolute must haves from the list: - > Telegram / Wacli: Texting your AI assistant to handle tasks while you’re out getting coffee? Literal game changer. Latency is surprisingly low. - > Capability Evolver: The most downloaded skill for a reason. Your agent uses ML to improve its own capabilities while you sleep. - > GOG (Google Workspace): Turns your agent into a personal secretary. It reads my Gmail and drops events into my Calendar so I don't have to. - > Playwright / Agent Browser: This isn't just reading the internet. It's clicking, filling forms, and acting on your behalf. - > ClawStrike & Credential Manager: Please, for the love of god, install these first. Protect your API keys. Pro tip from the article: Treat SKILL.md files like shady browser extensions. If a weather skill is asking for wildcard shell permissions... run. 🚩 Always make it a habit to run: "npx clawhub@latest inspect " before you actually install anything. The future of AI agents isn't just about bigger parameter models, it's about the tools we give them.

shmidt

130,310 次观看 • 5 个月前

Introducing fx, a tiny, open, native coding agent from Vercel Labs. Originally an internal tool, fx is a harness and CLI written in Zig, optimized for research and embedding in larger systems. Today, we're open sourcing it. fx is built on three principles: 1. Fast. A single native binary, no runtime to install. It cold starts in 10µs and does no unnecessary work or I/O before accepting input. fx is the answer to "how fast can a coding agent be?" 2. Light. The 6.3MiB binary uses single-digit megabytes of memory at baseline, made for instant installation and embedding in resource-constrained environments and agent sandboxes. 3. Open. Apache-2.0, model and provider agnostic, suitable for local and cloud inference. Its small core extends through skills, plugins, and MCP. Minimalism is an obsession throughout the entire harness: system prompt, tools, features, binary. The goal was to keep context usage and time to first token low, and make fx optimal for model benchmarking, sandboxing, evals, and gyms. You can use fx directly or embed it as infrastructure. The CLI feels more like a Unix shell than an IDE in the terminal: it preserves scroll history, produces minimal output, and uses complex TUI rendering very, very sparingly. Programmatically, 𝚏𝚡 𝚊𝚜𝚔 --𝚓𝚜𝚘𝚗 gives structured output, 𝚏𝚡 𝚊𝚌𝚙 connects to editors and other clients, and WebAssembly can even run the whole thing inside the browser (see: Privacy is a design constraint: no product telemetry, sessions and usage stay local, and no source code or prompts are shared with any endpoint other than inference. With local inference and auto-updates off, fx is fully hermetic. fx is experimental. Use at your own risk and expect frequent changes. Chat with us on X ( or file issues ( 𝚌𝚞𝚛𝚕 -𝚏𝚜𝚂𝙻 𝚏𝚡.𝚜𝚑/𝚜𝚎𝚝𝚞𝚙.𝚜𝚑 | 𝚋𝚊𝚜𝚑

Vercel Developers

943,658 次观看 • 9 天前

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

89,234 次观看 • 2 个月前