A good technical LLM interview question: Your LLM chatbot... takes 12s before it generates the first token, and the users are complaining. So you move the model onto a GPU with 3x the computing power. The time to first token barely improves. Why did this happen? (answer below) Latency in an LLM app is a placement problem disguised as a model problem. If you profile the 12 seconds, the model's prefill itself may only account for around 1.5 seconds of it. So halving the prefill step saves just 750ms out of 12000, which is under 7%. The rest is spread across stages that never touch the GPU. The request first travels to whatever region the app runs in, and a cross-continent round trip could cost over a second before any code executes. Then the request handler starts. On a container-based serverless platform under load, this adds several seconds of cold start, paid before auth, rate limiting, or prompt assembly even begins. Retrieval adds its own hop, and the response streams back across the same distance. Optimizing a stage that was already fast cannot alter the latency that's majorly affected by other stages. Those other stages are slow for a structural reason. An LLM app runs two workloads that want opposite machines. - The request path is short, spiky, and needs to sit close to users - Inference is long-running, GPU-bound, and billed hourly, whether requests arrive or not. So the actual decision is not which model to run, but where each of these two workloads runs. There are three options, each with its own tradeoffs: > A dedicated GPU box removes inference cold starts, but it bills around the clock and lives in one location, so distant users wait out the round trip on every request > Container-based serverless scales to zero, but the request path pays a cold start, and most of these platforms have no GPU behind them. > Edge runtimes start in under a millisecond, because a WebAssembly module carries no OS or container image to boot. They handle the request path well and cannot hold a model. So the answer is not to pick one, but to split the app across two of them. The request path runs close to users, and inference runs on a dedicated GPU it calls into. That also explains the failed upgrade. More compute made a stage that was already fast faster, and left the 10.5 seconds around it untouched. To actually learn how it's done in practice, Akamai's GitHub has a reference implementation for each half. - vllm-on-lke serves Qwen2.5-7B-Instruct behind an OpenAI-compatible endpoint on one RTX 4000 Ada GPU in Linode Kubernetes Engine, with Terraform creating the cluster, both firewalls, and the GPU operator in one apply. - akamai-functions-llm-chatbot covers the front, where a WebAssembly API checks a KV cache and only calls the GPU-backed instance on a miss. Both are available on Akamai’s new Developer Hub, alongside their tutorials and code samples. It also links to Edge Case, their Discord, where four developer advocates architect and deploy a production app live every other Wednesday. If you create a new Akamai Cloud account, you can also get $300 in credits for joining. Join here: That said, this post treats generation as a single 1.5s block, but that block has its own structure, and knowing it well tells you whether a model is slow to start or slow to stream. I wrote a first-principles walkthrough of it, covering the prefill and decode split, KV caching, and where the time actually goes inside each one. Read it below. Thanks to Akamai Cloud for partnering today!show more

A good technical LLM interview question: Your RAG chatbot... is working as expected locally. You deploy it behind a load balancer with 3 replicas. Users report that it forgets what they just asked, and answers get worse with each restart. Why did this happen? (answer below) A local setup has one process that owns everything. - The vector index is a variable in memory. - Conversation history is a Python list. - The documents are on local disk. You never treat any of them as infrastructure, because restarting rebuilds all three in seconds and there is only ever one copy. The setup does not carry over to production directly. The vector index might disappear on restart, so the app re-embeds everything on boot and serves empty results until it finishes. Conversation history may belong to one replica, so a follow-up routed elsewhere has no memory of the previous turn. Documents could be on whichever container ingested them, so the three replicas hold three different corpora. None of this is evident with one user and one process. So the actual work in shipping RAG is not just the retrieval logic, but also storing the vector index, the conversation history, and the documents outside the app, where every replica reads and writes the same copy. Which comes down to three requirements: > The vector store needs persistence and has to be reachable from every replica. pgvector inside Postgres keeps embeddings next to the rest of the data instead of adding another system to operate. > Conversation state has to be checkpointed outside the app. LangGraph writes its state to Postgres, so any replica can pick up a thread mid-conversation. > Docs need shared object storage, so ingestion happens once instead of once per replica. If you get those three right, the retrieval logic you wrote in the notebook works unchanged. To learn how all of it is wired together, Akamai's GitHub has a working reference implementation. - rag-langgraph-k8s-quickstart is an airline policy Q&A assistant built with FastAPI, LangChain, and LangGraph. Terraform provisions the LKE cluster, a Postgres instance with pgvector for embeddings, a second Postgres for LangGraph checkpointing, and an object storage bucket for the policy documents, in one apply. - akamai-workshop-ai-inference covers the next step, running the model yourself instead of calling an API, with prefill and decode, KV cache tradeoffs, and continuous batching under real concurrency. Both are available on Akamai's new Developer Hub, alongside their tutorials and code samples. It also links to Edge Case, their Discord, where four developer advocates architect and deploy a production app live every other Wednesday. If you create a new Akamai Cloud account, you can also get $300 in credits for joining. Join here: That said, this post assumes the retrieval logic was right to begin with, and that is doing a lot of work. Most RAG systems fail earlier, at the point where a chunk gets treated as a self-contained unit of meaning. I wrote about the two skills that fix that gap, and why the chunk is usually the wrong thing to embed. Read it below. Thanks to Akamai Cloud for partnering today!show more

Day 11/90 of Inference Engineering How does vLLM work... and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.show more

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU... THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developersshow more

sorry, they just did WHAT someone gave a machine... one disease name, the leading cause of blindness in the developed world with 1.5 million americans already in its path, and it came back pointing at a drug that has sat in pharmacies for years under a different label: 551 papers read in 30 minutes against the 294 hours a human would have needed, and the loop that did it is public on GitHub most agent setups answer one question at a time, so the ceiling on the work is the quality of the question you happened to think of this one was handed a single question and wrote the second one itself. turns out that follow-up is where the real find was: a target called ABCA1, upregulated threefold, in an experiment no human ordered i read the whole paper looking for the trick, and the trick is structural. that is the second question, and it is the gap between an assistant and a factory: - hand the loop a field rather than a task: it was given a disease, and choosing the mechanism was part of its job - make it rank before it spends: 151 papers in, ten candidate mechanisms out, scored against each other before anything touched a bench - split reading from judging, so the agent that forms the theory is a different agent from the one grading it - close every cycle on physical reality: the verdict was an experiment, and another model's opinion was never allowed to stand in for one - feed each result back as the next question rather than a log line, which is the step almost nobody builds - search what already passed inspection first: the winner was an approved compound with a safety file already on record - write down what the round learned before opening the next one, so round two starts where round one stopped my read, and i think it is the uncomfortable one: reading was the entire bottleneck in that field, and everybody spent the decade optimising the writing. people ran every physical experiment here, the analysis agent needs a domain expert writing its prompts, and the authors decline to call this the leap it resembles. the thinking got replaced, and the hands did not so the question i cannot answer for my own setup: which step of your loop still stops dead until you sit down and type something bookmark this one. the four parts that turn one model into a line that runs like this, the queue, the rooms, the write permissions and the gate, are built file by file in the piece below ↓show more

You've probably scrolled past a dozen posts about Jev... this week without anyone telling you what it actually is. It's the first model from TypeSafe, a lab started by one of the researchers behind ChatGPT. It's a decision engine: you give it options, it picks one and tells you how sure it is. It cannot write a single word, and that is the interesting part. Every other AI you use writes. That is the whole interface. So when software needs a plain yes or no, we make a model write a paragraph and then dig the answer back out of it. Fine in a chat window where a human reads it. Bad inside software, where code has to act on it. The bet is that the valuable half was never the writing. It was the deciding. That problem showed up in WordPress years ago, and it is the reason WPVibe works the way it does. The AI does the work. Anything permanent stops and waits, because a delete that skips the trash is not something software should decide on its own.show more

50% cheaper Claude inference with just one line of... code change! - Remove → model="claude-opus-4-8" - Add → model="ship-like/claude-opus-4-8" I verified the cost saving in my own terminal by invoking the same Anthropic model with the same prompt. The underlying engineering by Ship is actually interesting, and the patterns can be used in any production LLM stack. Essentially, a trained model is a frozen artifact. Every request performs the same forward-pass, whether it extracts a date or refactors a module, because the compute decision was made at training time, before the request existed. Ship makes that decision at inference time instead. After seeing a request, it searches over executions, involving single models, cascades, ensembles, or harnesses with tools, and serves the cheapest one that will match the reference model's quality. This is not a basic router, because picking a cheaper model per query doesn't ensure the cheaper model preserves the original's behavior, like output shape, tool-call patterns, and refusals. Ship measures this equivalence directly. Outputs stay distributionally indistinguishable from the reference model, not token-identical, since two calls to the same model already differ, but they are indistinguishable in capability and behavior. Of course, some requests execute cheaply and some cost Ship more than the customer pays, but the price per request is still a flat 50% off either way, so the execution-cost variance moves off the application's bill entirely. The video below depicts the cost savings and output in my real invocation, and I partnered with the team to put this together.show more

Next on Darwin: your agent's wallet is its API... key. Today an agent needs a Darwin key, and that key works for anyone holding a copy. If it leaks into a log, a repo or a shared config, someone else spends your credit. We're replacing it with the wallet the agent already has. The agent signs each request with its Solana keypair, and Darwin checks the signature before it runs anything. A signature covers one request and expires in seconds, so a copied one is useless. The private key never leaves the wallet. Credit comes from the $darwin that wallet burned, so there's no signup. Burn from the agent's wallet, point the agent at Darwin, and it can run Qwen3 30B on Darwin's GPU, swarms and the MCP server. Keys keep working for anyone who wants one.show more

Vorflux has opened its new cloud platform, allowing users... to run tasks from a plan to a merged PR on a dedicated machine. It can plan, build, test, and review on its own. > Diff reviews are done by a different model family than the one that wrote it by default. > First boot is saved as a snapshot, with all subsequent sessions waking up from that same starting point. > A browser agent walks through the real user flow, and the recording is included in the pull request. Video proof in the PR 👀show more

Jev + SERV is actually insane. We already showed... you can increase Jev's performance with SERV Reasoning. Now we're taking it further, bringing Jev-powered Decision nodes into Graph Sharding with the upcoming SERV v3. Here's a breakdown of how it works: Jev is a decision-making model. Given a task and a set of options, it predicts which path is more likely. Think of the octopus that predicted World Cup results. Jev does that for your business, except it's not luck. It weighs every option and tells you how sure it is. It does this by assigning probabilities to outcomes. It doesn't generate text on its own, so you can't expect it to create a new outcome for you. But that's also what enables it to be lightning fast and dirt cheap. For example, in customer service you can ask Jev how to triage an incoming query and route it to the correct department. It can only select from the list of departments you provide it. This also means it can't hallucinate a new outcome outside the options it's given, which makes it incredibly interesting for OpenServ. In Graph Sharding, we take a single system prompt and break it down into multiple LLM steps with deterministic input and output shapes. Some of these steps require an LLM to produce new output, while others are simply decision routers that determine the next possible path. Traditionally, LLMs are slow and expensive. Breaking a single prompt into multiple steps increases accuracy and reliability by a ton, but it also introduces latency. Jev takes on those decision nodes, which are the backbone of a business process and therefore SERV graphs, and makes them super consistent and lightning fast, lowering the overall cost and latency of graph execution. SERV Reasoning on its own is a great force multiplier for Jev because, like all other models, it works by interpreting input instructions. The clearer those instructions are, the better the model performs. That's where SERV Reasoning comes into play. Just like amplifying any other model, we also amplify the accuracy and consistency of Jev's responses. And now we're bringing Jev-powered Decision nodes into Graph Sharding with SERV v3.show more

This guy pulls $72,000 a month off a network... of 12 influencers who do not exist and not one of them lives past his webcam. He got tired of watching brands burn $8,000 a month on castings and influencer contracts and a film crew for a couple of clips. So he built a generative studio and raised a whole roster from scratch not just one face. One of his clips costs $0.40 in compute. One post from a real influencer runs $1,500. He never shows up on camera himself. On camera there is only his motion under someone else's face. No model. No film crew. No casting. Here is the exact breakdown: → First he builds not a face but an operator core: one set of his expressions and one rig and a library of personas that snap onto him → Then he locks each face and only once it holds 98% across 1,000 frames even on a sharp head turn does he push the persona into the feed. Today a blonde from the US tomorrow an Asian girl but the operator behind them all stays one → A real-time rig lays a face over his webcam. Flux and LoRA hold each persona. ElevenLabs gives each one her own voice and accent. Higgsfield and Kling build the clips → TikTok and Reels push the clips into the tags. One clip pulls 200,000 to 1,200,000 views and the face is matched to the market. It reads as a live person not a render The key move that 96% skip: they invest in the face not in the operator behind it. Generate another pretty face and join a line of 800 just like it this week. Put a live operator behind the face with timing and pauses and reactions and the persona lands in 2 seconds. Pull the operator and the face glazes over and the watch time collapses. Here is where the money actually gets pulled: → Affiliate where each persona drives offers matched to her audience at 8-15% on every sale → His own stores where the top personas front his own product at 65% margin on a $52 average order → Face licensing where a brand keeps a persona on retainer at $2,500 a month so that is $7,500 a quarter not one check → Managed network as a service where he runs other people's personas at $1,800 a month each → A network of faces 12 personas across 12 markets off one operator The numbers are silly. A clip costs $0.40 in compute. One clip drives up to 1,200,000 views straight to the offers. At a 0.7% click and a $52 order that is hundreds of sales off one video. And the whole network runs on $120 a month in software and one webcam. The first night a new persona went up her clip pulled 640,000 views in 9 hours. Before an agency would have answered one email the offers cleared 280 sales for $13,700. 3,900 comments under a girl who 72 hours earlier was a prompt and one reference frame. The comments fill with "wait is this all one guy" and that argument is the fuel the algorithm runs on. Real agencies are still casting influencers and clearing contracts for weeks. And he ships a new face before they answer one email. Not a single casting. Not a single shoot. Not a dollar on ads. Just one operator. Twelve faces over him. And the discipline to sell the brain behind the persona not the face itself. Half of you are already typing that this is too much. The other half are already building their first persona. Which half are you in?show more

AI has had exactly two scaling axes that worked... so far, and the second one is starting to look finite too the first one was pretraining: with scaling parameters and data, we got world knowledge (i.e. ChatGPT had read enough to know things), but it started saturating a while ago the second one was RL, and people had been doing RL the whole time before that: RLHF is RL but it never scaled far because it was trying to control the exact output, which tokens come out, how the text reads, but you can only push that so far before you’re just polishing RLVR dropped that constraint: giving the model a task, then checking whether the final answer is right, and ignoring everything in between -- so the model does whatever it wants in the middle and only the endpoint gets graded, and that’s much closer to actual RL and it’s what bought us planning and reasoning (arguably, tool use sits around 2.5 on this list -- while useful, it's not a different kind of thing) so one axis gave knowledge, the other gave reasoning, and both of them are one model working alone the next axis is how many models you can get working on the same problem, which is a different kind of axis than the previous two we know that multi-agent RL has always been the harder problem: I spent years in that literature and the gap between single-agent and multi-agent is definitely not incremental -- it’s a whole different class of difficulty! which is also why the derivatives are steep at the start, nobody has picked the easy wins yet... and the thing that gates this multi-agent coordination is communication: models can only coordinate as well as they can exchange information, and right now they do that by writing sentences to each other imagine what could we possibly achieve if we properly open that third axis development by letting models to exchange information in their native "language" without loosing any computational data that they produce during inferenceshow more

The cheapest way to sell an app right now... is a face and a screen I'm running it and the conversion is stupid It's one video doing two jobs → A reaction that buys you the first second → Your app running on the screen behind it Neither half has to explain the other That's the whole thing The screen recording never changes You only ever swap the reaction on top An account with 1,584 followers did 908K views and 66.5K saves with this Ten minutes of filming is a month of adsshow more

Researchers made KMeans 200x faster. And the new technique... also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.show more

Every man lives two lives. The second begins when... he realises that he can max out hypertrophy in 15 second sets. Four or five reps build muscle. The last four or five, where the bar slows on its own whether you like it or not. That is where the high-threshold motor units finally get called up. Where the cross-bridges are attached in maximum number at once. Where the fibre takes the tension it has been waiting all set for. Mechanical tension is the only language the muscle reads. It does not have a second one. Everything either side of that is filler, and somebody is paid to sell you filler. Tempo is filler. You are 30 to 50 per cent stronger lowering a weight than lifting it, so no descent on earth is slow enough to threaten you. Drag it out and all you have done is put a lighter weight on the half that counted. The pump is filler. Lactate infused straight into the blood grows nothing. Blood flow restricted on a resting muscle grows nothing. Swelling is a feeling and it gets no vote. The mind-muscle connection is filler. You cannot instruct a muscle in which fibres to use. You tell it to contract. The load decides the rest. So: a weight where the fourth rep slows by itself. Drive it. Stop one short. Rack it. Fifteen seconds. That set has already delivered everything your three sets of twelve spent forty minutes promising. Nobody has ever built a business on fifteen seconds of hard work, so they built one on everything you could bolt onto it.show more

A GIRL WALKED OFF THE TOP OF A WATERSLIDE,... THREW TWO FLIPS DOWN THE FACE OF IT, AND LANDED IN THE POOL 20,000 likes None of it is generated. That is exactly why it is worth eleven seconds, because your feed has spent a year training you to assume the opposite run it as a test. here is what is in the frame that no model reliably puts there yet: → the water. spray leaves her body at the right angle and volume for her speed, and the sheet on the slide deforms under her weight instead of flowing past her → the rotation conserves. she tucks and the spin accelerates, she opens and it slows. angular momentum is a constraint, and generators approximate it rather than obey it → the crowd reacts late. people on the platform turn after she has already gone, because they are reacting, not choreographed → the camera operator loses her. the frame lags the subject and catches up. that is a mistake, and mistakes are the expensive thing to fake → and the landing is ugly. real impacts are. generated ones almost always resolve too cleanly none of that is a checklist you can memorise, and pretending otherwise is how people get caught. every one of those tells has a shelf life. three of them were already unreliable a year ago what does not expire is where you look. continuity, physics, and mistakes - the three places a model has to simulate a system instead of reproducing a surface. surfaces are solved. systems are not, yet and the reason this gets more useful every month is not fakes. it is that real footage is starting to get accused. the cost of being wrong now runs in both directions the fastest way to calibrate your own eye is to make one yourself. image-to-video from a still, one line about the motion - Picsart runs it from a phone. you start spotting the tells about ten minutes after you have made your own this one is real. the fact that you had to check is the actual storyshow more

This is what happens when 48GB of VRAM isn't... enough. Someone put 4× NVIDIA RTX A6000 inside an HP Z8 Fury G5. That gives you 192GB of VRAM across four GPUs. But here's the interesting part: You don't magically get one 192GB GPU. The memory is split across four cards, and the software has to manage how the model is distributed between them. That's why these machines are interesting for local AI. The problem isn't always compute. Sometimes you just need somewhere to put the model. Would you build this today, or go for a newer GPU with more bandwidth?show more

whoever leaked this has bigger balls than sense Google... Research and MIT ran the same agent jobs 260 different ways for Nature last month: they held the prompts, the tools and the compute budget identical and moved nothing but the wiring between the agents, and the same work swung from 70% worse than a single agent to 80.8% better, averaging out at 0.0% i ran my own single agent against the task list first and it cleared 6 of 10 alone, already past the line where a crew starts subtracting this is Graph Engineering, the layer that decides whether a crew is worth 80% more or 70% less, and it installs into the agent you already pay for: - score your solo agent on the real task first: above roughly 45% success that study predicts zero to negative returns from any crew you put around it - under that line, put one supervisor over the fan out: crews with no correction step amplified their own errors to 17.2x the single agent rate, supervised aggregation held it to 4.4x - give every worker one output and let none of them read a peer's draft, so a wrong step reaches the supervisor instead of four other agents - run the comparison again after every model upgrade, because a better model raises your baseline and a higher baseline is what makes a crew stop paying - keep the single agent alive as the control, the only number that says the wiring is earning its calls turns out the shape does not travel: the biggest win came off a finance task under one supervisor and the worst collapse off a planning task with independent agents my position, and it is the arguable one: a crew is a bet on your own diagram, and the model you pick moves that bet less than one arrow does bookmark this, the three moves that draw those arrows before you pay for one extra call are in the post below ↓show more

The new Product Page Header image in the App... Store will not matter. First you need users to actually want to enter your App Store listing. And the reality is that a lot of users NEVER enter the listing. They search for something, see the app in the search results and download it directly from there. So the important update here is the new Search Results image. It's an app banner inside the App Store search results. And you should think about it as a persuasive ad. COPYWRITING will be what matters. Not the fancy "branding". Users are searching because they want an answer to something. Your creative needs to give them that answer immediately. And this becomes even more important with Apple Ads. You can pay to appear for the keyword, but your creative still needs to convince the user that your app is the answer they were searching for.show more

Nvidia just put a $250,000 cloud workload on your... desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇show more