正在加载视频...

视频加载失败

返回首页

🚨 Anthropic committed up to 1M TPU chips for Claude. Openai is leasing TPUs for chatgpt inference. Here's How kernels work on TPUs (deep dive 2/6 by emi) pallas is Google's answer to kernel writing. a python kernel SDK built on JAX. still very experimental (jax.experimental.pallas). on TPU it...

wafer_ai's profile picture

wafer

3,844 subscribers

33,134 次观看 • 1 个月前 •via X (Twitter)

科学技术新闻政治教育
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it the first full training setup for 3D Gaussian Splatting with zero dependencies on existing ML frameworks. It’s not just about independence, it's about control! We now manage every byte of GPU memory, which opens the door to tighter optimization and finer performance tuning. The framework footprint is minimal, without pulling in gigabytes of ML runtime code that was never designed for real-time or graphics-driven applications. A few modules, such as the metrics and 3DGUT interfaces, are still being ported, and some operations are temporarily naïve, so performance is not yet on par with master. But this refactor lays the groundwork for: - A fully self-contained binary - Fine-grained memory optimization - Easier experimentation without the weight of an ML stack We’re getting close.
0:17

First fully ML-framework-free 3D Gaussian Splatting implementation in LichtFeld Studio. I’ve completed the migration of the full training pipeline to a custom CUDA-based tensor library. No PyTorch, no LibTorch, no autograd. Every gradient is implemented by hand, either through CUDA kernels or minimal abstractions on top. This makes it the first full training setup for 3D Gaussian Splatting with zero dependencies on existing ML frameworks. It’s not just about independence, it's about control! We now manage every byte of GPU memory, which opens the door to tighter optimization and finer performance tuning. The framework footprint is minimal, without pulling in gigabytes of ML runtime code that was never designed for real-time or graphics-driven applications. A few modules, such as the metrics and 3DGUT interfaces, are still being ported, and some operations are temporarily naïve, so performance is not yet on par with master. But this refactor lays the groundwork for: - A fully self-contained binary - Fine-grained memory optimization - Easier experimentation without the weight of an ML stack We’re getting close.

MrNeRF

50,571 次观看 • 10 个月前

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.
0:14

Day 11/90 of Inference Engineering How does vLLM work and how is it used in production? Before we discuss how vLLM works internally, it helps to understand what vLLM is. At a high level, vLLM is an inference engine that is designed to serve LLMs to thousands of concurrent users efficiently while managing scarce compute and memory. The goal for vLLM is to maximize throughput and minimize latency; optimizing for the best inference economics and experience for end users. With every request from the end user, it eventually ends up in the engine core, gets scheduled alongside other requests from other concurrent users, executes on the GPU, and updates the KV cache with the new key and value vectors, and streams the tokens back to the user. The Scheduler decides what requests should execute next while continuously batching requests together to maximize GPU utilization. Continuous batching is an inference optimization that allows new requests to join a running batch as other requests finish generating tokens. This helps with keeping the GPU utilization high instead of letting it sit idle waiting for an entire batch to complete generating. After the scheduler dispatches the selected batch to the Model Executor, the Model Executor prepares the tensors and metadata required for inference, retrieves each request’s block table from KV Cache Manager, launches the optimized transformer forward pass on the GPU, computes the logits, updates the KV cache with the new key and value vectors, and finally returns the results for sampling and streaming. The KV Cache Manager uses the PagedAttention memory layout to allocate fixed-size cache blocks on demand and maintains a Free Block Queue on the CPU that tracks which blocks in the GPU’s Paged KV Cache are currently free. When a request needs additional KV cache space, the KV Cache manager takes a free block from the queue and assigns it to that request, thus avoiding an expensive search through GPU memory for available cache blocks. All of these components form the core of vLLM’s inference engine. The Scheduler determines what requests are executed, the Model Executor determines how those requests are executed, the KV Cache Manager determines where each request’s KV cache lives using the PagedAttention Memory Layout. This architecture enables vLLM to serve thousands of concurrent requests with high throughput, low latency, and efficient GPU memory utilization. Heres a little animation that visualizes everything! - I've also completed the forward pass for my mnist.c project. I had a nice chat with shrey birmiwal, such a knowledgeable guy. Excited to learn more about vLLM and implement a tiny-vLLM one day.

max fu

70,543 次观看 • 1 个月前

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.
0:12

Researchers made KMeans 200x faster. And the new technique also beats approaches like cuML and FAISS. Flash-KMeans is an IO-aware implementation of exact KMeans that redesigns the algorithm around modern GPU bottlenecks. By attacking the memory bottlenecks directly, Flash-KMeans achieves: - 33x speedup over cuML - 200x speedup over FAISS This speedup comes from how it moves through GPU memory. Standard KMeans runs in two steps, and both are bottlenecked by reads and writes to GPU memory: 1) The first step matches every point to its nearest centroid. Standard KMeans computes the full point-to-centroid distance matrix, writes it out to GPU memory, then reads it back to find each nearest centroid. That write-then-read round trip is the bottleneck. Flash-KMeans combines the distance calculation with the nearest-centroid step, so the result is computed on-chip and the full matrix is never written out. 2) The second step recomputes each centroid by averaging the points assigned to it. Standard KMeans has thousands of threads writing into the same centroid slots at once, so they stall waiting for their turn. Flash-KMeans sorts points by cluster first, turning scattered writes into sequential reductions that read and write memory in one efficient pass. Using these two optimizations at the million-scale, Flash-KMeans completes a standard KMeans iteration in a few milliseconds. The video below depicts this in action. Several reasons why this is important: KMeans has always been an offline primitive. Something you run once to preprocess data and move on. These speedups make the approach viable in several runtime-critical systems. ↳ Vector indices like FAISS use KMeans to build search indices. Faster KMeans means you can re-index dynamically as data changes. ↳ LLM quantization methods need KMeans to find optimal weight codebooks, per layer, repeatedly. What takes hours could now take minutes. ↳ MoE models need fast token routing at inference time. Flash-KMeans makes it viable to run this inside the inference loop, not just in preprocessing. I have shared the paper in the replies. That said, memory is the real constraint Flash-KMeans solves, and the problem is not just limited to clustering. The vectors a RAG system stores after indexing create similar bottlenecks. I wrote a detailed walkthrough recently on cutting this vector memory by 32x with binary quantization, querying 36M+ vectors in a few milliseconds. Read it below.

Avi Chawla

89,234 次观看 • 2 个月前

OpenClaw, but built for normal people. Sim is an open-source platform that lets you build AI agent workflows on a drag-and-drop canvas. Connect them to channels like Telegram and WhatsApp and deploy without writing a single line of code. They also have a built-in Copilot that generates entire workflows from plain English, which you can then tweak and customize in the UI. Key features: - Free and open-source (Apache 2.0) - Vector store integration for RAG-grounded agents - Self-host with one command (`npx simstudio`) - Run fully local with Ollama, no API keys needed - Supports vLLM for production-grade self-hosted inference The thing I really like about Sim is the level of control you get. You can add conditional branching, parallel execution, human-in-the-loop approval gates, and even nest workflows inside other workflows. Everything is visible on the canvas, so you know exactly what your agent is doing at every step. And you can build a workflow in Sim, deploy it as an MCP server, and plug it into any agent, including OpenClaw. I've shared the link to Sim's GitHub repo in the next tweet.
0:14

OpenClaw, but built for normal people. Sim is an open-source platform that lets you build AI agent workflows on a drag-and-drop canvas. Connect them to channels like Telegram and WhatsApp and deploy without writing a single line of code. They also have a built-in Copilot that generates entire workflows from plain English, which you can then tweak and customize in the UI. Key features: - Free and open-source (Apache 2.0) - Vector store integration for RAG-grounded agents - Self-host with one command (`npx simstudio`) - Run fully local with Ollama, no API keys needed - Supports vLLM for production-grade self-hosted inference The thing I really like about Sim is the level of control you get. You can add conditional branching, parallel execution, human-in-the-loop approval gates, and even nest workflows inside other workflows. Everything is visible on the canvas, so you know exactly what your agent is doing at every step. And you can build a workflow in Sim, deploy it as an MCP server, and plug it into any agent, including OpenClaw. I've shared the link to Sim's GitHub repo in the next tweet.

Akshay 🚀

52,426 次观看 • 6 个月前

In 2025, the AgentFlayer exploit highlighted a new category of risk in AI systems. It was not a traditional breach involving stolen credentials or broken encryption. Instead, it demonstrated how an autonomous AI agent could be manipulated into executing unintended actions by processing malicious instructions embedded inside content it automatically processes. The incident did not expose a flaw in one specific integration. It revealed a structural weakness in how many modern AI agents are built. Today’s agents are no longer passive language models. They read documents automatically, scan emails, connect to SaaS tools, access cloud storage, and execute actions across multiple systems. To be useful, they are granted meaningful permissions. That capability creates value, but it also expands the attack surface. Most agent environments operate in a trusted, plaintext execution model. Data is encrypted at rest and in transit, but it is typically decrypted during inference so the model can process it. That runtime visibility is where potential risk lies. In a zero-click scenario like AgentFlayer, an attacker can embed hidden instructions inside a document that the AI processes automatically. Because the agent may have access to connected systems such as Google Drive, Slack, or GitHub, it can potentially be influenced to retrieve sensitive information or perform unintended actions. The user does not need to click a malicious link or approve a suspicious request. Therefore, the core issue is that during execution, the system may have access to sensitive data and broad privileges, meaning whoever controls the execution environment ultimately controls access to that data. Now consider a different architectural approach. If a system is designed so that data remains protected during execution, the risk profile changes. On Nesa, privacy is enforced at the execution layer through Equivariant Encryption. Computation can occur on encrypted data, reducing the visibility surface during runtime. Sensitive inputs and models do not need to be exposed in plain text to infrastructure operators for inference to occur. This does not eliminate prompt injection, logic manipulation, or tool misuse. Encryption alone cannot prevent an agent from being instructed to take an unintended action if it has been granted that permission. What it does do is materially reduce confidentiality risk. By limiting access to readable sensitive data during execution and reducing unilateral visibility at the infrastructure layer, the potential blast radius of a successful manipulation attempt is constrained. As AI agents become more autonomous and embedded into enterprise workflows, security must move deeper into architecture. The goal is not to claim invulnerability. It is to reduce trust concentration and contain systemic exposure when failures occur. AgentFlayer was not simply a one-off exploit. It was a reminder that in autonomous systems, execution-layer design determines how risk propagates.
0:11

In 2025, the AgentFlayer exploit highlighted a new category of risk in AI systems. It was not a traditional breach involving stolen credentials or broken encryption. Instead, it demonstrated how an autonomous AI agent could be manipulated into executing unintended actions by processing malicious instructions embedded inside content it automatically processes. The incident did not expose a flaw in one specific integration. It revealed a structural weakness in how many modern AI agents are built. Today’s agents are no longer passive language models. They read documents automatically, scan emails, connect to SaaS tools, access cloud storage, and execute actions across multiple systems. To be useful, they are granted meaningful permissions. That capability creates value, but it also expands the attack surface. Most agent environments operate in a trusted, plaintext execution model. Data is encrypted at rest and in transit, but it is typically decrypted during inference so the model can process it. That runtime visibility is where potential risk lies. In a zero-click scenario like AgentFlayer, an attacker can embed hidden instructions inside a document that the AI processes automatically. Because the agent may have access to connected systems such as Google Drive, Slack, or GitHub, it can potentially be influenced to retrieve sensitive information or perform unintended actions. The user does not need to click a malicious link or approve a suspicious request. Therefore, the core issue is that during execution, the system may have access to sensitive data and broad privileges, meaning whoever controls the execution environment ultimately controls access to that data. Now consider a different architectural approach. If a system is designed so that data remains protected during execution, the risk profile changes. On Nesa, privacy is enforced at the execution layer through Equivariant Encryption. Computation can occur on encrypted data, reducing the visibility surface during runtime. Sensitive inputs and models do not need to be exposed in plain text to infrastructure operators for inference to occur. This does not eliminate prompt injection, logic manipulation, or tool misuse. Encryption alone cannot prevent an agent from being instructed to take an unintended action if it has been granted that permission. What it does do is materially reduce confidentiality risk. By limiting access to readable sensitive data during execution and reducing unilateral visibility at the infrastructure layer, the potential blast radius of a successful manipulation attempt is constrained. As AI agents become more autonomous and embedded into enterprise workflows, security must move deeper into architecture. The goal is not to claim invulnerability. It is to reduce trust concentration and contain systemic exposure when failures occur. AgentFlayer was not simply a one-off exploit. It was a reminder that in autonomous systems, execution-layer design determines how risk propagates.

Nesa

17,038 次观看 • 5 个月前

The largest theft in history has already happened. The people behind it just cannot open what they stole yet. Right now, intelligence agencies and criminal groups are quietly copying the world's encrypted data, bank records, medical files, state secrets, private messages, and storing every byte untouched. They cannot read any of it. They are collecting it anyway, because they know the key is about to be invented. The strategy has a name, harvest now, decrypt later, and in 2026 it stopped being theory. Washington declared this the Year of Quantum Security in January, backed by the FBI, the NSA, and NIST. Canada ordered every federal agency to file a migration plan by April. Europe set its deadline for December. Governments do not impose operational deadlines on a someday problem. They do it when the clock is already running. Here is what moved the clock. Every password, every transfer, every secret on Earth is protected by one assumption, that a certain math problem is too hard to solve. Quantum computers solve exactly that problem. For years the machine that could do it looked decades away. Then in late 2025 Google's Willow chip cracked the hardest part of building one, and in March 2026 Google's own researchers estimated that breaking the encryption behind Bitcoin might take fewer than 500,000 qubits, down from 20 million, and could run in minutes. The day this becomes real has a name, Q-Day, and the latest estimates place it between 2030 and 2033. Now make it concrete. Roughly 6.5 million Bitcoin, about a third of every coin that will ever exist, worth close to 500 billion dollars, sit in addresses that have already exposed the very key a quantum computer needs. That includes the coins of Satoshi, the anonymous creator. On Q-Day they become, in the researchers' own word, trivially stealable. It would not look like a crash or a whale selling. It would look like half a trillion dollars of the most secure money ever built simply walking out the door. The asset designed to trust no one and no institution turns out to rest on a single unverified bet, that one math problem stays hard forever. This is what sits beneath the entire digital world. A bank balance, a Bitcoin, a classified cable, all of it is real only because of a proof you supposedly cannot forge. Quantum breaks the proof. Everything we call secure is true only until someone finally checks, and for the first time the check is visible on the horizon. You cannot know whether your data has already been copied. You cannot know the exact day the key arrives. The trust holding up the digital age is a clock counting down to a zero no one can see. The honest counter matters. No machine on Earth can break this encryption today, and serious cryptographers still argue the real threat is a decade or more away. The timeline is far from certain. Quantum-safe codes already exist, the migration has started, and Bitcoin can move its coins to safety before Q-Day if it acts in time. The danger is not that everything breaks tomorrow. It is that anything which must stay secret into the 2030s, a state secret, an identity, a private key, is being stolen today and is already on the clock. The breach is not coming. It is already here, sitting in storage, perfectly encrypted, waiting for a machine that does not exist yet to read it out loud. Research and opinion, not investment advice.
0:10

The largest theft in history has already happened. The people behind it just cannot open what they stole yet. Right now, intelligence agencies and criminal groups are quietly copying the world's encrypted data, bank records, medical files, state secrets, private messages, and storing every byte untouched. They cannot read any of it. They are collecting it anyway, because they know the key is about to be invented. The strategy has a name, harvest now, decrypt later, and in 2026 it stopped being theory. Washington declared this the Year of Quantum Security in January, backed by the FBI, the NSA, and NIST. Canada ordered every federal agency to file a migration plan by April. Europe set its deadline for December. Governments do not impose operational deadlines on a someday problem. They do it when the clock is already running. Here is what moved the clock. Every password, every transfer, every secret on Earth is protected by one assumption, that a certain math problem is too hard to solve. Quantum computers solve exactly that problem. For years the machine that could do it looked decades away. Then in late 2025 Google's Willow chip cracked the hardest part of building one, and in March 2026 Google's own researchers estimated that breaking the encryption behind Bitcoin might take fewer than 500,000 qubits, down from 20 million, and could run in minutes. The day this becomes real has a name, Q-Day, and the latest estimates place it between 2030 and 2033. Now make it concrete. Roughly 6.5 million Bitcoin, about a third of every coin that will ever exist, worth close to 500 billion dollars, sit in addresses that have already exposed the very key a quantum computer needs. That includes the coins of Satoshi, the anonymous creator. On Q-Day they become, in the researchers' own word, trivially stealable. It would not look like a crash or a whale selling. It would look like half a trillion dollars of the most secure money ever built simply walking out the door. The asset designed to trust no one and no institution turns out to rest on a single unverified bet, that one math problem stays hard forever. This is what sits beneath the entire digital world. A bank balance, a Bitcoin, a classified cable, all of it is real only because of a proof you supposedly cannot forge. Quantum breaks the proof. Everything we call secure is true only until someone finally checks, and for the first time the check is visible on the horizon. You cannot know whether your data has already been copied. You cannot know the exact day the key arrives. The trust holding up the digital age is a clock counting down to a zero no one can see. The honest counter matters. No machine on Earth can break this encryption today, and serious cryptographers still argue the real threat is a decade or more away. The timeline is far from certain. Quantum-safe codes already exist, the migration has started, and Bitcoin can move its coins to safety before Q-Day if it acts in time. The danger is not that everything breaks tomorrow. It is that anything which must stay secret into the 2030s, a state secret, an identity, a private key, is being stolen today and is already on the clock. The breach is not coming. It is already here, sitting in storage, perfectly encrypted, waiting for a machine that does not exist yet to read it out loud. Research and opinion, not investment advice.

Shanaka Anslem Perera ⚡

185,515 次观看 • 2 个月前

Free NVIDIA GPU with 16 GB VRAM GPU for Running Local LLMs! If you want to master local LLMs but you're waiting until you can afford a $1,500 GPU, you're honestly not going to make it. The open source AI ecosystem is moving way too fast for you to wait on your budget to catch up. Especially when you can build a bleeding edge inference engine from scratch right now, completely for free. You don't need a heavy local rig to start. Google is literally letting you use an enterprise grade NVIDIA Tesla T4 GPU for $0/hour. At standard cloud computing rates (~$0.20/hr), Google Colab’s 4 hour daily free tier hands you roughly $24 worth of data center tier GPU compute every single month. And most people just waste it. Let’s talk about the hardware you get access to for free. The NVIDIA Tesla T4 is an absolute workhorse: - Architecture: NVIDIA Turing (TU104) - VRAM: 16GB GDDR6 (320 GB/s bandwidth) - Compute: 320 Tensor Cores | 2560 CUDA Cores - Performance: 130 TOPS INT8 | 8.1 TFLOPS FP32 - Power: Sipping energy at a max 70W TDP This is the exact same hardware I used to run DeepMind's Gemma 4 26B A4B QAT MoE at a 250,000 context window without a single Out Of Memory (OOM) crash. If you have a web browser and 10 minutes, you have everything you need. I’ve put together a fully documented, cell by cell Google Colab notebook that teaches you exactly how to do this. Here is what the notebook actually teaches you: - How to provision an Ubuntu Linux environment with CUDA 13.0 and verify your driver stack. - How to pull the source code and compile the latest llama.cpp C++ binaries from scratch, specifically optimizing the build for your exact GPU using the -DCMAKE_CUDA_ARCHITECTURES=native flag. - How to directly download quantized local LLMs (GGUF format) straight from HuggingFace using the CLI. - How to manage 16GB VRAM limits, offload neural network layers to the GPU, and push massive context windows. Compile raw llama.cpp, ollama run a model, or spin up the LM Studio CLI. Pick whatever stack you are comfortable with. just start building. No hardware. No credit card. No excuses. Bookmark this post right now so you don't lose the tutorial. Even if you don't have time to run it today, you are going to want this workflow in your engineering toolkit. The link to the free Colab Notebook is in the comments below. Lemme know if you need more tutorials like this.
0:11

Free NVIDIA GPU with 16 GB VRAM GPU for Running Local LLMs! If you want to master local LLMs but you're waiting until you can afford a $1,500 GPU, you're honestly not going to make it. The open source AI ecosystem is moving way too fast for you to wait on your budget to catch up. Especially when you can build a bleeding edge inference engine from scratch right now, completely for free. You don't need a heavy local rig to start. Google is literally letting you use an enterprise grade NVIDIA Tesla T4 GPU for $0/hour. At standard cloud computing rates (~$0.20/hr), Google Colab’s 4 hour daily free tier hands you roughly $24 worth of data center tier GPU compute every single month. And most people just waste it. Let’s talk about the hardware you get access to for free. The NVIDIA Tesla T4 is an absolute workhorse: - Architecture: NVIDIA Turing (TU104) - VRAM: 16GB GDDR6 (320 GB/s bandwidth) - Compute: 320 Tensor Cores | 2560 CUDA Cores - Performance: 130 TOPS INT8 | 8.1 TFLOPS FP32 - Power: Sipping energy at a max 70W TDP This is the exact same hardware I used to run DeepMind's Gemma 4 26B A4B QAT MoE at a 250,000 context window without a single Out Of Memory (OOM) crash. If you have a web browser and 10 minutes, you have everything you need. I’ve put together a fully documented, cell by cell Google Colab notebook that teaches you exactly how to do this. Here is what the notebook actually teaches you: - How to provision an Ubuntu Linux environment with CUDA 13.0 and verify your driver stack. - How to pull the source code and compile the latest llama.cpp C++ binaries from scratch, specifically optimizing the build for your exact GPU using the -DCMAKE_CUDA_ARCHITECTURES=native flag. - How to directly download quantized local LLMs (GGUF format) straight from HuggingFace using the CLI. - How to manage 16GB VRAM limits, offload neural network layers to the GPU, and push massive context windows. Compile raw llama.cpp, ollama run a model, or spin up the LM Studio CLI. Pick whatever stack you are comfortable with. just start building. No hardware. No credit card. No excuses. Bookmark this post right now so you don't lose the tutorial. Even if you don't have time to run it today, you are going to want this workflow in your engineering toolkit. The link to the free Colab Notebook is in the comments below. Lemme know if you need more tutorials like this.

Alok

178,744 次观看 • 1 个月前

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:
0:16

HTML Artifacts are a big part of how I work with agents now. Artifacts can be more than just static files. When combined with agents, they can take action or help you take action. This unlocks all kinds of interesting ways to work with agents. This is clearly the future. Check out this writing and scheduler artifact I built in a few minutes. It uses a bit of HTML and JS. All the data is in markdown (Obsidian vaults), so the agent can access and modify it at any time. No DB needed. No sophisticated functionalities. The agent decides all that for me based on the skills, context, and memory it has access to. The best part about this simple stack is that all the important information stays with me. This has allowed me to build a recursive self-improving system and automations that can better tap into coding agents like Codex or Claude Code. I could have paid or built an entire app for scheduling posts, and there are so many of them out there. But I don't need to. I've realized a simple artifact does the job. And the simplicity of it is actually an advantage. Very little maintenance for very high returns on personalization, time, and efficiency. The other benefit of this is that I can add features as I please. That level of personalization feels magical, and we should all be pursuing more of it. All of this just keeps compounding. Of course, this example is just about writing. But I have similar artifacts for research, design, experimentation, evaluation, and so much more. And no, I didn't actually publish the post example I shared in the clip. It was just for demonstration purposes. I actually spend more time than this when writing together with agents. Lastly, having built my own agent orchestrator tool has made me realize that simplifying the tool stack is a superpower. If you are curious about how all this works, I will do a live session next week:

elvis

18,374 次观看 • 3 个月前

$IREN "we haven't disclosed the specific amount of GPUs" 1. 🤮 reminds me of $NBIS 2. Setting a terrible precedent here for future deals 3. Making it purposely difficult, to not let analysts properly value your 2027 revenue 4. Increasing the polarized view on IREN by the market However: "approximately 60MW of air-cooled Blackwells" 1. You typically don't talk about gross capacity in a deployment like this 2. If it would be gross capacity, the GPU hour rate at IT level would be crazy high (at PUE 1.2, $680m / 50 = 13.6m/MW) 3. At 60MW IT load, and ~14kW draw at DGX server level, we can get to ~4,286 DGX systems with 8 GPUs per. 4. Based on this we can conclude that 60MW of IT load can run approximately 34k DGX B300. 5. 34k DGX B300 at $680m/yr, would represent a GPU hour price of $2.28 Now this is the problem with not disclosing your GPU quantity. You purposely make your business model look bad, because by approach, you get to a GPU hour price that would imply a payback period of 4 years, where only the last year of the contract is 100% margin. But of course, we can also take "the glass is half full" approach. IREN has ordered 50K B300s from Dell. They have 2 purchase orders for this, 1 between Dell Canada and IE CA Leasing Ltd for 4 phases, and 1 between Dell USA and IE US Hardware 1 Inc (amended from IE US Hardware 4 Inc on April 27, 2026). The order for Canada is divided in 4 phases, and are going to Mackenzie for 80MW of gross capacity, which happens to be 4 buildings of 20MW. The order for Childress is divided in 2 phases, and are going to DC35 and DC36, (as depicted in the earnings presentation) and those are 50MW gross. The purchase price of the order for Childress was $1.2B, and for Canada it was $2.3B If we go with 50,000 B300s for a total of $3.5B then $1.2 would represent 34.285% of the 50,000 GPUs, or 17,140 B300s rounded down. For this calculation I will consider that $IREN will deploy 17,140 GPUs in 50MW gross capacity in DC35 and DC36 of block 3 in Childress.. That would imply at 1.2 PUE, IREN can run 17,140 B300s in 41.67MW IT load. Now by that ratio, they can run 24,680 GPUs in 60MW IT load — a massive difference with 34k units through the Nvidia DGX reference calculation. If common sense is applied, you can still get to 2 completely different outcomes, that show a difference of more than 9k GPUs. The GPU hour rate at 24.68k GPUs would be $3.145 per B300, as MASSIVE difference from the earlier calculated $2.28. Sure, the DGX system may be a factor here. And I'm sure that the reality is somewhere in the middle. But I personally hate this as an investor, to be unable to calculate profitability on unit economic basis. After all, contracts are signed on a $/GPU hour basis. Why hide this from your investors? Not being able to calculate payback periods, unable to calculate ROIC. And most importantly, we cannot properly assess the $NVDA deal on a contract basis. I really hope the payback period of this contract is not 4 years. I want the glass to be half full, but by starting to censor the purchases, IREN is taking a step in the wrong direction. Not a fan of this.
0:14

$IREN "we haven't disclosed the specific amount of GPUs" 1. 🤮 reminds me of $NBIS 2. Setting a terrible precedent here for future deals 3. Making it purposely difficult, to not let analysts properly value your 2027 revenue 4. Increasing the polarized view on IREN by the market However: "approximately 60MW of air-cooled Blackwells" 1. You typically don't talk about gross capacity in a deployment like this 2. If it would be gross capacity, the GPU hour rate at IT level would be crazy high (at PUE 1.2, $680m / 50 = 13.6m/MW) 3. At 60MW IT load, and ~14kW draw at DGX server level, we can get to ~4,286 DGX systems with 8 GPUs per. 4. Based on this we can conclude that 60MW of IT load can run approximately 34k DGX B300. 5. 34k DGX B300 at $680m/yr, would represent a GPU hour price of $2.28 Now this is the problem with not disclosing your GPU quantity. You purposely make your business model look bad, because by approach, you get to a GPU hour price that would imply a payback period of 4 years, where only the last year of the contract is 100% margin. But of course, we can also take "the glass is half full" approach. IREN has ordered 50K B300s from Dell. They have 2 purchase orders for this, 1 between Dell Canada and IE CA Leasing Ltd for 4 phases, and 1 between Dell USA and IE US Hardware 1 Inc (amended from IE US Hardware 4 Inc on April 27, 2026). The order for Canada is divided in 4 phases, and are going to Mackenzie for 80MW of gross capacity, which happens to be 4 buildings of 20MW. The order for Childress is divided in 2 phases, and are going to DC35 and DC36, (as depicted in the earnings presentation) and those are 50MW gross. The purchase price of the order for Childress was $1.2B, and for Canada it was $2.3B If we go with 50,000 B300s for a total of $3.5B then $1.2 would represent 34.285% of the 50,000 GPUs, or 17,140 B300s rounded down. For this calculation I will consider that $IREN will deploy 17,140 GPUs in 50MW gross capacity in DC35 and DC36 of block 3 in Childress.. That would imply at 1.2 PUE, IREN can run 17,140 B300s in 41.67MW IT load. Now by that ratio, they can run 24,680 GPUs in 60MW IT load — a massive difference with 34k units through the Nvidia DGX reference calculation. If common sense is applied, you can still get to 2 completely different outcomes, that show a difference of more than 9k GPUs. The GPU hour rate at 24.68k GPUs would be $3.145 per B300, as MASSIVE difference from the earlier calculated $2.28. Sure, the DGX system may be a factor here. And I'm sure that the reality is somewhere in the middle. But I personally hate this as an investor, to be unable to calculate profitability on unit economic basis. After all, contracts are signed on a $/GPU hour basis. Why hide this from your investors? Not being able to calculate payback periods, unable to calculate ROIC. And most importantly, we cannot properly assess the $NVDA deal on a contract basis. I really hope the payback period of this contract is not 4 years. I want the glass to be half full, but by starting to censor the purchases, IREN is taking a step in the wrong direction. Not a fan of this.

Frans Bakker

148,167 次观看 • 3 个月前

"PRICE IS WHAT YOU PAY. VALUE IS WHAT YOU GET." I keep buying $Kekec and I have a strong conviction. Here's Why: While the market is down, and Kekec is declining with it, there are data points that few are considering. Kekec borned in October and since then has been posting a different and original 30-second video every day, which I find extremely funny. For the past couple of months, they have also been posting daily on Instagram, and the attention on Kekec (which doesn't present itself on social media as a memecoin) is growing, moreover, it's increasing exponentially. The number of followers is increasing by about 500-1000 a day. This is largely due to the fact that they are not just focused on the main account but have several others that post reels and redirect to the main one. In short, an excellent strategy to keep growing more and more. Instagram link: Guess What? Not only are the followers increasing, but the team's workload is also growing. In fact, for a little over a month, they have also started pushing on YouTube, and the data here is promising as well. YouTube link: If we want to make a comparison, we can take Pudgy Penguins as an example, which has shown it can reach millions and millions of users without mentioning that they are a WEB3 company that owns an NFT collection. Or, if we want to be more appropriate by comparing one memecoin to another, we could take PONKE. Thanks to the use of social media and the quality of their content, they managed to achieve incredible numbers, which then translated into an increase in the coin's price. Kekec came before PONKE, but that doesn't necessarily mean it's better than PONKE. I believe PONKE is unbeatable in terms of content, but I want to make you reflect on an important point. PONKE came after KEKEC, and after PONKE's success, many coins have emerged trying to imitate it. One of KEKEC's strengths, in my opinion, is precisely the fact that it leverages social media without being a copy-paste. Instead, it is a unique meme derived from a 90's film, and it uses a unique form of content. In short, KEKEC > KEKEC and no one else. I want to conclude by suggesting you follow them on Instagram and evaluate not only the exponential growth of their followers day by day but also observe how the views of each reel increase accordingly. Pay special attention to the comments. Many of the people commenting have no idea what it is, and you can see from the comments how Kekec generates particular emotions in people—strange but still emotions. Personally, I believe that when something is unique and even very strange, it needs time to be adopted. However, once it happens, it usually explodes and spreads like never before. A few days ago, a Kekec video was posted by a very popular meme page. They probably don't know what Kekec is about but thought the video could spark interest among their followers. How many other pages will do the same? Lastly, but not least, I want to point out how Kekec maintains a good market cap despite everything that has happened in the crypto world since October 2023. As far as I know and have personally observed, everything is extremely organic. There is no cabal behind it, and the quality is not reflected in a single jpeg but in work that has been ongoing daily for months. Every day they work harder, and the quality of their videos grows as well. I have no affiliations with the team, but I believe that Kekec truly deserves more in this world where we push celebrity or cabal-backed coins to hundreds of millions in market cap. I keep buying because the numbers suggest so. Don't just evaluate the chart (price), evaluate the data (value). BÂLKÂN DWÂRF
0:15

"PRICE IS WHAT YOU PAY. VALUE IS WHAT YOU GET." I keep buying $Kekec and I have a strong conviction. Here's Why: While the market is down, and Kekec is declining with it, there are data points that few are considering. Kekec borned in October and since then has been posting a different and original 30-second video every day, which I find extremely funny. For the past couple of months, they have also been posting daily on Instagram, and the attention on Kekec (which doesn't present itself on social media as a memecoin) is growing, moreover, it's increasing exponentially. The number of followers is increasing by about 500-1000 a day. This is largely due to the fact that they are not just focused on the main account but have several others that post reels and redirect to the main one. In short, an excellent strategy to keep growing more and more. Instagram link: Guess What? Not only are the followers increasing, but the team's workload is also growing. In fact, for a little over a month, they have also started pushing on YouTube, and the data here is promising as well. YouTube link: If we want to make a comparison, we can take Pudgy Penguins as an example, which has shown it can reach millions and millions of users without mentioning that they are a WEB3 company that owns an NFT collection. Or, if we want to be more appropriate by comparing one memecoin to another, we could take PONKE. Thanks to the use of social media and the quality of their content, they managed to achieve incredible numbers, which then translated into an increase in the coin's price. Kekec came before PONKE, but that doesn't necessarily mean it's better than PONKE. I believe PONKE is unbeatable in terms of content, but I want to make you reflect on an important point. PONKE came after KEKEC, and after PONKE's success, many coins have emerged trying to imitate it. One of KEKEC's strengths, in my opinion, is precisely the fact that it leverages social media without being a copy-paste. Instead, it is a unique meme derived from a 90's film, and it uses a unique form of content. In short, KEKEC > KEKEC and no one else. I want to conclude by suggesting you follow them on Instagram and evaluate not only the exponential growth of their followers day by day but also observe how the views of each reel increase accordingly. Pay special attention to the comments. Many of the people commenting have no idea what it is, and you can see from the comments how Kekec generates particular emotions in people—strange but still emotions. Personally, I believe that when something is unique and even very strange, it needs time to be adopted. However, once it happens, it usually explodes and spreads like never before. A few days ago, a Kekec video was posted by a very popular meme page. They probably don't know what Kekec is about but thought the video could spark interest among their followers. How many other pages will do the same? Lastly, but not least, I want to point out how Kekec maintains a good market cap despite everything that has happened in the crypto world since October 2023. As far as I know and have personally observed, everything is extremely organic. There is no cabal behind it, and the quality is not reflected in a single jpeg but in work that has been ongoing daily for months. Every day they work harder, and the quality of their videos grows as well. I have no affiliations with the team, but I believe that Kekec truly deserves more in this world where we push celebrity or cabal-backed coins to hundreds of millions in market cap. I keep buying because the numbers suggest so. Don't just evaluate the chart (price), evaluate the data (value). BÂLKÂN DWÂRF

m0ment0

133,250 次观看 • 2 年前

Elon Musk gave the entire entertainment industry its expiration date, and he is the one building the thing that kills it. Musk: “My guess is that we see the first compelling half hour, pure AI show next year.” Next year. A complete show generated entirely by AI. No writers. No actors. No cameras. No sets. No crew. No studio. Just a prompt and enough compute to render a reality that never physically existed. And shows are the easy part. Musk: “I say probably we’re maybe three years away from AI does the whole video game.” A show plays the same way every time. A game has to generate a living world that reacts to every decision in real time across every single frame. That is a fundamentally harder class of problem. And Musk put three years on it. Right now a single AAA title takes seven years and half a billion dollars across thousands of engineers and artists just to ship it. Musk is describing a world where one person types a paragraph and gets something comparable. The entire value proposition of a multi-billion dollar industry lives inside that gap. And it closes in thirty-six months. But the prediction is not the story. The person making it is. This is not an analyst speculating from the sidelines. This is the man building the largest AI compute clusters on the planet. The man who built xAI from zero in under two years. The man stacking hundreds of thousands of GPUs into facilities designed to do exactly what he is describing. When Musk says three years, he is not guessing about what someone else might eventually ship. He is reading you a delivery date off his own roadmap. Every media company on Earth is valued on a single assumption. That quality content is expensive and difficult to produce at scale. That one assumption is the structural foundation underneath every studio, every network, and every publisher in existence. Musk is dismantling it with raw compute. The studios still parading thousand-person production teams are not demonstrating strength. They are advertising the exact cost structure that one person with a prompt and a GPU allocation is about to make irrelevant. And it does not stop at entertainment. If AI can generate an interactive world that responds to human input in real time, it can generate anything. Advertising. Architecture. Training simulations. Product design. Every industry built on humans manually constructing visual experiences frame by frame is sitting on the same countdown Musk just read out loud. Now zoom out. Because this is not just an industry story. For the entire history of human civilization, the distance between imagining a world and actually creating one required thousands of people, millions of hours, and billions of dollars. That distance built Hollywood. That distance built the gaming industry. That distance made content scarce and studios powerful. Musk is collapsing that distance to zero. When the gap between imagining something and it existing disappears, every business model built on the difficulty of creation disappears with it. That is not disruption. That is a full inversion of how human beings create. Musk did not make a casual prediction on that podcast. He told you what he is building. He told you the timeline. And he told you which industries do not survive it. The entertainment industry is still debating whether this future is real. Musk is not part of that debate. He is building. And he just told you the delivery date.
0:17

Elon Musk gave the entire entertainment industry its expiration date, and he is the one building the thing that kills it. Musk: “My guess is that we see the first compelling half hour, pure AI show next year.” Next year. A complete show generated entirely by AI. No writers. No actors. No cameras. No sets. No crew. No studio. Just a prompt and enough compute to render a reality that never physically existed. And shows are the easy part. Musk: “I say probably we’re maybe three years away from AI does the whole video game.” A show plays the same way every time. A game has to generate a living world that reacts to every decision in real time across every single frame. That is a fundamentally harder class of problem. And Musk put three years on it. Right now a single AAA title takes seven years and half a billion dollars across thousands of engineers and artists just to ship it. Musk is describing a world where one person types a paragraph and gets something comparable. The entire value proposition of a multi-billion dollar industry lives inside that gap. And it closes in thirty-six months. But the prediction is not the story. The person making it is. This is not an analyst speculating from the sidelines. This is the man building the largest AI compute clusters on the planet. The man who built xAI from zero in under two years. The man stacking hundreds of thousands of GPUs into facilities designed to do exactly what he is describing. When Musk says three years, he is not guessing about what someone else might eventually ship. He is reading you a delivery date off his own roadmap. Every media company on Earth is valued on a single assumption. That quality content is expensive and difficult to produce at scale. That one assumption is the structural foundation underneath every studio, every network, and every publisher in existence. Musk is dismantling it with raw compute. The studios still parading thousand-person production teams are not demonstrating strength. They are advertising the exact cost structure that one person with a prompt and a GPU allocation is about to make irrelevant. And it does not stop at entertainment. If AI can generate an interactive world that responds to human input in real time, it can generate anything. Advertising. Architecture. Training simulations. Product design. Every industry built on humans manually constructing visual experiences frame by frame is sitting on the same countdown Musk just read out loud. Now zoom out. Because this is not just an industry story. For the entire history of human civilization, the distance between imagining a world and actually creating one required thousands of people, millions of hours, and billions of dollars. That distance built Hollywood. That distance built the gaming industry. That distance made content scarce and studios powerful. Musk is collapsing that distance to zero. When the gap between imagining something and it existing disappears, every business model built on the difficulty of creation disappears with it. That is not disruption. That is a full inversion of how human beings create. Musk did not make a casual prediction on that podcast. He told you what he is building. He told you the timeline. And he told you which industries do not survive it. The entertainment industry is still debating whether this future is real. Musk is not part of that debate. He is building. And he just told you the delivery date.

Dustin

22,390 次观看 • 1 个月前

Stateless History Node is almost like a regular Ethereum node, but it doesn't store state and it doesn't have EVM execution. It's used only for syncing events and thus - is faster and gives you FREE INDEXING. You don't have to pay 6 figures for RPC anymore! Just spin up a Stateless History Node, plug rindexer or Ponder there, and enjoy free (AND FAST!!) indexing! This node is syncing >1000 blocks per second at my local pc (less than 6hrs for the whole Ethereum), and it should use less than 200GB - which means you can host it on a MacMini, Hetzner or whatever. You can futhermore filter that by using block ranges or bloom filters, etc - I haven't developed this yet. What you see is a proof of concept. It works via native devp2p 'eth' protocol, but with EIP4444 and The Prune we would have to also support era1 archives and Portal Network. But so far it works - there are plenty of peers serving historical receipts, and they serve them FAST! If you run Stateless History Node you can also serve the blocks and receipts - so that could help to preserve archival data too. For now there is no data validation yet (and even no data storage - that's a very early PoC), but we can verify validity of chain by simultaneously running a lightweight CL node (or not lightweight if you're extremely paranoid). And then support verifying the hashes of receipts and blocks with their parents, maintaining full integrity and zero trust. It's also written in rust, btw. So, I guess, at least for Ethereum Mainnet the era of RPC's pumping moneybags is over - there's finally a local, trustless and free indexing alternative available. Too sad this won't work for Optimism / Base , cause despite introducing P2P after Bedrock - they haven't enabled receipts transfer in the protocol (or at least I couldn't find one). Arbitrum is even sadder - I don't believe there is a P2P layer at all - you just have to run your own node, hold state and execute blocks to get events. There is hope - Paradigm recently released Ress - stateless execution, but it requires nodes to support Witness preparation & exchange - but this could work for L2s - cause the main blocker for local RPCs rn is huge state (VPS with TB storage cost a lot), and the second blocker is EVM forks makes it hard to hold a node - it needs to be maintained, upgraded, etc. Ress at least solves the state part. But anyways, I will try to continue working on this and release some MVP version with RPC endpoint and data storage soon - follow the updates!
0:16

Stateless History Node is almost like a regular Ethereum node, but it doesn't store state and it doesn't have EVM execution. It's used only for syncing events and thus - is faster and gives you FREE INDEXING. You don't have to pay 6 figures for RPC anymore! Just spin up a Stateless History Node, plug rindexer or Ponder there, and enjoy free (AND FAST!!) indexing! This node is syncing >1000 blocks per second at my local pc (less than 6hrs for the whole Ethereum), and it should use less than 200GB - which means you can host it on a MacMini, Hetzner or whatever. You can futhermore filter that by using block ranges or bloom filters, etc - I haven't developed this yet. What you see is a proof of concept. It works via native devp2p 'eth' protocol, but with EIP4444 and The Prune we would have to also support era1 archives and Portal Network. But so far it works - there are plenty of peers serving historical receipts, and they serve them FAST! If you run Stateless History Node you can also serve the blocks and receipts - so that could help to preserve archival data too. For now there is no data validation yet (and even no data storage - that's a very early PoC), but we can verify validity of chain by simultaneously running a lightweight CL node (or not lightweight if you're extremely paranoid). And then support verifying the hashes of receipts and blocks with their parents, maintaining full integrity and zero trust. It's also written in rust, btw. So, I guess, at least for Ethereum Mainnet the era of RPC's pumping moneybags is over - there's finally a local, trustless and free indexing alternative available. Too sad this won't work for Optimism / Base , cause despite introducing P2P after Bedrock - they haven't enabled receipts transfer in the protocol (or at least I couldn't find one). Arbitrum is even sadder - I don't believe there is a P2P layer at all - you just have to run your own node, hold state and execute blocks to get events. There is hope - Paradigm recently released Ress - stateless execution, but it requires nodes to support Witness preparation & exchange - but this could work for L2s - cause the main blocker for local RPCs rn is huge state (VPS with TB storage cost a lot), and the second blocker is EVM forks makes it hard to hold a node - it needs to be maintained, upgraded, etc. Ress at least solves the state part. But anyways, I will try to continue working on this and release some MVP version with RPC endpoint and data storage soon - follow the updates!

Convergence Boy

29,799 次观看 • 7 个月前

AI just hit a wall that no amount of money can move. The planet itself. There is not enough power, water, or land on Earth to build the data centers the AI race now demands. So the most valuable bet in artificial intelligence is no longer a chip company or a model. It is a rocket company. The plan is to leave. In January, SpaceX filed with the FCC to launch up to 1 million solar-powered data center satellites into orbit. In February it bought xAI, the maker of Grok, folding an entire frontier AI lab into a rocket company in the largest corporate merger ever recorded. On June 8 it unveiled the AI1, a compute satellite with a 70-meter wingspan, wider than a Boeing 747, powered by the sun, cooled by the vacuum of space, and wired to the ground through Starlink. Four days later it went public in the largest IPO in history, near 1.77 trillion dollars, touched 2.1 trillion on its first day, raised close to 86 billion, and made one man the first trillionaire alive. Now read the direction of that merger, because it is the whole story. A rocket company bought the AI lab. Not the reverse. For three years everyone assumed the constraint on AI was chips, or data, or talent. It is none of them anymore. It is energy and heat and dirt. The head of Anthropic said his company grew faster than the exponential, 80 times in a single year, and that is exactly why it ran out of compute. The answer was not to build more data centers in Virginia. It was to leave the atmosphere, where the sun never sets and a solar panel does five times the work. The moat in artificial intelligence is no longer the model. It is the launch. And the first rent is already being paid. A rival lab, Anthropic, is reported to be sending roughly 1.25 billion dollars a month to Musk for compute. Google near 920 million. If intelligence moves to orbit, the company that owns the only affordable road there becomes the landlord of the next layer of the internet, the way one bookstore became the landlord of the cloud. The merger is the proof of concept. The IPO is the war chest. Those monthly checks are the lease. Here is the part the price tag does not want you to read. Close to a trillion dollars of that valuation rests on orbital data centers that do not yet exist, and on a chip factory, Terafab, that SpaceX's own public filing calls a general framework with no binding deal, one that may not achieve commercial viability. Musk said it on camera. This is not a promise. The largest IPO ever written is priced on a future the filing itself cannot verify. The other side is just as real. Compute in orbit costs about four times what it costs on the ground today, and the curve may not cross for fifteen years. The machines that print the chips are backordered for years. Shedding heat in a vacuum at this scale has never been done. Musk's timelines have a long history of meaning later. And Bezos is racing the same orbit with a constellation of 51,600 satellites of his own. But strip it all away and the trade underneath is one sentence. Earth has run out of room for intelligence, and whoever owns the road off the planet owns whatever gets built next. Call it the most expensive science fiction ever sold, or the first time the map of the internet pointed up.
0:10

AI just hit a wall that no amount of money can move. The planet itself. There is not enough power, water, or land on Earth to build the data centers the AI race now demands. So the most valuable bet in artificial intelligence is no longer a chip company or a model. It is a rocket company. The plan is to leave. In January, SpaceX filed with the FCC to launch up to 1 million solar-powered data center satellites into orbit. In February it bought xAI, the maker of Grok, folding an entire frontier AI lab into a rocket company in the largest corporate merger ever recorded. On June 8 it unveiled the AI1, a compute satellite with a 70-meter wingspan, wider than a Boeing 747, powered by the sun, cooled by the vacuum of space, and wired to the ground through Starlink. Four days later it went public in the largest IPO in history, near 1.77 trillion dollars, touched 2.1 trillion on its first day, raised close to 86 billion, and made one man the first trillionaire alive. Now read the direction of that merger, because it is the whole story. A rocket company bought the AI lab. Not the reverse. For three years everyone assumed the constraint on AI was chips, or data, or talent. It is none of them anymore. It is energy and heat and dirt. The head of Anthropic said his company grew faster than the exponential, 80 times in a single year, and that is exactly why it ran out of compute. The answer was not to build more data centers in Virginia. It was to leave the atmosphere, where the sun never sets and a solar panel does five times the work. The moat in artificial intelligence is no longer the model. It is the launch. And the first rent is already being paid. A rival lab, Anthropic, is reported to be sending roughly 1.25 billion dollars a month to Musk for compute. Google near 920 million. If intelligence moves to orbit, the company that owns the only affordable road there becomes the landlord of the next layer of the internet, the way one bookstore became the landlord of the cloud. The merger is the proof of concept. The IPO is the war chest. Those monthly checks are the lease. Here is the part the price tag does not want you to read. Close to a trillion dollars of that valuation rests on orbital data centers that do not yet exist, and on a chip factory, Terafab, that SpaceX's own public filing calls a general framework with no binding deal, one that may not achieve commercial viability. Musk said it on camera. This is not a promise. The largest IPO ever written is priced on a future the filing itself cannot verify. The other side is just as real. Compute in orbit costs about four times what it costs on the ground today, and the curve may not cross for fifteen years. The machines that print the chips are backordered for years. Shedding heat in a vacuum at this scale has never been done. Musk's timelines have a long history of meaning later. And Bezos is racing the same orbit with a constellation of 51,600 satellites of his own. But strip it all away and the trade underneath is one sentence. Earth has run out of room for intelligence, and whoever owns the road off the planet owns whatever gets built next. Call it the most expensive science fiction ever sold, or the first time the map of the internet pointed up.

Shanaka Anslem Perera ⚡

54,458 次观看 • 2 个月前

Microsoft made 100B parameter models run on a single CPU. bitnet.cpp: The official inference framework for 1-bit LLMs. The math behind 1-bit LLMs is what makes them revolutionary. Traditional LLMs use 16-bit floating point weights. Every parameter is a number like 0.0023847 or -1.4729. When you run inference, you multiply these floats together. Billions of times. That's why you need GPUs, they're optimized for floating point matrix multiplication. BitNet b1.58 uses ternary weights: {-1, 0, 1}. That's not a simplification. That's a fundamental change in the math. When your weights are only -1, 0, or 1: → Multiply by 1 = keep the value → Multiply by -1 = flip the sign → Multiply by 0 = skip entirely Matrix multiplication becomes addition and subtraction. No floating point operations. No GPU required. This is why bitnet.cpp achieves: → 2.37x to 6.17x speedup on x86 CPUs → 1.37x to 5.07x speedup on ARM CPUs → 71.9% to 82.2% energy reduction on x86 → 55.4% to 70.0% energy reduction on ARM The speedups scale with model size. Larger models see bigger gains because there are more operations to simplify. A 100B parameter model running at human reading speed (5-7 tokens/second) on a single CPU. That's not optimization. That's a different paradigm. Why 1.58 bits? Because log₂(3) ≈ 1.58. Three possible values = 1.58 bits of information per weight. The key insight: These models aren't quantized after training. They're trained from scratch with ternary weights. The model learns to work within the constraint. No precision loss. No quality tradeoff.
0:13

Microsoft made 100B parameter models run on a single CPU. bitnet.cpp: The official inference framework for 1-bit LLMs. The math behind 1-bit LLMs is what makes them revolutionary. Traditional LLMs use 16-bit floating point weights. Every parameter is a number like 0.0023847 or -1.4729. When you run inference, you multiply these floats together. Billions of times. That's why you need GPUs, they're optimized for floating point matrix multiplication. BitNet b1.58 uses ternary weights: {-1, 0, 1}. That's not a simplification. That's a fundamental change in the math. When your weights are only -1, 0, or 1: → Multiply by 1 = keep the value → Multiply by -1 = flip the sign → Multiply by 0 = skip entirely Matrix multiplication becomes addition and subtraction. No floating point operations. No GPU required. This is why bitnet.cpp achieves: → 2.37x to 6.17x speedup on x86 CPUs → 1.37x to 5.07x speedup on ARM CPUs → 71.9% to 82.2% energy reduction on x86 → 55.4% to 70.0% energy reduction on ARM The speedups scale with model size. Larger models see bigger gains because there are more operations to simplify. A 100B parameter model running at human reading speed (5-7 tokens/second) on a single CPU. That's not optimization. That's a different paradigm. Why 1.58 bits? Because log₂(3) ≈ 1.58. Three possible values = 1.58 bits of information per weight. The key insight: These models aren't quantized after training. They're trained from scratch with ternary weights. The model learns to work within the constraint. No precision loss. No quality tradeoff.

Tech with Mak

23,036 次观看 • 4 个月前

hey if you have a 3060, or any GPU with 8GB or more sitting in a drawer right now, that thing can run 9 billion parameters of intelligence autonomously. and you don't know it yet. 2 hours ago i posted that 9B hit a ceiling. 2,699 lines across 11 files. blank screen. said the limit for autonomous multifile coding on 9 billion parameters is real. then i audited every file. found 11 bugs. exact file, exact line, exact fix. duplicate variable declarations killing the script loader. a canvas reference never connected to the DOM. enemies with no movement logic. particle systems called on the class instead of the instance. fed that list as a single prompt to the same Qwen 3.5 9B on the same RTX 3060 through Hermes Agent. it fixed all 11. surgically. patch level edits across 4 files. no rewrites. no hallucinated changes. game boots. enemies spawn, move, collide. background renders. particles fire. and here's what nobody is talking about. this is a 9 billion parameter model running a full agentic framework. Hermes Agent with 31 tools. file operations, terminal, browser, code execution. not a single tool call failed. the agent chain never broke. most people think you need 70B+ for reliable tool use. this is 9B on 12 gigs doing it clean. the model didn't fail. my prompting strategy did. the ceiling is not the parameter count. the ceiling is how you prompt it. this is not done. bullets don't fire yet. boss fights need wiring. but the screen that was black 2 hours ago now has a full game rendering in real time. iterating right now. anyone with a GPU from the last 5 years should be paying attention to what is happening right now.
0:12

hey if you have a 3060, or any GPU with 8GB or more sitting in a drawer right now, that thing can run 9 billion parameters of intelligence autonomously. and you don't know it yet. 2 hours ago i posted that 9B hit a ceiling. 2,699 lines across 11 files. blank screen. said the limit for autonomous multifile coding on 9 billion parameters is real. then i audited every file. found 11 bugs. exact file, exact line, exact fix. duplicate variable declarations killing the script loader. a canvas reference never connected to the DOM. enemies with no movement logic. particle systems called on the class instead of the instance. fed that list as a single prompt to the same Qwen 3.5 9B on the same RTX 3060 through Hermes Agent. it fixed all 11. surgically. patch level edits across 4 files. no rewrites. no hallucinated changes. game boots. enemies spawn, move, collide. background renders. particles fire. and here's what nobody is talking about. this is a 9 billion parameter model running a full agentic framework. Hermes Agent with 31 tools. file operations, terminal, browser, code execution. not a single tool call failed. the agent chain never broke. most people think you need 70B+ for reliable tool use. this is 9B on 12 gigs doing it clean. the model didn't fail. my prompting strategy did. the ceiling is not the parameter count. the ceiling is how you prompt it. this is not done. bullets don't fire yet. boss fights need wiring. but the screen that was black 2 hours ago now has a full game rendering in real time. iterating right now. anyone with a GPU from the last 5 years should be paying attention to what is happening right now.

Sudo su

684,336 次观看 • 5 个月前

Introducing fx, a tiny, open, native coding agent from Vercel Labs. Originally an internal tool, fx is a harness and CLI written in Zig, optimized for research and embedding in larger systems. Today, we're open sourcing it. fx is built on three principles: 1. Fast. A single native binary, no runtime to install. It cold starts in 10µs and does no unnecessary work or I/O before accepting input. fx is the answer to "how fast can a coding agent be?" 2. Light. The 6.3MiB binary uses single-digit megabytes of memory at baseline, made for instant installation and embedding in resource-constrained environments and agent sandboxes. 3. Open. Apache-2.0, model and provider agnostic, suitable for local and cloud inference. Its small core extends through skills, plugins, and MCP. Minimalism is an obsession throughout the entire harness: system prompt, tools, features, binary. The goal was to keep context usage and time to first token low, and make fx optimal for model benchmarking, sandboxing, evals, and gyms. You can use fx directly or embed it as infrastructure. The CLI feels more like a Unix shell than an IDE in the terminal: it preserves scroll history, produces minimal output, and uses complex TUI rendering very, very sparingly. Programmatically, 𝚏𝚡 𝚊𝚜𝚔 --𝚓𝚜𝚘𝚗 gives structured output, 𝚏𝚡 𝚊𝚌𝚙 connects to editors and other clients, and WebAssembly can even run the whole thing inside the browser (see: Privacy is a design constraint: no product telemetry, sessions and usage stay local, and no source code or prompts are shared with any endpoint other than inference. With local inference and auto-updates off, fx is fully hermetic. fx is experimental. Use at your own risk and expect frequent changes. Chat with us on X ( or file issues ( 𝚌𝚞𝚛𝚕 -𝚏𝚜𝚂𝙻 𝚏𝚡.𝚜𝚑/𝚜𝚎𝚝𝚞𝚙.𝚜𝚑 | 𝚋𝚊𝚜𝚑
0:11

Introducing fx, a tiny, open, native coding agent from Vercel Labs. Originally an internal tool, fx is a harness and CLI written in Zig, optimized for research and embedding in larger systems. Today, we're open sourcing it. fx is built on three principles: 1. Fast. A single native binary, no runtime to install. It cold starts in 10µs and does no unnecessary work or I/O before accepting input. fx is the answer to "how fast can a coding agent be?" 2. Light. The 6.3MiB binary uses single-digit megabytes of memory at baseline, made for instant installation and embedding in resource-constrained environments and agent sandboxes. 3. Open. Apache-2.0, model and provider agnostic, suitable for local and cloud inference. Its small core extends through skills, plugins, and MCP. Minimalism is an obsession throughout the entire harness: system prompt, tools, features, binary. The goal was to keep context usage and time to first token low, and make fx optimal for model benchmarking, sandboxing, evals, and gyms. You can use fx directly or embed it as infrastructure. The CLI feels more like a Unix shell than an IDE in the terminal: it preserves scroll history, produces minimal output, and uses complex TUI rendering very, very sparingly. Programmatically, 𝚏𝚡 𝚊𝚜𝚔 --𝚓𝚜𝚘𝚗 gives structured output, 𝚏𝚡 𝚊𝚌𝚙 connects to editors and other clients, and WebAssembly can even run the whole thing inside the browser (see: Privacy is a design constraint: no product telemetry, sessions and usage stay local, and no source code or prompts are shared with any endpoint other than inference. With local inference and auto-updates off, fx is fully hermetic. fx is experimental. Use at your own risk and expect frequent changes. Chat with us on X ( or file issues ( 𝚌𝚞𝚛𝚕 -𝚏𝚜𝚂𝙻 𝚏𝚡.𝚜𝚑/𝚜𝚎𝚝𝚞𝚙.𝚜𝚑 | 𝚋𝚊𝚜𝚑

Vercel Developers

945,463 次观看 • 10 天前

What Actually is Sei Network's “Giga” Upgrade? Sei Network’s (Sei) Giga upgrade is a major overhaul designed to make the network faster, more scalable and better suited for high-performance onchain trading. Put simply, Giga is rebuilding three critical parts of the blockchain: consensus, execution and storage. (1) The first track focuses on consensus, with upgrades such as Autobahn designed to improve how Sei validators agree on the state of the chain. (2) The Ares upgrade targets execution, the part of the blockchain responsible for actually processing transactions. (3) Eidos focuses on storage, which is becoming increasingly important as blockchain throughput rises. Why does storage matter? Every transaction a blockchain processes has to be recorded. If the database cannot write data as quickly as the network executes transactions, higher throughput eventually becomes meaningless. Eidos is designed to solve that bottleneck. (4) Sei plans to replace the traditional Merkle-tree structure used for EVM state with FlatKV, a flat key-value database where updating one piece of state requires essentially one write. A lattice hash, or LtHash, is then used to maintain a verifiable fingerprint of the entire state without repeatedly recalculating an entire hash path. (5) Eidos also separates live EVM state from other blockchain data. This means transactions accessing current state no longer have to compete with historical data for the same database resources. (6) Sei is also introducing LittDB-backed storage for blocks and receipts. These records are written once but queried repeatedly, making them a different workload from constantly changing blockchain state. Older historical data will eventually move away from active nodes into archival storage, allowing nodes to focus their resources on the data needed for real-time operations. The interesting part is how Sei plans to deploy all of this. Instead of shutting down the network and migrating the entire database at once, Eidos is designed to migrate storage while Sei continues producing blocks. The old and new systems can run side by side during the transition, with data moved in batches and integrity checks performed throughout the process. The first phase arrived on Sei mainnet with the v6.6 release in August 2026, beginning the separation of EVM state and introducing improvements to the pruning process. The broader Eidos architecture, including FlatKV, LtHash, the new receipt store and off-node archival storage, is expected to arrive through subsequent releases. Sei’s ultimate Giga target is 200,000 transactions per second. But reaching that kind of execution speed requires more than a faster transaction engine. The blockchain also needs a storage system capable of keeping up. That is essentially what Eidos is trying to build. Giga is not just about making Sei execute transactions faster. It is about rebuilding the infrastructure underneath that speed so the network can actually sustain it.
0:15

What Actually is Sei Network's “Giga” Upgrade? Sei Network’s (Sei) Giga upgrade is a major overhaul designed to make the network faster, more scalable and better suited for high-performance onchain trading. Put simply, Giga is rebuilding three critical parts of the blockchain: consensus, execution and storage. (1) The first track focuses on consensus, with upgrades such as Autobahn designed to improve how Sei validators agree on the state of the chain. (2) The Ares upgrade targets execution, the part of the blockchain responsible for actually processing transactions. (3) Eidos focuses on storage, which is becoming increasingly important as blockchain throughput rises. Why does storage matter? Every transaction a blockchain processes has to be recorded. If the database cannot write data as quickly as the network executes transactions, higher throughput eventually becomes meaningless. Eidos is designed to solve that bottleneck. (4) Sei plans to replace the traditional Merkle-tree structure used for EVM state with FlatKV, a flat key-value database where updating one piece of state requires essentially one write. A lattice hash, or LtHash, is then used to maintain a verifiable fingerprint of the entire state without repeatedly recalculating an entire hash path. (5) Eidos also separates live EVM state from other blockchain data. This means transactions accessing current state no longer have to compete with historical data for the same database resources. (6) Sei is also introducing LittDB-backed storage for blocks and receipts. These records are written once but queried repeatedly, making them a different workload from constantly changing blockchain state. Older historical data will eventually move away from active nodes into archival storage, allowing nodes to focus their resources on the data needed for real-time operations. The interesting part is how Sei plans to deploy all of this. Instead of shutting down the network and migrating the entire database at once, Eidos is designed to migrate storage while Sei continues producing blocks. The old and new systems can run side by side during the transition, with data moved in batches and integrity checks performed throughout the process. The first phase arrived on Sei mainnet with the v6.6 release in August 2026, beginning the separation of EVM state and introducing improvements to the pruning process. The broader Eidos architecture, including FlatKV, LtHash, the new receipt store and off-node archival storage, is expected to arrive through subsequent releases. Sei’s ultimate Giga target is 200,000 transactions per second. But reaching that kind of execution speed requires more than a faster transaction engine. The blockchain also needs a storage system capable of keeping up. That is essentially what Eidos is trying to build. Giga is not just about making Sei execute transactions faster. It is about rebuilding the infrastructure underneath that speed so the network can actually sustain it.

BSCN

27,310 次观看 • 10 天前

Introducing Pods Hyperspace Pods lets a small group of people - a family, a startup, a few friends, to pool their laptops and desktops into one AI cluster. Everyone installs the CLI, someone creates a pod, shares an invite link, and the machines form a mesh. Models like Qwen 3.5 32B or GLM-5 Turbo that need more memory than any single laptop has get automatically sharded across the group's devices - layers split proportionally, inference pipelined through the ring. From the outside it looks like one OpenAI-compatible API endpoint with a pk_* key that drops straight into your AI tools and products. No configuration beyond pasting the key and changing the base URL. A team of five paying for cloud AI burns $500–2,000 a month on API calls. The same team's existing machines can serve Qwen 3.5 (competitive on SWE-bench) and GLM-5 Turbo (#1 on BrowseComp for tool-calling and web research) for free - the hardware is already on their desks. When a query genuinely needs a frontier model nobody has locally, the pod falls back to cloud at wholesale rates from a shared treasury. But for the daily work - code reviews, refactors, research, drafting - local models handle it and nobody gets billed. And when it is idle, you can rent out your pod on the compute marketplace, with fine-grained permissions for access management. There's no central server involved in inference. Prompts go from your machine to your pod members' machines and back: all of this enabled by the fully peer-to-peer Hyperspace network. Pod state - who's a member, which API keys are valid, how much treasury is left - is replicated across members with consensus, so the whole thing works on a local network. Members behind home routers don't need port forwarding either. The practical setup for most pods is three models covering different jobs: Qwen 3.5 32B for code and reasoning, GLM-5 Turbo for browsing and research, Gemma 4 for fast lightweight tasks. All running on hardware you already own. Pods ship today in Hyperspace v5.19. Model sharding, API keys, treasury, and Raft coordinator are all live. What Makes This Different - No middleman. Your prompts travel from your IDE to your pod members' hardware and back. There is no server in between reading your data. - No vendor lock-in. Pod membership, API keys, and treasury are replicated across your own machines using Raft consensus. If the internet goes down, your local network keeps working. There is no database in someone else's cloud that your pod depends on. - Automatic sharding. You don't configure layer ranges or calculate VRAM budgets. Tell the pod which model you want. It figures out how to split it across whatever hardware is online. - Real NAT traversal. Your friend behind a home router with a dynamic IP? Works. No VPN, no Tailscale, no port forwarding. The nodes handle it. - Free when local. This is the part that matters most. Cloud AI bills scale with usage. Pod inference on local hardware scales with nothing. The marginal cost of your 10,000th prompt is the electricity your laptop was already using. Coming soon: - Pod federation: pods form alliances with other pods. - Marketplace: pods with spare capacity can sell inference to other pods.
0:19

Introducing Pods Hyperspace Pods lets a small group of people - a family, a startup, a few friends, to pool their laptops and desktops into one AI cluster. Everyone installs the CLI, someone creates a pod, shares an invite link, and the machines form a mesh. Models like Qwen 3.5 32B or GLM-5 Turbo that need more memory than any single laptop has get automatically sharded across the group's devices - layers split proportionally, inference pipelined through the ring. From the outside it looks like one OpenAI-compatible API endpoint with a pk_* key that drops straight into your AI tools and products. No configuration beyond pasting the key and changing the base URL. A team of five paying for cloud AI burns $500–2,000 a month on API calls. The same team's existing machines can serve Qwen 3.5 (competitive on SWE-bench) and GLM-5 Turbo (#1 on BrowseComp for tool-calling and web research) for free - the hardware is already on their desks. When a query genuinely needs a frontier model nobody has locally, the pod falls back to cloud at wholesale rates from a shared treasury. But for the daily work - code reviews, refactors, research, drafting - local models handle it and nobody gets billed. And when it is idle, you can rent out your pod on the compute marketplace, with fine-grained permissions for access management. There's no central server involved in inference. Prompts go from your machine to your pod members' machines and back: all of this enabled by the fully peer-to-peer Hyperspace network. Pod state - who's a member, which API keys are valid, how much treasury is left - is replicated across members with consensus, so the whole thing works on a local network. Members behind home routers don't need port forwarding either. The practical setup for most pods is three models covering different jobs: Qwen 3.5 32B for code and reasoning, GLM-5 Turbo for browsing and research, Gemma 4 for fast lightweight tasks. All running on hardware you already own. Pods ship today in Hyperspace v5.19. Model sharding, API keys, treasury, and Raft coordinator are all live. What Makes This Different - No middleman. Your prompts travel from your IDE to your pod members' hardware and back. There is no server in between reading your data. - No vendor lock-in. Pod membership, API keys, and treasury are replicated across your own machines using Raft consensus. If the internet goes down, your local network keeps working. There is no database in someone else's cloud that your pod depends on. - Automatic sharding. You don't configure layer ranges or calculate VRAM budgets. Tell the pod which model you want. It figures out how to split it across whatever hardware is online. - Real NAT traversal. Your friend behind a home router with a dynamic IP? Works. No VPN, no Tailscale, no port forwarding. The nodes handle it. - Free when local. This is the part that matters most. Cloud AI bills scale with usage. Pod inference on local hardware scales with nothing. The marginal cost of your 10,000th prompt is the electricity your laptop was already using. Coming soon: - Pod federation: pods form alliances with other pods. - Marketplace: pods with spare capacity can sell inference to other pods.

Varun

309,424 次观看 • 4 个月前