Loading video...

Video Failed to Load

Go Home

Running AI directly on $5 microcontroller hardware keeps getting easier. ESP32-AI adds a 28.9M LLM to your microcontroller. It's very impressive for the chip size. It ​gives you lightweight AI capabilities right on ESP32 microcontrollers. ​Key highlights: • On-device edge inference without relying on cloud APIs • Ultra-low power...

82,409 views • 2 days ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading speed. Here's how it works: Every other LLM stores weights in 32-bit or 16-bit floats. BitNet uses 1.58 bits. Weights are ternary just -1, 0, or +1. That's it. No floats. No expensive matrix math. Pure integer operations your CPU was already built for. The result: - 100B model runs on a single CPU at 5-7 tokens/second - 2.37x to 6.17x faster than llama.cpp on x86 - 82% lower energy consumption on x86 CPUs - 1.37x to 5.07x speedup on ARM (your MacBook) - Memory drops by 16-32x vs full-precision models The wildest part: Accuracy barely moves. BitNet b1.58 2B4T their flagship model was trained on 4 trillion tokens and benchmarks competitively against full-precision models of the same size. The quantization isn't destroying quality. It's just removing the bloat. What this actually means: - Run AI completely offline. Your data never leaves your machine - Deploy LLMs on phones, IoT devices, edge hardware - No more cloud API bills for inference - AI in regions with no reliable internet The model supports ARM and x86. Works on your MacBook, your Linux box, your Windows machine. 27.4K GitHub stars. 2.2K forks. Built by Microsoft Research. 100% Open Source. MIT License.

Guri Singh

2,180,357 views • 4 months ago

Introducing Pods Hyperspace Pods lets a small group of people - a family, a startup, a few friends, to pool their laptops and desktops into one AI cluster. Everyone installs the CLI, someone creates a pod, shares an invite link, and the machines form a mesh. Models like Qwen 3.5 32B or GLM-5 Turbo that need more memory than any single laptop has get automatically sharded across the group's devices - layers split proportionally, inference pipelined through the ring. From the outside it looks like one OpenAI-compatible API endpoint with a pk_* key that drops straight into your AI tools and products. No configuration beyond pasting the key and changing the base URL. A team of five paying for cloud AI burns $500–2,000 a month on API calls. The same team's existing machines can serve Qwen 3.5 (competitive on SWE-bench) and GLM-5 Turbo (#1 on BrowseComp for tool-calling and web research) for free - the hardware is already on their desks. When a query genuinely needs a frontier model nobody has locally, the pod falls back to cloud at wholesale rates from a shared treasury. But for the daily work - code reviews, refactors, research, drafting - local models handle it and nobody gets billed. And when it is idle, you can rent out your pod on the compute marketplace, with fine-grained permissions for access management. There's no central server involved in inference. Prompts go from your machine to your pod members' machines and back: all of this enabled by the fully peer-to-peer Hyperspace network. Pod state - who's a member, which API keys are valid, how much treasury is left - is replicated across members with consensus, so the whole thing works on a local network. Members behind home routers don't need port forwarding either. The practical setup for most pods is three models covering different jobs: Qwen 3.5 32B for code and reasoning, GLM-5 Turbo for browsing and research, Gemma 4 for fast lightweight tasks. All running on hardware you already own. Pods ship today in Hyperspace v5.19. Model sharding, API keys, treasury, and Raft coordinator are all live. What Makes This Different - No middleman. Your prompts travel from your IDE to your pod members' hardware and back. There is no server in between reading your data. - No vendor lock-in. Pod membership, API keys, and treasury are replicated across your own machines using Raft consensus. If the internet goes down, your local network keeps working. There is no database in someone else's cloud that your pod depends on. - Automatic sharding. You don't configure layer ranges or calculate VRAM budgets. Tell the pod which model you want. It figures out how to split it across whatever hardware is online. - Real NAT traversal. Your friend behind a home router with a dynamic IP? Works. No VPN, no Tailscale, no port forwarding. The nodes handle it. - Free when local. This is the part that matters most. Cloud AI bills scale with usage. Pod inference on local hardware scales with nothing. The marginal cost of your 10,000th prompt is the electricity your laptop was already using. Coming soon: - Pod federation: pods form alliances with other pods. - Marketplace: pods with spare capacity can sell inference to other pods.

Varun

308,519 views • 3 months ago

Elon Musk just identified the next crisis in AI. It’s not a shortage. It’s an unusable surplus. Musk: “By the end of this year, chip production will outpace the ability to turn chips on.” For three years the world was starved for silicon. Every lab, every government, every company racing to secure the chips that determine who wins the AI era. That bottleneck is ending. A new one is replacing it. Musk: “The chips are going to be piling up and not be able to be turned on.” Billions of dollars of the most advanced AI hardware ever built. Sitting dark. Not because the chips don’t work. Because there isn’t enough electricity to run them. You can’t print a power plant the way you print a chip. The fabrication plants scaled. The grid didn’t. And now the most valuable hardware in history is about to hit a wall that no amount of capital can instantly solve. Compute is about to become abundant. Electricity is about to become the most valuable commodity on earth. Three years obsessing over silicon yields. Physics doesn’t care about your chip architecture if your data center can’t pull enough megawatts. The war isn’t about who can manufacture the most silicon anymore. It’s about who has the raw power to plug it in. Whoever solves energy first doesn’t just win. They own the infrastructure everyone else needs to compete. The losers stack useless chips in warehouses waiting for power that never arrives. We built a trillion dollar engine and forgot the fuel. That’s the AI race right now.

Dustin

705,220 views • 5 months ago