Загрузка видео...

Не удалось загрузить видео

На главную

Cloud inference will never be fast enough for robotics? Here's MolmoAct2 VLA model running over cloud on an SO-101 arm. Smooth motions despite the DC being 1000km away.

11,309 просмотров • 2 месяцев назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading speed. Here's how it works: Every other LLM stores weights in 32-bit or 16-bit floats. BitNet uses 1.58 bits. Weights are ternary just -1, 0, or +1. That's it. No floats. No expensive matrix math. Pure integer operations your CPU was already built for. The result: - 100B model runs on a single CPU at 5-7 tokens/second - 2.37x to 6.17x faster than llama.cpp on x86 - 82% lower energy consumption on x86 CPUs - 1.37x to 5.07x speedup on ARM (your MacBook) - Memory drops by 16-32x vs full-precision models The wildest part: Accuracy barely moves. BitNet b1.58 2B4T their flagship model was trained on 4 trillion tokens and benchmarks competitively against full-precision models of the same size. The quantization isn't destroying quality. It's just removing the bloat. What this actually means: - Run AI completely offline. Your data never leaves your machine - Deploy LLMs on phones, IoT devices, edge hardware - No more cloud API bills for inference - AI in regions with no reliable internet The model supports ARM and x86. Works on your MacBook, your Linux box, your Windows machine. 27.4K GitHub stars. 2.2K forks. Built by Microsoft Research. 100% Open Source. MIT License.

Guri Singh

2,180,357 просмотров • 7 месяцев назад

i spent $26,600 on cloud GPU rentals over 14 months before i found a NVIDIA DGX Spark at $2,999 (founder's edition) or $3,999 (shipping price) it paid for itself in 6 weeks i run 200B parameter models locally now and my old cloud provider keeps sending me loyalty discount emails the math on that $26,600 is embarrassing to type out loud $1,900/month for 14 months, H100 instances on a specialist cloud provider, because anything bigger than a 70B model simply would not fit anywhere else i paid the invoices like they were a utility bill and told myself it was just the cost of doing serious AI work it took me over a year to find out it wasn't 14 months, broken down: → months 1-4: $1,400-1,600/month - felt like manageable infrastructure overhead → months 5-9: crept to $1,900-2,100 as i started running DeepSeek-class experiments, costs tracking directly with model size → months 10-12: one agent loop ran for 36 hours against a 130B model while i slept, that month hit $2,400 → month 13: ran the cumulative total for the first time, saw $23,800, felt physically sick → month 14: another $2,800 month while i waited for the hardware to ship the box is the NVIDIA DGX Spark - roughly the footprint of a large mac mini, powered by a GB10 Grace Blackwell chip with 128GB of unified LPDDR5X memory that unified memory is the whole thing an RTX 4090 has 24GB of VRAM, which means a 70B model in full BF16 precision physically does not fit, you're quantizing down or you're renting cloud, those are your options this box loads a 200B parameter model quantized and serves it through vLLM over localhost, same API interface the cloud endpoint used the migration took one line of code - i changed the base URL from the provider's endpoint to 127.0.0.1:8000 and everything just worked electricity to run continuous 200B inference locally comes out to about $12/month the payback arithmetic is almost too clean: $2,999 hardware cost against $1,900/month saved, the box paid for itself before i'd owned it two months what i didn't account for was how completely the cost model changes your behavior when there's no hourly meter running, you greenlight experiments you'd never approve on cloud - agent loops that churn for hours, running 10,000 documents through a reasoning pass at 3am, speculative fine-tuning jobs you'd normally skip because the cost felt unjustifiable i ran more experiments in the first 30 days after the box arrived than in the four months before it the loyalty discount email landed about 8 weeks after i cancelled the cloud subscription 15% off my next three months, valued customer, we'd love to have you back i didn't reply the box was already running

Argona

22,467 просмотров • 4 месяцев назад

Proud to announce the in-depth collaboration between Kingnet and Alibaba Cloud in AI Gaming. Alibaba Cloud provides world-leading cloud computing, big data, and AI services, with disclosed revenue exceeding $15 billion in 2024, which is one of the most renowned global server providers. When two superpowers collide, the game changes. 🌊AI Gaming R&D By integrating Qwen 's LLM and Alibaba Cloud 's PAI platform (including PAI-iTAG, PAI-Designer, PAI-DSW, PAI-DLC, and PAI-EAS), Kingnet has emerged as one of the gaming industry's pioneers in AIGC-powered content generation and AI rendering. Together, we are accelerating the realization of no-code game development. 🌊GPU Computing Resources Alibaba Cloud delivers GPU-accelerated elastic computing services with exceptional processing power, supporting diverse workloads including deep learning, scientific computing, graphics visualization, and video processing - providing robust GPU computing capabilities for KingnetAI's demanding requirements. 🌊Cloud Service Optimization Cloud server deployment has become the mainstream choice for small and mid-sized game studios in global operations. Leveraging Alibaba Cloud server advantages, we will develop and deploy more cloud-native games to meet user demands. The disruptive innovation we're bringing to the industry: 🔸Minute-scale game asset production replaces traditional week/month-long cycles 🔸Single-digit dollar development costs VS traditional four-figure entry thresholds 🔸AI-powered NPCs with behavioral engines deliver dynamic player interactions, breaking static story constraints, etc. 🔜Kingnet AI V2 is approaching launch. The Agent system and game generation engine will be officially deployed across 3 chains: 🔹Leveraging Solana high throughput and low gas fee , Solana has consistently been a developer favorite, latest product will be deployed on Solana - with users paying $SOL for on-demand asset creation fees. 🔹Another key partner is BNB Chain ,We are actively participating in both the #BNBAIHack and the latest MVB 10. Powered by BNB Chain long-standing support for AI innovation. Kingnet V2 and NFT drop will be deployed on BNB Chain, providing developers and the community with comprehensive game-generation tools and support. 🔹As an early strategic partner of Kingnet, TON 💎 @TONEastAsia was one of the earliest chain to connect Web2 and Web3, Kingnet V2 will be deployed on TON, providing TON game developers with low-cost, high-efficiency asset generation, and supporting users to use $TON as an asset generation cost. The Future of AI Gaming is coming.

Kingnet AI

149,965 просмотров • 1 год назад