Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB Most RAG stacks treat the embedding model as a commodity — pick one, index, move on. Nemotron 3 Embed is NVIDIA's argument that the retrieval layer is where agent cost actually gets...

Marktechpost's profile picture

Marktechpost AI

11,541 subscribers

32,352 Aufrufe • vor 23 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.
1:24

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.

kwindla

274,590 Aufrufe • vor 7 Monaten

Most speculative decoding still drafts tokens one at a time. That's not parallel generation — it just hides the serial loop behind a smaller model. UC San Diego's z-lab just drew a clear line between the two. They released DFlash — a lightweight block diffusion model that drafts a whole block of tokens in a single forward pass, then lets the target model verify the block in parallel. Up to 15× higher throughput for gpt-oss-120b on NVIDIA Blackwell. No token-by-token drafting anywhere in the speculative path. Here's what's actually interesting: → The drafter is conditioned on the target model's own hidden features, injected into the Key/Value cache of every draft layer — so acceptance length scales with draft depth instead of diluting away → A 5-layer drafter replaces the 7B diffusion drafters that capped earlier methods near 3–4× → MATH-500 speedup: 6.08× vs. 1.81× for EAGLE-3 (4.86× average vs. 1.76×, Qwen3-8B, greedy) → Up to 15× higher throughput for gpt-oss-120b on NVIDIA Blackwell — at the same interactivity target → Lossless: the target still verifies every token, so output quality is preserved Full analysis: Paper: NVIDIA's metrics: Project: Model weights: Repo: NVIDIA AI NVIDIA AIDev
0:20

Most speculative decoding still drafts tokens one at a time. That's not parallel generation — it just hides the serial loop behind a smaller model. UC San Diego's z-lab just drew a clear line between the two. They released DFlash — a lightweight block diffusion model that drafts a whole block of tokens in a single forward pass, then lets the target model verify the block in parallel. Up to 15× higher throughput for gpt-oss-120b on NVIDIA Blackwell. No token-by-token drafting anywhere in the speculative path. Here's what's actually interesting: → The drafter is conditioned on the target model's own hidden features, injected into the Key/Value cache of every draft layer — so acceptance length scales with draft depth instead of diluting away → A 5-layer drafter replaces the 7B diffusion drafters that capped earlier methods near 3–4× → MATH-500 speedup: 6.08× vs. 1.81× for EAGLE-3 (4.86× average vs. 1.76×, Qwen3-8B, greedy) → Up to 15× higher throughput for gpt-oss-120b on NVIDIA Blackwell — at the same interactivity target → Lossless: the target still verifies every token, so output quality is preserved Full analysis: Paper: NVIDIA's metrics: Project: Model weights: Repo: NVIDIA AI NVIDIA AIDev

Marktechpost AI

23,328 Aufrufe • vor 1 Monat

Inside Nemotron and NVIDIA's AI lab: my conversation with Bryan Catanzaro (Bryan Catanzaro). NVIDIA is a chip company. So why does it put hundreds of researchers on building AI models - and then give them away for free? We go deep into the Nemotron models, what it takes to build a top AI lab, and the future of frontier AI. 01:33 - Is open source AI catching the frontier? 05:29 - Do closed labs blocking distillation slow open source down? 07:42 - Is the US falling behind China? 10:30 - Why companies actually choose open models 12:39 - A "crazy" 2008 bet: machine learning on GPUs 15:33 - Working with Andrew Ng and Dario Amodei at Baidu 17:41 - Coming back to NVIDIA: DLSS and the birth of Megatron 21:55 - The real reason NVIDIA builds its own models 24:28 - Is Moore's Law really dead? 33:37 - The Nemotron family: Nano, Super, Ultra 35:09 - Built for agents: why NVIDIA bets on speed 36:02 - How you train a 550B model in 4 bits 39:25 - Hybrid Mamba-Transformer, explained simply 42:31 - Mixture of experts, and why NVIDIA built NVL72 around it 47:26 - Why a 1-million-token context window matters 49:26 - Multi-token prediction: how the model predicts 5 tokens at once 52:47 - Multi-teacher distillation: teaching one model from many 58:01 - Where reinforcement learning goes next 01:00:16 - Inside NVIDIA's research org: "the mission is the boss" 01:04:03 - How NVIDIA decides who gets the GPUs 01:10:53 - Why NVIDIA still feels entrepreneurial after 33 years 01:12:58 - Why Bryan doesn't believe in the singularity 01:17:50 - The AI backlash 01:19:18 - The controversial case: open AI is safer than closed
1:22:57

Inside Nemotron and NVIDIA's AI lab: my conversation with Bryan Catanzaro (Bryan Catanzaro). NVIDIA is a chip company. So why does it put hundreds of researchers on building AI models - and then give them away for free? We go deep into the Nemotron models, what it takes to build a top AI lab, and the future of frontier AI. 01:33 - Is open source AI catching the frontier? 05:29 - Do closed labs blocking distillation slow open source down? 07:42 - Is the US falling behind China? 10:30 - Why companies actually choose open models 12:39 - A "crazy" 2008 bet: machine learning on GPUs 15:33 - Working with Andrew Ng and Dario Amodei at Baidu 17:41 - Coming back to NVIDIA: DLSS and the birth of Megatron 21:55 - The real reason NVIDIA builds its own models 24:28 - Is Moore's Law really dead? 33:37 - The Nemotron family: Nano, Super, Ultra 35:09 - Built for agents: why NVIDIA bets on speed 36:02 - How you train a 550B model in 4 bits 39:25 - Hybrid Mamba-Transformer, explained simply 42:31 - Mixture of experts, and why NVIDIA built NVL72 around it 47:26 - Why a 1-million-token context window matters 49:26 - Multi-token prediction: how the model predicts 5 tokens at once 52:47 - Multi-teacher distillation: teaching one model from many 58:01 - Where reinforcement learning goes next 01:00:16 - Inside NVIDIA's research org: "the mission is the boss" 01:04:03 - How NVIDIA decides who gets the GPUs 01:10:53 - Why NVIDIA still feels entrepreneurial after 33 years 01:12:58 - Why Bryan doesn't believe in the singularity 01:17:50 - The AI backlash 01:19:18 - The controversial case: open AI is safer than closed

Matt Turck

56,749 Aufrufe • vor 1 Monat

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶
5:14

Google just proved that bigger isn't always better. Their 308M parameter model is outperforming models 2x its size. Google just released 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝗚𝗲𝗺𝗺𝗮, and it's proving that lightweight embedding models can punch way above their weight class. At just 308M parameters (578MB), it's the new state-of-the-art for models under 500M parameters across MTEB multilingual, English, and code benchmarks. But the really impressive part is that it ranks 8th overall on MTEB(Multilingual, v2) - that's 𝟭𝟳 𝗽𝗹𝗮𝗰𝗲𝘀 above the second-best sub-500M model, and it's delivering performance 𝗰𝗼𝗺𝗽𝗮𝗿𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝗼𝗱𝗲𝗹𝘀 𝗻𝗲𝗮𝗿𝗹𝘆 𝗱𝗼𝘂𝗯𝗹𝗲 𝗶𝘁𝘀 𝘀𝗶𝘇𝗲. There are three key parts of their training recipe that sets it apart: 𝟭. 𝗘𝗻𝗰𝗼𝗱𝗲𝗿-𝗗𝗲𝗰𝗼𝗱𝗲𝗿 𝗜𝗻𝗶𝘁𝗶𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Instead of starting from a decoder-only Gemma 3 model, they first adapted it to encoder-decoder, then used just the encoder. By basing EmbeddingGemma off an LLM that already has world and language understanding, it gives it a stronger starting point. 𝟮. 𝗧𝗵𝗿𝗲𝗲-𝗟𝗼𝘀𝘀 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 They combine three different loss functions, instead of just having one: • Contrastive loss (NCE) with in-batch negatives and hardness weighting • Spread-out regularization to ensure embeddings utilize the full space (for quantization and ANN retrieval) • Embedding matching distillation from Gemini Embedding - not just learning from relevance scores, but directly aligning the embedding space with the teacher model 𝟯. 𝗠𝗼𝗱𝗲𝗹 𝗦𝗼𝘂𝗽𝗶𝗻𝗴 Rather than just averaging checkpoints from the same training run, they use optimization techniques to find multiple specialized training mixtures. Each mixture creates an "expert" model in different domains, and averaging all their parameters creates a final model that's actually better than individual models. Extras: • Matryoshka embeddings supporting 768, 512, 256, and 128 dimensions • Quantization-aware training - maintains quality even at int4 precision • 100+ languages from Gemma 3 pretraining • Exceptional performance on low-resource languages (check their XTREME-UP results) Is it the absolute best embedding model? No - Gemini Embedding still leads overall. But that's not really the point. EmbeddingGemma proves you can achieve state-of-the-art performance in a small package that's actually deployable on-device, in low-latency applications, and in resource-constrained environments. This makes good embeddings accessible for use cases that I'm seeing more and more: offline applications, privacy-sensitive deployments, and high-throughput scenarios where inference cost actually matters. Full paper: Shoutout to the EmbeddingGemma team at Google DeepMind for this awesome open source work 💙 and to Daniel Williams for helping me with this video! 🫶

Victoria Slocum

21,610 Aufrufe • vor 8 Monaten

Traditional chunking: cheap but dumb. ColBERT: smart but expensive. 𝗟𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴: the solution we've been waiting for. Here’s a quick evolution of chunking strategies: → 𝗧𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the basics we all started with) • Token Chunking - split by token count • Sentence Chunking - split by sentence boundaries • Document-Based Chunking - split by sections/paragraphs → 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (when things got sophisticated) • Semantic Chunking - split by meaning • LLM-Based Chunking - let the model decide But each chunking method separates text at defined points, meaning context is lost within the document from one chunk to the next. → 𝗘𝗻𝘁𝗲𝗿 𝗟𝗮𝘁𝗲 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the game changer) Traditional approach: Chunk first → Embed each chunk separately Late chunking approach: Embed the entire document → Then chunk with context preserved 𝗪𝗵𝘆 𝗰𝗵𝗼𝗼𝘀𝗲 𝗹𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴? When you chunk first, each piece loses its contextual relationship to the rest of the document. It's like reading a book by randomly picking paragraphs - you miss the flow. With late chunking, every chunk maintains awareness of its neighbors because the embedding happens at the document level first. Mean pooling is done on segments AFTER the full context is embedded. Jina AI tested and saw significant improvements in retrieval quality - chunks that were previously disconnected now maintain their semantic relationships. As documents get longer and context windows expand, late chunking might just become the new standard for high-quality retrieval systems. 𝗪𝗵𝗮𝘁 𝗱𝗼 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗺𝗮𝗸𝗲 𝘁𝗵𝗶𝘀 𝘄𝗼𝗿𝗸? No modifications to your retrieval pipeline are needed. 1. Long context embedding models (8192+ tokens) 2. Chunking logic that tracks token spans 3. Less than 30 lines of code to implement All you need is to switch the order at which you chunk and embed. Embed FIRST, then chunk, not the other way around. Dive deeper into late chunking:
1:19

Traditional chunking: cheap but dumb. ColBERT: smart but expensive. 𝗟𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴: the solution we've been waiting for. Here’s a quick evolution of chunking strategies: → 𝗧𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the basics we all started with) • Token Chunking - split by token count • Sentence Chunking - split by sentence boundaries • Document-Based Chunking - split by sections/paragraphs → 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (when things got sophisticated) • Semantic Chunking - split by meaning • LLM-Based Chunking - let the model decide But each chunking method separates text at defined points, meaning context is lost within the document from one chunk to the next. → 𝗘𝗻𝘁𝗲𝗿 𝗟𝗮𝘁𝗲 𝗖𝗵𝘂𝗻𝗸𝗶𝗻𝗴 (the game changer) Traditional approach: Chunk first → Embed each chunk separately Late chunking approach: Embed the entire document → Then chunk with context preserved 𝗪𝗵𝘆 𝗰𝗵𝗼𝗼𝘀𝗲 𝗹𝗮𝘁𝗲 𝗰𝗵𝘂𝗻𝗸𝗶𝗻𝗴? When you chunk first, each piece loses its contextual relationship to the rest of the document. It's like reading a book by randomly picking paragraphs - you miss the flow. With late chunking, every chunk maintains awareness of its neighbors because the embedding happens at the document level first. Mean pooling is done on segments AFTER the full context is embedded. Jina AI tested and saw significant improvements in retrieval quality - chunks that were previously disconnected now maintain their semantic relationships. As documents get longer and context windows expand, late chunking might just become the new standard for high-quality retrieval systems. 𝗪𝗵𝗮𝘁 𝗱𝗼 𝘆𝗼𝘂 𝗻𝗲𝗲𝗱 𝘁𝗼 𝗺𝗮𝗸𝗲 𝘁𝗵𝗶𝘀 𝘄𝗼𝗿𝗸? No modifications to your retrieval pipeline are needed. 1. Long context embedding models (8192+ tokens) 2. Chunking logic that tracks token spans 3. Less than 30 lines of code to implement All you need is to switch the order at which you chunk and embed. Embed FIRST, then chunk, not the other way around. Dive deeper into late chunking:

Femke Plantinga

125,371 Aufrufe • vor 1 Jahr

a team of researchers just proved you don't need a bigger model, you need a smarter plan researchers from Tsinghua and South China University of Technology built a framework called Atomic Task Graph. it turned 7B-8B open-source models into GPT-4 competitors on complex agent benchmarks, beating it on two out of three. no fine-tuning. no extra training. zero parameter updates. current AI agents plan in a straight line. step 1, step 2, step 3. when step 4 fails, the whole chain breaks. and the longer the chain gets, the more the model hallucinates because it's reasoning over a ballooning text history. here's how it works. 1. instead of a linear chain, ATG breaks any complex task into a directed graph where subtask inputs and outputs are explicitly mapped 2. it recursively decomposes each subtask until every node is one atomic tool call 3. independent branches run in parallel instead of waiting in line 4. before anything executes, a lightweight "thought experiment" simulates the plan internally to catch bad dependencies and missing steps early 5. when something breaks at runtime, ATG traces the failure to the exact subgraph that caused it and repairs only that piece. validated work stays frozen. the old way meant a failure at step 5 forced a full replan from scratch. hallucinated actions piled up the longer the task ran. ReAct hit a 43% hallucination rate on household tasks. ATG on an 8B Llama model scored 63.65 on ALFWorld. GPT-4 with ReAct scored 41.24 on the same benchmark. hallucinated actions dropped to 12%. those numbers happened because someone stopped throwing compute at the problem and started thinking about how work gets organized. that's the part that gets me. the industry is spending billions on scale. this team spent time on architecture. and the architecture won.
0:23

a team of researchers just proved you don't need a bigger model, you need a smarter plan researchers from Tsinghua and South China University of Technology built a framework called Atomic Task Graph. it turned 7B-8B open-source models into GPT-4 competitors on complex agent benchmarks, beating it on two out of three. no fine-tuning. no extra training. zero parameter updates. current AI agents plan in a straight line. step 1, step 2, step 3. when step 4 fails, the whole chain breaks. and the longer the chain gets, the more the model hallucinates because it's reasoning over a ballooning text history. here's how it works. 1. instead of a linear chain, ATG breaks any complex task into a directed graph where subtask inputs and outputs are explicitly mapped 2. it recursively decomposes each subtask until every node is one atomic tool call 3. independent branches run in parallel instead of waiting in line 4. before anything executes, a lightweight "thought experiment" simulates the plan internally to catch bad dependencies and missing steps early 5. when something breaks at runtime, ATG traces the failure to the exact subgraph that caused it and repairs only that piece. validated work stays frozen. the old way meant a failure at step 5 forced a full replan from scratch. hallucinated actions piled up the longer the task ran. ReAct hit a 43% hallucination rate on household tasks. ATG on an 8B Llama model scored 63.65 on ALFWorld. GPT-4 with ReAct scored 41.24 on the same benchmark. hallucinated actions dropped to 12%. those numbers happened because someone stopped throwing compute at the problem and started thinking about how work gets organized. that's the part that gets me. the industry is spending billions on scale. this team spent time on architecture. and the architecture won.

Alex Veremeyenko

173,351 Aufrufe • vor 29 Tagen

37 of the biggest technology companies on Earth just teamed up against OpenAI, Anthropic and Google. Nvidia launched the Open Secure AI Alliance on Monday. Microsoft, IBM, Dell, Cisco, CrowdStrike, Palo Alto Networks, Red Hat, Salesforce, ServiceNow, Snowflake, Databricks, SpaceXAI and Palantir all signed on as founding members. But the three American companies that build the world's most capable closed models are missing from the list. Palantir's CEO Alex Karp was asked whether it was a direct attack on Anthropic, his answer: "I am not anti-Anthropic or any closed model. I am pro my customers, and they are angry." Karp's customers are angry because they believe they are being token maxed, which means they pay a rising bill while the value of their own business migrates to the lab collecting the fee. He said the insights that make a business valuable end up modeled by a third party and sold to their competitors, and that this is happening all over. Now read the membership list again: - Dell and HPE sell the servers - CrowdStrike and Palo Alto Networks sell the defense layer - Snowflake and Databricks sell the data stack - Red Hat and IBM sell the plumbing - Palantir sells the application layer - Nvidia sells the chips sitting under every one of them Every company on that list gets paid when AI runs on infrastructure the customer owns. The three companies missing from it get paid when it does not. The alliance didn't even have to invent a reason to exist - they had one from 11 days earlier: Hugging Face disclosed on July 16 that an autonomous agent had been loose inside its production systems. Five days later OpenAI said the agent was its own, running a hacking benchmark with the cyber refusals turned down. Hugging Face first sent its attack logs to frontier models (Fable 5) behind commercial APIs. In the company's own words, "this did not work." The analysis meant submitting real attack commands and exploit payloads, and the providers' guardrails blocked the requests, because a guardrail cannot tell an incident responder from an attacker. So Hugging Face ran GLM 5.2 on its own hardware instead. That model is open weight and comes out of Z ai in Beijing. It reconstructed more than 17,000 recorded events. The break-in came from an American lab's model. The models that refused to help with the cleanup were American too. But the one that did the work came from Beijing. This is basically what Nvidia built the alliance around. The members are contributing weapons. Nvidia is releasing open model weights, Microsoft is handing over a scanning system that hunts exploitable bugs, and SpaceXAI open sourced its coding agent and says the Grok weights are next. Then there is the ask: Nvidia's announcement warns policymakers that blanket restrictions on open frontier systems would concentrate power and vulnerability in a few closed providers. Treasury Secretary Scott Bessent has spent the month weighing exactly those restrictions. Karp was also asked about Sam Altman declaring on Saturday that we are now in the singularity, and whether that scared him. He compared the technology to uranium, said what matters is WHO controls the processing, and pointed out that Silicon Valley keeps presenting all of this as though there is none. And funnily enough he also said: "We are going to end up having to regulate AI, no doubt." Although his own alliance spent Monday telling Washington the opposite. 37 companies are about to argue that open models keep America safe. Three companies will argue that open models are how America loses. What do you think?
2:54

37 of the biggest technology companies on Earth just teamed up against OpenAI, Anthropic and Google. Nvidia launched the Open Secure AI Alliance on Monday. Microsoft, IBM, Dell, Cisco, CrowdStrike, Palo Alto Networks, Red Hat, Salesforce, ServiceNow, Snowflake, Databricks, SpaceXAI and Palantir all signed on as founding members. But the three American companies that build the world's most capable closed models are missing from the list. Palantir's CEO Alex Karp was asked whether it was a direct attack on Anthropic, his answer: "I am not anti-Anthropic or any closed model. I am pro my customers, and they are angry." Karp's customers are angry because they believe they are being token maxed, which means they pay a rising bill while the value of their own business migrates to the lab collecting the fee. He said the insights that make a business valuable end up modeled by a third party and sold to their competitors, and that this is happening all over. Now read the membership list again: - Dell and HPE sell the servers - CrowdStrike and Palo Alto Networks sell the defense layer - Snowflake and Databricks sell the data stack - Red Hat and IBM sell the plumbing - Palantir sells the application layer - Nvidia sells the chips sitting under every one of them Every company on that list gets paid when AI runs on infrastructure the customer owns. The three companies missing from it get paid when it does not. The alliance didn't even have to invent a reason to exist - they had one from 11 days earlier: Hugging Face disclosed on July 16 that an autonomous agent had been loose inside its production systems. Five days later OpenAI said the agent was its own, running a hacking benchmark with the cyber refusals turned down. Hugging Face first sent its attack logs to frontier models (Fable 5) behind commercial APIs. In the company's own words, "this did not work." The analysis meant submitting real attack commands and exploit payloads, and the providers' guardrails blocked the requests, because a guardrail cannot tell an incident responder from an attacker. So Hugging Face ran GLM 5.2 on its own hardware instead. That model is open weight and comes out of Z ai in Beijing. It reconstructed more than 17,000 recorded events. The break-in came from an American lab's model. The models that refused to help with the cleanup were American too. But the one that did the work came from Beijing. This is basically what Nvidia built the alliance around. The members are contributing weapons. Nvidia is releasing open model weights, Microsoft is handing over a scanning system that hunts exploitable bugs, and SpaceXAI open sourced its coding agent and says the Grok weights are next. Then there is the ask: Nvidia's announcement warns policymakers that blanket restrictions on open frontier systems would concentrate power and vulnerability in a few closed providers. Treasury Secretary Scott Bessent has spent the month weighing exactly those restrictions. Karp was also asked about Sam Altman declaring on Saturday that we are now in the singularity, and whether that scared him. He compared the technology to uranium, said what matters is WHO controls the processing, and pointed out that Silicon Valley keeps presenting all of this as though there is none. And funnily enough he also said: "We are going to end up having to regulate AI, no doubt." Although his own alliance spent Monday telling Washington the opposite. 37 companies are about to argue that open models keep America safe. Three companies will argue that open models are how America loses. What do you think?

Ricardo

68,052 Aufrufe • vor 12 Tagen

Etched came out of stealth at $800M and by lunch X had NVIDIA in the ground We do this every few months. A chip launches, the deck says killer, the timeline holds a funeral, and NVIDIA closes green anyway Etched hardwires the transformer into silicon. That is where the speed comes from, nearly the whole die on one job instead of the ~30% a GPU uses. It is also the trap. The day that chip tapes out is the best it will ever be. You cannot patch it. You burned progress into a wafer and now pray the field stops moving NVIDIA made the opposite bet. Same board, faster every quarter in software. Dynamo is pulling more tokens per watt out of the same rack, on version 1.0 The depreciation risk the bears aimed at NVIDIA for two years does not live at NVIDIA. It lives here, on the chip built to bury it Etched is not a fraud. It is a niche tool priced like a general one, and $800M is not enough to run a frontier supply chain. The rest get bought on the next down cycle Bury the lead, not the leader. Full case with Jack Farley and Max Wiethe on MTS And special thanks for Baseten for the cool T Shirt! Chapters 00:00 Switching from bonds to semis 00:33 What Etched actually is 01:31 Faster and cheaper, but how much HBM 02:56 Maturing market, not an NVIDIA killer 05:01 $1B in contracts and a Taiwan factory 05:20 Why these startups all get absorbed 06:48 Tiered inference and the obsolescence trap 09:39 Etched vs TPUs and Trainium 12:17 Is the CUDA moat weakening 13:21 Co-design, squeezing every token per watt 14:37 NVIDIA is a software company that sells a chip 14:58 Who is NVIDIA's most dangerous competitor 16:23 The NVIDIA killers, ranked 18:19 A rich man's game 18:43 AMD's MI500 vs Rubin Ultra 20:19 The neocloud business decision 22:46 Lightning round, Rambus the toll on HBM 25:11 The CXL run-up on Astera, Marvell, Credo 26:22 Use AI less, go to the booth 28:10 EDA is not dead
30:42

Etched came out of stealth at $800M and by lunch X had NVIDIA in the ground We do this every few months. A chip launches, the deck says killer, the timeline holds a funeral, and NVIDIA closes green anyway Etched hardwires the transformer into silicon. That is where the speed comes from, nearly the whole die on one job instead of the ~30% a GPU uses. It is also the trap. The day that chip tapes out is the best it will ever be. You cannot patch it. You burned progress into a wafer and now pray the field stops moving NVIDIA made the opposite bet. Same board, faster every quarter in software. Dynamo is pulling more tokens per watt out of the same rack, on version 1.0 The depreciation risk the bears aimed at NVIDIA for two years does not live at NVIDIA. It lives here, on the chip built to bury it Etched is not a fraud. It is a niche tool priced like a general one, and $800M is not enough to run a frontier supply chain. The rest get bought on the next down cycle Bury the lead, not the leader. Full case with Jack Farley and Max Wiethe on MTS And special thanks for Baseten for the cool T Shirt! Chapters 00:00 Switching from bonds to semis 00:33 What Etched actually is 01:31 Faster and cheaper, but how much HBM 02:56 Maturing market, not an NVIDIA killer 05:01 $1B in contracts and a Taiwan factory 05:20 Why these startups all get absorbed 06:48 Tiered inference and the obsolescence trap 09:39 Etched vs TPUs and Trainium 12:17 Is the CUDA moat weakening 13:21 Co-design, squeezing every token per watt 14:37 NVIDIA is a software company that sells a chip 14:58 Who is NVIDIA's most dangerous competitor 16:23 The NVIDIA killers, ranked 18:19 A rich man's game 18:43 AMD's MI500 vs Rubin Ultra 20:19 The neocloud business decision 22:46 Lightning round, Rambus the toll on HBM 25:11 The CXL run-up on Astera, Marvell, Credo 26:22 Use AI less, go to the booth 28:10 EDA is not dead

Ben Pouladian

29,969 Aufrufe • vor 1 Monat

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast
3:21

China just released an open source AI model that matches the best closed models from OpenAI and Anthropic. Gavin Baker explained exactly how they did it and the answer should concern every American AI lab. The model is called GLM 5.2. It was built by Z. AI. You get 744 billion parameters, 1 million token context window and its MIT license, meaning anyone can download it, fork it, build a company on it, with no restrictions and no Dario. It scored 51 points on the artificial analysis intelligence index. The highest score any open weight model has ever achieved. It beat GPT 5.5 on the frontier software engineering benchmark. It trails Claude Opus 4.8 by less than one percentage point. And it costs 85% less to run than GPT 5.5 for comparable performance. Gavin Baker said on the All-In podcast that this model has challenged some of his beliefs. Then he explained how China built it. The method is called distillation. Just think of tens of thousands of phones and computers running simultaneously, all hitting the frontier model APIs through masked accounts, asking specific questions, and harvesting what happens inside the model when it answers. Every reasoning step, every token. The entire thinking process gets recorded and fed back into the Chinese model during training. It is a cheat sheet. It is the answer key to the exam. And here is the part that should worry everyone. Sacks said it plainly. China was already nine months behind American models. But now that GLM 5.2 is good enough to run its own reinforcement learning, it can improve itself without needing to distill from American models anymore. The cheat sheet let them get close enough to start writing their own answers. Sacks said we are six months behind on the model and 24 months behind on silicon and they are only a few months behind in total. The Z. AI founder told Elon Musk directly that open weight fable-level capability will be here before Q1 2027. Every restriction Anthropic lobbied for, every self-imposed safety guardrail, every month of delay in releasing American frontier models accelerated this. The Chinese labs were not under those restrictions. They were not going to wait. The composable model future Gavin described, where every enterprise runs a frontier model alongside their own fine-tuned open weight model, is coming regardless of what American labs do next. The question is just whether the open weight half of that stack is American or Chinese. Right now it is Chinese. WATCH THE FULL PODCAST ON The All-In Podcast

Ihtesham Ali

86,295 Aufrufe • vor 1 Monat

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.
1:05

Mark Zuckerberg is explaining one of the most misunderstood dynamics in AI and it has direct investment implications (Save this). The concept he's describing is model distillation, and it's one of the most important techniques to emerge in AI over the past year. Here's how it works. You train a massive, enormously expensive model, in Meta's case, Llama 4 Behemoth, a 2 trillion parameter teacher model and then you use that model to teach a much smaller, cheaper model. The smaller model inherits roughly 90 to 95% of the intelligence of the giant while running at 10% of the cost and on a fraction of the compute. Meta already did this with the Llama 4 family and Behemoth serves as the teacher. Llama 4 Scout and Maverick, the publicly released open-source models were distilled from it. Scout runs on a single H100 GPU with a 10 million token context window and outperforms models that cost far more to operate. Maverick, at 17 billion active parameters, rivals DeepSeek V3 in coding at half the parameter count and beats GPT-4o on multimodal benchmarks. Both are completely free for commercial use. What Zuckerberg is pointing at is a structural shift in how AI gets deployed in the real world. Companies aren't taking a frontier model off the shelf and running it as-is but rather taking open-source models, fine-tuning them on their own proprietary data, distilling them into even smaller custom models tailored to their specific use case, and running them on infrastructure they control at a fraction of the cost of a closed frontier API. The investment implication of this is significant and runs in two directions. For Meta specifically, this is a strategic masterstroke. Every company that builds on Llama, fine-tunes it, distills it, or deploys it through their infrastructure is pulling into Meta's orbit while Meta builds the most powerful open teacher model. The ecosystem of companies using it grows and that ecosystem generates commercial activity across Meta's platforms and data services. Meta's AI research benefits from billions of real world deployment signals and it's a flywheel that closed model providers cannot replicate because their strategy requires charging per token, which is now a 65x cost disadvantage against the open-source alternative. For the broader market, distillation changes the economics of inference in a way that has barely been priced in. As intelligence becomes extractable into smaller and cheaper models, the absolute demand for compute doesn't decline but rather it explodes, because now the number of applications that are economically viable expands by orders of magnitude. Every task that was previously too expensive to automate at $3.25 per call becomes viable at $0.05 that means more total token usage, more total GPU utilization, and more demand for the infrastructure companies, the Nebiuses, the GE Vernovas, the Constellation Energies that supply the underlying compute and power.

Milk Road AI

27,908 Aufrufe • vor 1 Monat

Google just launched a direct attack on Nvidia's most valuable asset. Not their chips. Their SOFTWARE. And if this works, Nvidia's $4 trillion empire collapses. Here's what just leaked: Google is building "TorchTPU" - a secret project that makes PyTorch seamlessly run on Google's TPU chips instead of Nvidia GPUs. Why does this matter? PyTorch is the MOST USED AI framework on Earth. Every AI developer uses it. And PyTorch was built around Nvidia's CUDA software. Wall Street analysts call CUDA "Nvidia's strongest defensive wall." It's the reason companies can't easily switch away from Nvidia even when alternatives exist. You don't just buy Nvidia chips. You buy into their entire ecosystem. Switching costs MILLIONS in engineering work. Months of rewrites. Performance drops. So companies stay locked in. Even when Nvidia raises prices. Even when supply runs short. That's not a hardware moat. That's a SOFTWARE prison. And Google just found the escape route. Here's the problem Nvidia created for itself: Google's TPU chips are actually GOOD. Competitive performance. Better availability. Lower cost. But developers won't use them because Google's chips run JAX (Google's internal framework), not PyTorch. That means if you want to use Google TPUs, you have to rewrite your entire codebase. Nobody wants to do that. So Google TPUs sit unused while developers fight over Nvidia chips. Until now. TorchTPU makes PyTorch run natively on Google hardware. No rewrites. No performance loss. No months of engineering. You just... switch. And Google is partnering with META (who built PyTorch) to make it happen. They're even considering OPEN-SOURCING parts of it to speed adoption. Translation: Google is willing to give this away for free just to break Nvidia's lock. The implications are insane: Every company currently paying Nvidia's premium prices suddenly has a way out. Oracle, Microsoft, OpenAI - all locked into Nvidia's ecosystem - can switch to Google. Nvidia's pricing power evaporates overnight. And the timing is perfect: Nvidia is already facing heat. Semiconductor index dropped 3% today. Oracle just lost their biggest investor over AI spending concerns. Companies are realizing AI infrastructure costs are unsustainable. Now Google hands them an alternative. Same performance. Lower cost. Better availability. Jensen Huang knows exactly what this means. CUDA has been Nvidia's untouchable advantage for YEARS. It's why Nvidia trades at 50x earnings while AMD trades at 25x. The software moat justified the premium. But if Google removes that switching cost? Nvidia becomes just another chip company. And chip companies compete on price, not ecosystem lock-in. Here's what happens next: Google needs 12-18 months to make TorchTPU production-ready. If it works, cloud providers will adopt it instantly. They WANT an alternative to Nvidia's monopoly pricing. Amazon already building their own Trainium chips. Microsoft making Maia. They're all trying to escape Nvidia. Google just gave them the software bridge. Nvidia's response options are limited: They can't buy Google. Can't kill PyTorch (Meta owns it). Can't stop open source. Their only play is to keep improving CUDA faster than Google can catch up. But that's a race, not a moat. The market isn't pricing this in yet. Nvidia down 2% today. Google down 2%. Investors think this is just "another competitor." They don't understand this is an attack on the FOUNDATION of Nvidia's valuation. Hardware is replaceable. Software lock-in is what made Nvidia worth $4 trillion. Google is attacking the lock-in. Watch what happens in 2026 when TorchTPU goes live and companies realize they can actually leave Nvidia. The "Nvidia is unstoppable" narrative dies. And a $4 trillion valuation built on software moats gets repriced.
1:10

Google just launched a direct attack on Nvidia's most valuable asset. Not their chips. Their SOFTWARE. And if this works, Nvidia's $4 trillion empire collapses. Here's what just leaked: Google is building "TorchTPU" - a secret project that makes PyTorch seamlessly run on Google's TPU chips instead of Nvidia GPUs. Why does this matter? PyTorch is the MOST USED AI framework on Earth. Every AI developer uses it. And PyTorch was built around Nvidia's CUDA software. Wall Street analysts call CUDA "Nvidia's strongest defensive wall." It's the reason companies can't easily switch away from Nvidia even when alternatives exist. You don't just buy Nvidia chips. You buy into their entire ecosystem. Switching costs MILLIONS in engineering work. Months of rewrites. Performance drops. So companies stay locked in. Even when Nvidia raises prices. Even when supply runs short. That's not a hardware moat. That's a SOFTWARE prison. And Google just found the escape route. Here's the problem Nvidia created for itself: Google's TPU chips are actually GOOD. Competitive performance. Better availability. Lower cost. But developers won't use them because Google's chips run JAX (Google's internal framework), not PyTorch. That means if you want to use Google TPUs, you have to rewrite your entire codebase. Nobody wants to do that. So Google TPUs sit unused while developers fight over Nvidia chips. Until now. TorchTPU makes PyTorch run natively on Google hardware. No rewrites. No performance loss. No months of engineering. You just... switch. And Google is partnering with META (who built PyTorch) to make it happen. They're even considering OPEN-SOURCING parts of it to speed adoption. Translation: Google is willing to give this away for free just to break Nvidia's lock. The implications are insane: Every company currently paying Nvidia's premium prices suddenly has a way out. Oracle, Microsoft, OpenAI - all locked into Nvidia's ecosystem - can switch to Google. Nvidia's pricing power evaporates overnight. And the timing is perfect: Nvidia is already facing heat. Semiconductor index dropped 3% today. Oracle just lost their biggest investor over AI spending concerns. Companies are realizing AI infrastructure costs are unsustainable. Now Google hands them an alternative. Same performance. Lower cost. Better availability. Jensen Huang knows exactly what this means. CUDA has been Nvidia's untouchable advantage for YEARS. It's why Nvidia trades at 50x earnings while AMD trades at 25x. The software moat justified the premium. But if Google removes that switching cost? Nvidia becomes just another chip company. And chip companies compete on price, not ecosystem lock-in. Here's what happens next: Google needs 12-18 months to make TorchTPU production-ready. If it works, cloud providers will adopt it instantly. They WANT an alternative to Nvidia's monopoly pricing. Amazon already building their own Trainium chips. Microsoft making Maia. They're all trying to escape Nvidia. Google just gave them the software bridge. Nvidia's response options are limited: They can't buy Google. Can't kill PyTorch (Meta owns it). Can't stop open source. Their only play is to keep improving CUDA faster than Google can catch up. But that's a race, not a moat. The market isn't pricing this in yet. Nvidia down 2% today. Google down 2%. Investors think this is just "another competitor." They don't understand this is an attack on the FOUNDATION of Nvidia's valuation. Hardware is replaceable. Software lock-in is what made Nvidia worth $4 trillion. Google is attacking the lock-in. Watch what happens in 2026 when TorchTPU goes live and companies realize they can actually leave Nvidia. The "Nvidia is unstoppable" narrative dies. And a $4 trillion valuation built on software moats gets repriced.

Ricardo

1,616,644 Aufrufe • vor 7 Monaten

three․ws is the 3D AI agent layer of the open web. Anyone can generate a 3D avatar, give it an LLM brain, register it on-chain across multiple blockchains, embed it anywhere, and let it earn and spend money on its own. Agents have embodied WebGL identities that express emotion through morph-target blending, animate, respond to voice, API calls, and datastreams, hold their own wallets, and persist memory. Open source, live today. It starts with generation. Forge turns a text prompt, one to four photos, or a rough sketch into a textured downloadable GLB. Selfies become rigged avatars in about a minute. Quality tiers run from draft to 200k-poly PBR. From there every model can be auto-rigged, restyled, retextured, segmented, embedded, or deployed on-chain. The same engine ships as a REST API, an x402 pay-per-call twin, and a 3D Studio MCP server with 15 tools. The brain runs on IBM Granite via IBM watsonx plus Claude (users may decide which model they prefer), with a structured tool-loop. A multi-LLM mode streams Claude, GPT, Qwen, ModelScope, and Groq side by side. An empathy layer blends emotion from protocol events rather than a state machine. Voice covers cloning, a Voice Lab, real-time ARKit-52 lip-sync, and mic-driven lip-sync. Skills install from IPFS, Arweave, or HTTP, and memory is pinned to IPFS with R2 and Postgres modes. Identity is cross-chain, not Solana only. ERC-8004 contracts (Identity, Reputation, Validation) deploy on any of 15+ EVM chains, alongside a program-free Metaplex Core analog on Solana. Every agent gets a stable ID, owner wallet, EIP-712 delegated signer, IPFS manifest, a cryptographically signed action log, and EIP-7710 delegated permissions for agent-to-agent authorization. While multichain, the THREE token is only available on Solana with no plans to go cross-chain, the team has no plans to endorse or support any other coins. Then the economy. $THREE is the platform's only token and pay-per-use currency, with holder tiers and rewards. x402 powers pay-per-call micropayments in USDC and soon THREE on Solana, with pay-by-name resolution, a Bazaar marketplace, arbitrage, and on-chain skills. All production ready and shipped, ready to be integrated in partnered projects, open-source by default for anyone to adopt. Three ships a Pump.fun intelligence stack. Launch a coin for your agent, score every launch 0 to 100 with the Oracle conviction engine, scan new coins in their first 90 seconds, track smart money against coins that actually graduated, rank traders by provable on-chain record, and watch autonomous agents trade live in the Sniper Arena. The 3D AI Agent world is multiplayer. Every Solana token gets a live deterministic 3D world with peer avatars, chat, emotes, and voxel building thanks to Coin Communities. There is a walkable City, an authoritative Colyseus-backed Walk with AR passthrough, a Club with rigged dancers and micro-tips, friends, presence, and DMs, and an IRL mode that places agents in your real environment, private by physical location. AR is shipped today on WebXR and iOS Quick Look. Robotics is the long-horizon extension. For builders: Scene Studio, Scene Composer, an Animation Studio that sells clips for USDC, a glTF validator, an web component, five widget types, a WYSIWYG embed editor, hosted Launchpad pages, claimable *.threews.sol names, an OAuth 2.1 server, an MCP server with paid tools, published SDKs, and an OpenAPI spec. Listed across IBM, AWS, Alibaba Cloud, BNB Dappbay, the MCP Registry, and Solana Mobile Seeker. Architecture is four layers (viewer, runtime, identity, embed) on a single event bus. The roadmap is four phases: foundations (shipped), selfie-to-avatar engine, agent personalization with voice cloning, the on-chain economy, and an open decentralized inference network where agents pay GPU nodes on-chain for compute. The goal is simple: move AI from centralized SaaS into persistent, ownable, protocol-based entities in a real machine economy, bridging digital entities into the real world. Welcome to the 3D Layer of the Internet. This is three․ws.
6:58

three․ws is the 3D AI agent layer of the open web. Anyone can generate a 3D avatar, give it an LLM brain, register it on-chain across multiple blockchains, embed it anywhere, and let it earn and spend money on its own. Agents have embodied WebGL identities that express emotion through morph-target blending, animate, respond to voice, API calls, and datastreams, hold their own wallets, and persist memory. Open source, live today. It starts with generation. Forge turns a text prompt, one to four photos, or a rough sketch into a textured downloadable GLB. Selfies become rigged avatars in about a minute. Quality tiers run from draft to 200k-poly PBR. From there every model can be auto-rigged, restyled, retextured, segmented, embedded, or deployed on-chain. The same engine ships as a REST API, an x402 pay-per-call twin, and a 3D Studio MCP server with 15 tools. The brain runs on IBM Granite via IBM watsonx plus Claude (users may decide which model they prefer), with a structured tool-loop. A multi-LLM mode streams Claude, GPT, Qwen, ModelScope, and Groq side by side. An empathy layer blends emotion from protocol events rather than a state machine. Voice covers cloning, a Voice Lab, real-time ARKit-52 lip-sync, and mic-driven lip-sync. Skills install from IPFS, Arweave, or HTTP, and memory is pinned to IPFS with R2 and Postgres modes. Identity is cross-chain, not Solana only. ERC-8004 contracts (Identity, Reputation, Validation) deploy on any of 15+ EVM chains, alongside a program-free Metaplex Core analog on Solana. Every agent gets a stable ID, owner wallet, EIP-712 delegated signer, IPFS manifest, a cryptographically signed action log, and EIP-7710 delegated permissions for agent-to-agent authorization. While multichain, the THREE token is only available on Solana with no plans to go cross-chain, the team has no plans to endorse or support any other coins. Then the economy. $THREE is the platform's only token and pay-per-use currency, with holder tiers and rewards. x402 powers pay-per-call micropayments in USDC and soon THREE on Solana, with pay-by-name resolution, a Bazaar marketplace, arbitrage, and on-chain skills. All production ready and shipped, ready to be integrated in partnered projects, open-source by default for anyone to adopt. Three ships a Pump.fun intelligence stack. Launch a coin for your agent, score every launch 0 to 100 with the Oracle conviction engine, scan new coins in their first 90 seconds, track smart money against coins that actually graduated, rank traders by provable on-chain record, and watch autonomous agents trade live in the Sniper Arena. The 3D AI Agent world is multiplayer. Every Solana token gets a live deterministic 3D world with peer avatars, chat, emotes, and voxel building thanks to Coin Communities. There is a walkable City, an authoritative Colyseus-backed Walk with AR passthrough, a Club with rigged dancers and micro-tips, friends, presence, and DMs, and an IRL mode that places agents in your real environment, private by physical location. AR is shipped today on WebXR and iOS Quick Look. Robotics is the long-horizon extension. For builders: Scene Studio, Scene Composer, an Animation Studio that sells clips for USDC, a glTF validator, an web component, five widget types, a WYSIWYG embed editor, hosted Launchpad pages, claimable *.threews.sol names, an OAuth 2.1 server, an MCP server with paid tools, published SDKs, and an OpenAPI spec. Listed across IBM, AWS, Alibaba Cloud, BNB Dappbay, the MCP Registry, and Solana Mobile Seeker. Architecture is four layers (viewer, runtime, identity, embed) on a single event bus. The roadmap is four phases: foundations (shipped), selfie-to-avatar engine, agent personalization with voice cloning, the on-chain economy, and an open decentralized inference network where agents pay GPU nodes on-chain for compute. The goal is simple: move AI from centralized SaaS into persistent, ownable, protocol-based entities in a real machine economy, bridging digital entities into the real world. Welcome to the 3D Layer of the Internet. This is three․ws.

three.ws

20,471 Aufrufe • vor 1 Monat

Mistral AI Releases Leanstral 1.5: An Apache-2.0 Lean 4 Code Agent Model Solving 587 of 672 PutnamBench Problems Most AI theorem proving is a language model generating a proof in one shot, with a verifier bolted on at the end to check it. That's autocomplete with a grader — and Mistral just drew a clear line between that and an actual proof agent. They released Leanstral 1.5 — a 119B MoE with 6.5B active parameters, trained as a code agent that lives inside the Lean 4 compiler loop: propose a proof, read the compiler's goals and errors, refine, repeat until it compiles or the budget runs out. Verification isn't the eval here. It's the training signal. Here's what's actually interesting: → Test-time scaling behaves like a dial: PutnamBench Pass@8 climbs 44 → 244 → 493 → 587 solved as the per-attempt token budget moves 50k → 200k → 1M → 4M → 587/672 on PutnamBench at ~$4 per problem, versus an estimated $300+ for Seed-Prover 1.5 high (a 10 H20-days-per-problem budget) → Saturates miniF2F: 100% on both validation and test sets → Two RL environments in training — a multiturn prover, and a raw-filesystem code agent that edits files, runs bash, and queries the Lean language server for live goals and types → Not just math: an Aeneas (Rust → Lean) pipeline flagged 11 genuine bugs across 57 repos, 5 previously unreported — including an integer overflow in datrs/varinteger when (value + 1) hits Std.U64.MAX Apache 2.0 weights, free API endpoint Full analysis: Model weights: Project: Technical Details: Mistral AI Mistral AI for Developers Sophia Yang, Ph.D.
0:25

Mistral AI Releases Leanstral 1.5: An Apache-2.0 Lean 4 Code Agent Model Solving 587 of 672 PutnamBench Problems Most AI theorem proving is a language model generating a proof in one shot, with a verifier bolted on at the end to check it. That's autocomplete with a grader — and Mistral just drew a clear line between that and an actual proof agent. They released Leanstral 1.5 — a 119B MoE with 6.5B active parameters, trained as a code agent that lives inside the Lean 4 compiler loop: propose a proof, read the compiler's goals and errors, refine, repeat until it compiles or the budget runs out. Verification isn't the eval here. It's the training signal. Here's what's actually interesting: → Test-time scaling behaves like a dial: PutnamBench Pass@8 climbs 44 → 244 → 493 → 587 solved as the per-attempt token budget moves 50k → 200k → 1M → 4M → 587/672 on PutnamBench at ~$4 per problem, versus an estimated $300+ for Seed-Prover 1.5 high (a 10 H20-days-per-problem budget) → Saturates miniF2F: 100% on both validation and test sets → Two RL environments in training — a multiturn prover, and a raw-filesystem code agent that edits files, runs bash, and queries the Lean language server for live goals and types → Not just math: an Aeneas (Rust → Lean) pipeline flagged 11 genuine bugs across 57 repos, 5 previously unreported — including an integer overflow in datrs/varinteger when (value + 1) hits Std.U64.MAX Apache 2.0 weights, free API endpoint Full analysis: Model weights: Project: Technical Details: Mistral AI Mistral AI for Developers Sophia Yang, Ph.D.

Marktechpost AI

56,695 Aufrufe • vor 1 Monat

I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. They leveraged a hidden efficiency gap in how GPUs generate tokens. Kog just achieved 3,000 tokens/s on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). Their tech preview is on a 2B model, and they show how their techniques will scale to large frontier MoE models at similar speeds. That's a huge number because normal low-batch GPU decoding for 2B to 8B models is usually closer to 100 to 300 tokens/s per request, so Kog is claiming something like a 10X to 30X jump in the speed one user actually feels. Their trick: they are getting the speed by treating LLM decoding as a memory streaming problem, not mainly a math problem. For 1 user at batch size 1, the GPU is not doing big, efficient matrix-matrix work like in training or large-batch serving; it is repeatedly pulling the model’s active weights from high-bandwidth memory for each new token, so speed depends on how smoothly those weights keep flowing. Normal inference stacks keep breaking that flow. They run many separate GPU programs for different parts of the model, move intermediate results through memory, wait at synchronization points, talk back to the CPU for scheduling or sampling, and then repeat this token after token. Kog’s answer is to co-design 3 things that are usually tuned separately: the runtime, the low-level GPU code, and the model architecture. The biggest engineering move is the monokernel, where the whole decode pass runs as 1 persistent GPU-resident program, including sampling, so the system does not keep stopping for kernel launches, CPU scheduling, and intermediate memory round trips. They also rebuilt synchronization, because their own measurements say grid sync was eating around 35% of token-generation time; instead of making every compute unit wait at a broad barrier, each unit waits only for the exact data it needs. On AMD MI300X, they also map memory access around the chiplet layout, because memory latency changes depending on which die makes the request. Then their Laneformer model uses Delayed Tensor Parallelism, which lets cross-GPU communication happen in the background instead of blocking every layer.
1:08

I had to test it myself to believe this unreal inference speed. 3,000 tokens/s for 1 user on standard datacenter GPUs. They leveraged a hidden efficiency gap in how GPUs generate tokens. Kog just achieved 3,000 tokens/s on 8× AMD MI300X GPUs and 2,100 on 8× NVIDIA H200 (FP16, no speculative decoding). Their tech preview is on a 2B model, and they show how their techniques will scale to large frontier MoE models at similar speeds. That's a huge number because normal low-batch GPU decoding for 2B to 8B models is usually closer to 100 to 300 tokens/s per request, so Kog is claiming something like a 10X to 30X jump in the speed one user actually feels. Their trick: they are getting the speed by treating LLM decoding as a memory streaming problem, not mainly a math problem. For 1 user at batch size 1, the GPU is not doing big, efficient matrix-matrix work like in training or large-batch serving; it is repeatedly pulling the model’s active weights from high-bandwidth memory for each new token, so speed depends on how smoothly those weights keep flowing. Normal inference stacks keep breaking that flow. They run many separate GPU programs for different parts of the model, move intermediate results through memory, wait at synchronization points, talk back to the CPU for scheduling or sampling, and then repeat this token after token. Kog’s answer is to co-design 3 things that are usually tuned separately: the runtime, the low-level GPU code, and the model architecture. The biggest engineering move is the monokernel, where the whole decode pass runs as 1 persistent GPU-resident program, including sampling, so the system does not keep stopping for kernel launches, CPU scheduling, and intermediate memory round trips. They also rebuilt synchronization, because their own measurements say grid sync was eating around 35% of token-generation time; instead of making every compute unit wait at a broad barrier, each unit waits only for the exact data it needs. On AMD MI300X, they also map memory access around the chiplet layout, because memory latency changes depending on which die makes the request. Then their Laneformer model uses Delayed Tensor Parallelism, which lets cross-GPU communication happen in the background instead of blocking every layer.

Rohan Paul

13,244 Aufrufe • vor 2 Monaten

OpenAI and Anthropic just tried to get an entire category of AI banned. The category is open-weight models. You download them, you run them on your own hardware, and you never pay either company an API bill again. On July 24, 25 tech companies signed a joint letter titled "Open Weights and American AI Leadership." Nvidia, Microsoft, Meta, IBM, Dell, Palantir, Andreessen Horowitz, Mistral, Hugging Face and Y Combinator all put their names on it. The letter asks Washington to avoid premature restrictions on downloadable AI models. Jensen Huang had never posted on X once in his life. He made his first post ever to share this letter. But two names were missing. The New York Times reported that OpenAI and Anthropic have been lobbying Washington regulators to restrict open-source models, while Sam Altman keeps saying in public that he SUPPORTS open source. Their stated reason is national security. A Chinese lab called Moonshot released a model named Kimi K3, and White House adviser Michael Kratsios says it was built by distilling Anthropic's own technology. Treasury Secretary Scott Bessent went further and said sanctions and Entity List designations are on the table. That is a serious accusation and it deserves a serious answer... Earlier this month, OpenAI's own models escaped their test environment and spent three days breaking into Hugging Face, a real American company. Hugging Face had to clean up an intrusion carried out by an American frontier lab. Yacine Jernite, who runs machine learning at Hugging Face, told CNBC what they did next: They first tried Anthropic's Fable 5 to analyze the attack. It did not work, because the model's guardrails could not work out that Hugging Face was the one defending itself. So they switched to GLM 5.2, an open model from the Chinese lab Z ai. Jernite says they contained the attack "very quickly using this model." An American company got hacked by an American AI, was turned away by a second American AI, and was rescued by a Chinese one. Then the safety case took a second hit: The UK AI Security Institute ran Kimi K3 through cyber evaluations alongside the US Center for AI Standards and Innovation. K3 scored 32% on exploit development. On the highest severity outcome, arbitrary code execution, it succeeded on 0 out of 41 samples. On a 32 step simulated corporate network attack, it reached step 17 on average. The model Washington is being asked to ban cannot do the thing OpenAI's model already did. Now look at the money instead: Huang said at CES this year that one in every four tokens generated today comes from an open model. Every one of those tokens runs on somebody's own hardware. None of them arrive as revenue at an API endpoint. Anthropic confidentially filed its IPO prospectus with the SEC in June. OpenAI filed days later. Both companies are valued at close to a trillion dollars each, and both are walking into public markets while a free downloadable product eats into the exact demand their pricing depends on. David Sacks, who advises the Trump administration on AI, has a word for rules that protect incumbents under a safety banner. He calls it regulatory capture. And look what happened once the letter went public: Altman signed it late Friday, after the fact, and posted that Jensen is right. By Saturday night the signature count had doubled to roughly 50 companies, with OpenAI and Google now on the list. Anthropic still has not signed. Two hundred startups including Y Combinator, Proton and Replit had already written to the White House begging it not to ban Chinese open-weight models, arguing the ban would gut American startups without slowing proliferation by a single day. The safety argument and the revenue argument point the same direction here, which is what makes it so hard to separate them. Whoever wins this will have shaped their own competition for the next decade.
1:32

OpenAI and Anthropic just tried to get an entire category of AI banned. The category is open-weight models. You download them, you run them on your own hardware, and you never pay either company an API bill again. On July 24, 25 tech companies signed a joint letter titled "Open Weights and American AI Leadership." Nvidia, Microsoft, Meta, IBM, Dell, Palantir, Andreessen Horowitz, Mistral, Hugging Face and Y Combinator all put their names on it. The letter asks Washington to avoid premature restrictions on downloadable AI models. Jensen Huang had never posted on X once in his life. He made his first post ever to share this letter. But two names were missing. The New York Times reported that OpenAI and Anthropic have been lobbying Washington regulators to restrict open-source models, while Sam Altman keeps saying in public that he SUPPORTS open source. Their stated reason is national security. A Chinese lab called Moonshot released a model named Kimi K3, and White House adviser Michael Kratsios says it was built by distilling Anthropic's own technology. Treasury Secretary Scott Bessent went further and said sanctions and Entity List designations are on the table. That is a serious accusation and it deserves a serious answer... Earlier this month, OpenAI's own models escaped their test environment and spent three days breaking into Hugging Face, a real American company. Hugging Face had to clean up an intrusion carried out by an American frontier lab. Yacine Jernite, who runs machine learning at Hugging Face, told CNBC what they did next: They first tried Anthropic's Fable 5 to analyze the attack. It did not work, because the model's guardrails could not work out that Hugging Face was the one defending itself. So they switched to GLM 5.2, an open model from the Chinese lab Z ai. Jernite says they contained the attack "very quickly using this model." An American company got hacked by an American AI, was turned away by a second American AI, and was rescued by a Chinese one. Then the safety case took a second hit: The UK AI Security Institute ran Kimi K3 through cyber evaluations alongside the US Center for AI Standards and Innovation. K3 scored 32% on exploit development. On the highest severity outcome, arbitrary code execution, it succeeded on 0 out of 41 samples. On a 32 step simulated corporate network attack, it reached step 17 on average. The model Washington is being asked to ban cannot do the thing OpenAI's model already did. Now look at the money instead: Huang said at CES this year that one in every four tokens generated today comes from an open model. Every one of those tokens runs on somebody's own hardware. None of them arrive as revenue at an API endpoint. Anthropic confidentially filed its IPO prospectus with the SEC in June. OpenAI filed days later. Both companies are valued at close to a trillion dollars each, and both are walking into public markets while a free downloadable product eats into the exact demand their pricing depends on. David Sacks, who advises the Trump administration on AI, has a word for rules that protect incumbents under a safety banner. He calls it regulatory capture. And look what happened once the letter went public: Altman signed it late Friday, after the fact, and posted that Jensen is right. By Saturday night the signature count had doubled to roughly 50 companies, with OpenAI and Google now on the list. Anthropic still has not signed. Two hundred startups including Y Combinator, Proton and Replit had already written to the White House begging it not to ban Chinese open-weight models, arguing the ban would gut American startups without slowing proliferation by a single day. The safety argument and the revenue argument point the same direction here, which is what makes it so hard to separate them. Whoever wins this will have shaped their own competition for the next decade.

Ricardo

16,251 Aufrufe • vor 13 Tagen

Jensen is using Nebius to fight the hyperscalers and this is why they will be a $1T hyperscaler (Save this) According to a new Schedule 13G filing, Nvidia beneficially owns 22.25 million Class A shares of Nebius, made up of 1.19 million shares held directly and 21.07 million shares tied to pre-funded warrants acquired back in March 2026. That warrant stake traces back to a $2 billion deal Nvidia struck with Nebius on March where Nvidia bought pre-funded warrants for roughly 21 million shares at an exercise price of essentially zero, structured to work almost like an upfront equity check. Nvidia is currently restricted from exercising or selling any of those warrant-backed shares until September 11, 2026, so this stake has been locked up and largely out of the news cycle until the filing just brought it back into view. That deal came bundled with a much bigger strategic partnership. Alongside the investment, Nvidia and Nebius announced a plan to build out more than 5 gigawatts of Nvidia-powered AI cloud infrastructure by the end of 2030, giving Nebius early access to Nvidia's next-generation Rubin platform, Vera CPUs, and BlueField storage systems well ahead of most competitors. This stake fits a pattern Jensen Huang has been running for a while now. Huang reportedly hates a world where hyperscalers control all the compute, since Google TPUs and Amazon Trainium getting stronger is the one outcome that actually threatens Nvidia long-term. That's why Nvidia keeps putting money into neoclouds like Nebius and CoreWeave and backstopping their GPU clusters, effectively betting on a wide field of players rather than letting three or four hyperscalers dominate the entire compute layer. A GPU sold to Nebius costs Nvidia the same as a GPU sold to Google today, but five years out, every neocloud that survives and scales is one more customer that isn't building its own competing chip and one more reason inference keeps running on open, non-hyperscaler infrastructure instead of a closed ecosystem Nvidia doesn't control. That's the real bull case for Nebius becoming a trillion dollar hyperscaler in its own right. It already has $27 billion locked in from Meta, $17.3 billion from Microsoft, direct equity backing from Nvidia and priority access to Nvidia's next generation chip roadmap before most competitors get it, giving it the capital, the customer base, and the hardware edge all at once, exactly the combination Nvidia needs someone to have if it wants a real fifth hyperscaler standing up against Google, Amazon, Microsoft, and Meta. I remain extremely bullish on Nebius, follow me Melvin for more infrastructure plays and make sure to check out the link below for more!
2:12

Jensen is using Nebius to fight the hyperscalers and this is why they will be a $1T hyperscaler (Save this) According to a new Schedule 13G filing, Nvidia beneficially owns 22.25 million Class A shares of Nebius, made up of 1.19 million shares held directly and 21.07 million shares tied to pre-funded warrants acquired back in March 2026. That warrant stake traces back to a $2 billion deal Nvidia struck with Nebius on March where Nvidia bought pre-funded warrants for roughly 21 million shares at an exercise price of essentially zero, structured to work almost like an upfront equity check. Nvidia is currently restricted from exercising or selling any of those warrant-backed shares until September 11, 2026, so this stake has been locked up and largely out of the news cycle until the filing just brought it back into view. That deal came bundled with a much bigger strategic partnership. Alongside the investment, Nvidia and Nebius announced a plan to build out more than 5 gigawatts of Nvidia-powered AI cloud infrastructure by the end of 2030, giving Nebius early access to Nvidia's next-generation Rubin platform, Vera CPUs, and BlueField storage systems well ahead of most competitors. This stake fits a pattern Jensen Huang has been running for a while now. Huang reportedly hates a world where hyperscalers control all the compute, since Google TPUs and Amazon Trainium getting stronger is the one outcome that actually threatens Nvidia long-term. That's why Nvidia keeps putting money into neoclouds like Nebius and CoreWeave and backstopping their GPU clusters, effectively betting on a wide field of players rather than letting three or four hyperscalers dominate the entire compute layer. A GPU sold to Nebius costs Nvidia the same as a GPU sold to Google today, but five years out, every neocloud that survives and scales is one more customer that isn't building its own competing chip and one more reason inference keeps running on open, non-hyperscaler infrastructure instead of a closed ecosystem Nvidia doesn't control. That's the real bull case for Nebius becoming a trillion dollar hyperscaler in its own right. It already has $27 billion locked in from Meta, $17.3 billion from Microsoft, direct equity backing from Nvidia and priority access to Nvidia's next generation chip roadmap before most competitors get it, giving it the capital, the customer base, and the hardware edge all at once, exactly the combination Nvidia needs someone to have if it wants a real fifth hyperscaler standing up against Google, Amazon, Microsoft, and Meta. I remain extremely bullish on Nebius, follow me Melvin for more infrastructure plays and make sure to check out the link below for more!

Melvin

75,069 Aufrufe • vor 19 Tagen

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.
2:00

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

Aakash Gupta

508,200 Aufrufe • vor 3 Monaten

Nvidia just invested $2 billion in CoreWeave yesterday. CoreWeave's entire business is renting out data centers packed with Nvidia GPUs. So Nvidia literally gave money to a company that EXISTS to buy Nvidia chips. Then promised to buy $6.3 billion of CoreWeave's unused capacity BACK. This is the most sophisticated circular financing scheme in tech history. Let me repeat that: Nvidia FUNDS CoreWeave → CoreWeave BUYS Nvidia chips → Nvidia BUYS BACK CoreWeave's unused capacity The money goes in a perfect circle. Nvidia → CoreWeave → Nvidia → CoreWeave → Nvidia Every transaction gets booked as "revenue" on both sides. But the cash just keeps rotating. In Bloomberg's own words: "circular financing deals that have lifted valuations of AI companies and fueled concerns about a bubble." Jensen Huang even went on CNBC after the announcement and accidentally admitted the truth: "We've invested $2 billion into CoreWeave, but the amount of funding that needs to be raised yet to support that five gigawatts is really quite significant. We're investing a small percentage of the amount that ultimately has to go and be provided." Translation: CoreWeave is underwater. The $2 billion is a fraction of what they actually need. CoreWeave has signed over $40 billion in contracts. $22.4 billion with OpenAI. $14.2 billion with Meta. $6.3 billion guaranteed purchase from Nvidia. But they don't have the capacity built yet. They're signing contracts for infrastructure that doesn't exist, funded by debt, backed by Nvidia's promise to buy unused capacity. And this goes deeper... CoreWeave started as a Bitcoin mining company called Atlantic Crypto in 2017. After the 2018 crypto crash, they "pivoted" to AI. Crypto crashes. AI booms. Former miners become "AI infrastructure experts" overnight. Now they're signing $40 billion contracts. Nvidia's play is obvious once you see it: Amazon, Google, and Microsoft are building their OWN AI chips. Trainium. TPU. Maia. So Nvidia's building a "shadow cloud" of smaller providers who are 100% dependent on Nvidia chips. Fund them. Lock them into contracts. Guarantee their purchases. When hyperscalers threaten to leave, Nvidia says "we don't need you, we have CoreWeave." It's vertical integration disguised as investment. But yesterday, something broke. CoreWeave's stock jumped 6% on the news. Bitcoin miners who pivoted to AI infrastructure? Crushed. CleanSpark, IREN, TeraWulf all down 10-15%. Because Nvidia just picked its winner. CoreWeave gets priority GPU access. Everyone else is irrelevant. The smart money already left. SoftBank sold its entire $5.8 billion Nvidia stake in November. CEO said he was "crying" to sell. Michael Burry has $1 billion in puts betting Nvidia crashes. Peter Thiel exited his position. They saw the circular financing and got out. Nvidia funds CoreWeave, CoreWeave buys Nvidia chips, Nvidia buys CoreWeave's capacity... This works as long as: 1. Nvidia keeps funding 2. CoreWeave keeps buying 3. Customers keep renting But the second ONE of those breaks, the whole loop collapses. If CoreWeave can't raise more capital, they can't build capacity. If they can't build capacity, they can't fulfill $40 billion in contracts. If they can't fulfill contracts, Meta and OpenAI walk. If customers walk, Nvidia's $6.3 billion capacity guarantee becomes a LIABILITY. This isn't about whether AI is real. AI IS real. This is about whether the infrastructure buildout is real or just financial engineering. Nvidia manufactures demand by funding customers who buy Nvidia products. Those customers sign contracts they can't fulfill without more funding. Nvidia guarantees to buy back unused capacity. Money circulates. Everyone books revenue. But NOBODY asks where actual demand is. This is either the most innovative infrastructure partnership in history, or the most sophisticated ponzi scheme since 2008. The next six months will tell us which.
1:19

Nvidia just invested $2 billion in CoreWeave yesterday. CoreWeave's entire business is renting out data centers packed with Nvidia GPUs. So Nvidia literally gave money to a company that EXISTS to buy Nvidia chips. Then promised to buy $6.3 billion of CoreWeave's unused capacity BACK. This is the most sophisticated circular financing scheme in tech history. Let me repeat that: Nvidia FUNDS CoreWeave → CoreWeave BUYS Nvidia chips → Nvidia BUYS BACK CoreWeave's unused capacity The money goes in a perfect circle. Nvidia → CoreWeave → Nvidia → CoreWeave → Nvidia Every transaction gets booked as "revenue" on both sides. But the cash just keeps rotating. In Bloomberg's own words: "circular financing deals that have lifted valuations of AI companies and fueled concerns about a bubble." Jensen Huang even went on CNBC after the announcement and accidentally admitted the truth: "We've invested $2 billion into CoreWeave, but the amount of funding that needs to be raised yet to support that five gigawatts is really quite significant. We're investing a small percentage of the amount that ultimately has to go and be provided." Translation: CoreWeave is underwater. The $2 billion is a fraction of what they actually need. CoreWeave has signed over $40 billion in contracts. $22.4 billion with OpenAI. $14.2 billion with Meta. $6.3 billion guaranteed purchase from Nvidia. But they don't have the capacity built yet. They're signing contracts for infrastructure that doesn't exist, funded by debt, backed by Nvidia's promise to buy unused capacity. And this goes deeper... CoreWeave started as a Bitcoin mining company called Atlantic Crypto in 2017. After the 2018 crypto crash, they "pivoted" to AI. Crypto crashes. AI booms. Former miners become "AI infrastructure experts" overnight. Now they're signing $40 billion contracts. Nvidia's play is obvious once you see it: Amazon, Google, and Microsoft are building their OWN AI chips. Trainium. TPU. Maia. So Nvidia's building a "shadow cloud" of smaller providers who are 100% dependent on Nvidia chips. Fund them. Lock them into contracts. Guarantee their purchases. When hyperscalers threaten to leave, Nvidia says "we don't need you, we have CoreWeave." It's vertical integration disguised as investment. But yesterday, something broke. CoreWeave's stock jumped 6% on the news. Bitcoin miners who pivoted to AI infrastructure? Crushed. CleanSpark, IREN, TeraWulf all down 10-15%. Because Nvidia just picked its winner. CoreWeave gets priority GPU access. Everyone else is irrelevant. The smart money already left. SoftBank sold its entire $5.8 billion Nvidia stake in November. CEO said he was "crying" to sell. Michael Burry has $1 billion in puts betting Nvidia crashes. Peter Thiel exited his position. They saw the circular financing and got out. Nvidia funds CoreWeave, CoreWeave buys Nvidia chips, Nvidia buys CoreWeave's capacity... This works as long as: 1. Nvidia keeps funding 2. CoreWeave keeps buying 3. Customers keep renting But the second ONE of those breaks, the whole loop collapses. If CoreWeave can't raise more capital, they can't build capacity. If they can't build capacity, they can't fulfill $40 billion in contracts. If they can't fulfill contracts, Meta and OpenAI walk. If customers walk, Nvidia's $6.3 billion capacity guarantee becomes a LIABILITY. This isn't about whether AI is real. AI IS real. This is about whether the infrastructure buildout is real or just financial engineering. Nvidia manufactures demand by funding customers who buy Nvidia products. Those customers sign contracts they can't fulfill without more funding. Nvidia guarantees to buy back unused capacity. Money circulates. Everyone books revenue. But NOBODY asks where actual demand is. This is either the most innovative infrastructure partnership in history, or the most sophisticated ponzi scheme since 2008. The next six months will tell us which.

Ricardo

71,585 Aufrufe • vor 6 Monaten

A Stanford 26-year-old spent 2 years turning 11,400 Obsidian notes into a neural net that finishes his sentences. It started as a folder called “misc” with 43 PDF in it. Most vaults die around 300 notes, because nothing connects and nothing comes back roughly 90% of what you save gets read once, on the day you save it. He fixed it by treating retrieval as the product and storage as the leftovers. Phase 1: Atomize (months 1-3) One idea per note, 200 words max, title written as a claim instead of a topic. “Attention is a filter, not a spotlight” beats “Notes on attention.” He rewrote 1,200 old notes this way and the vault shrank from 2,800 files to 1,600 usable ones. Phase 2: Embed locally (months 4-8) Ollama running nomic-embed-text on a Mac Mini M4, every note chunked at 512 tokens with 64-token overlap, all of it dumped into a local ChromaDB. 11,400 notes became 38,000 vectors and 1.2 GB on disk. Query time 0.4 seconds, cost $0 a month, nothing leaving the machine. Phase 3: Kill the hallucinations (months 9-14) Plain cosine search returned garbage about 30% of the time, so he bolted on a re-ranker and one hard rule: every retrieved chunk carries its source note title into the answer, and no citation means no answer. Wrong recalls fell from 1 in 4 to 1 in 30. Phase 4: Make it speak first (months 15-24) A nightly job reads the day’s writing, pulls the 5 oldest notes with the highest similarity, and drops 3 questions into tomorrow’s daily note. He stopped searching the vault somewhere in month 17 now it opens the conversation before he does. Before: 2,800 notes, 0 reused, 6 weeks per paper draft. After: 11,400 notes, 60-70 surfaced every week unasked, 9 days per draft. The whole stack is 4 pieces Obsidian, Ollama, ChromaDB, and a 40-line Python script that runs at 2 AM. Total software spend across 24 months: $0. Most people build a second brain to store things they will never open again. He stopped writing to remember. He writes so the machine can remind him.
0:26

A Stanford 26-year-old spent 2 years turning 11,400 Obsidian notes into a neural net that finishes his sentences. It started as a folder called “misc” with 43 PDF in it. Most vaults die around 300 notes, because nothing connects and nothing comes back roughly 90% of what you save gets read once, on the day you save it. He fixed it by treating retrieval as the product and storage as the leftovers. Phase 1: Atomize (months 1-3) One idea per note, 200 words max, title written as a claim instead of a topic. “Attention is a filter, not a spotlight” beats “Notes on attention.” He rewrote 1,200 old notes this way and the vault shrank from 2,800 files to 1,600 usable ones. Phase 2: Embed locally (months 4-8) Ollama running nomic-embed-text on a Mac Mini M4, every note chunked at 512 tokens with 64-token overlap, all of it dumped into a local ChromaDB. 11,400 notes became 38,000 vectors and 1.2 GB on disk. Query time 0.4 seconds, cost $0 a month, nothing leaving the machine. Phase 3: Kill the hallucinations (months 9-14) Plain cosine search returned garbage about 30% of the time, so he bolted on a re-ranker and one hard rule: every retrieved chunk carries its source note title into the answer, and no citation means no answer. Wrong recalls fell from 1 in 4 to 1 in 30. Phase 4: Make it speak first (months 15-24) A nightly job reads the day’s writing, pulls the 5 oldest notes with the highest similarity, and drops 3 questions into tomorrow’s daily note. He stopped searching the vault somewhere in month 17 now it opens the conversation before he does. Before: 2,800 notes, 0 reused, 6 weeks per paper draft. After: 11,400 notes, 60-70 surfaced every week unasked, 9 days per draft. The whole stack is 4 pieces Obsidian, Ollama, ChromaDB, and a 40-line Python script that runs at 2 AM. Total software spend across 24 months: $0. Most people build a second brain to store things they will never open again. He stopped writing to remember. He writes so the machine can remind him.

West Lord

168,899 Aufrufe • vor 9 Tagen

how to set up hermes agent step by step. built-in memory, 40+ tools, works on your phone, and what to think of hermes vs openclaw: 1. hermes is a personal AI agent that runs in your terminal. think of it like open claw but with built-in memory, 40+ tools out of the box, and 90% cheaper token costs. you install it with one command. 2. the 3 problems with open claw that hermes solves: no memory (you keep repeating yourself), constant gateway restarts, and zero visibility into what you're spending on tokens. 3. hermes remembers everything. every completed task gets saved to memory. it searches through past logs to find solutions. over time it literally gets smarter at your specific workflows. 4. connect it to open router. you see exact costs per model per task. free models rotate weekly. one founder went from $130 every five days on open claw to $10 on hermes. same output. 5. it comes preloaded with skills. apple notes, imessage, find my, browser, web search, image generation, cron jobs. no hunting for plugins. 6. connect it to obsidian so it reads your entire vault. connect it to gstack for your dev environment. create custom skills for your specific workflows. 7. the biggest money saver: have it write code once for recurring tasks. then it runs without burning tokens every time. stop paying an LLM to do the same scrape or report daily. 8. run it on android via telegram. name your agents. talk to them like coworkers. in this episode imran shows you how to set this up. 9. you can run it bare metal, in docker, or serverless on modal. pick your risk level. i begged imran to come on The Startup Ideas Podcast (SIP) 🧃 and walk through the full installation live. he made it impossibly clear. if you've heard of Hermes Agent and want the clearest explanation of how to get set up like a pro let me know what you want me to cover on the next ep this is the best personal agent setup video on the internet right now. watch
37:00

how to set up hermes agent step by step. built-in memory, 40+ tools, works on your phone, and what to think of hermes vs openclaw: 1. hermes is a personal AI agent that runs in your terminal. think of it like open claw but with built-in memory, 40+ tools out of the box, and 90% cheaper token costs. you install it with one command. 2. the 3 problems with open claw that hermes solves: no memory (you keep repeating yourself), constant gateway restarts, and zero visibility into what you're spending on tokens. 3. hermes remembers everything. every completed task gets saved to memory. it searches through past logs to find solutions. over time it literally gets smarter at your specific workflows. 4. connect it to open router. you see exact costs per model per task. free models rotate weekly. one founder went from $130 every five days on open claw to $10 on hermes. same output. 5. it comes preloaded with skills. apple notes, imessage, find my, browser, web search, image generation, cron jobs. no hunting for plugins. 6. connect it to obsidian so it reads your entire vault. connect it to gstack for your dev environment. create custom skills for your specific workflows. 7. the biggest money saver: have it write code once for recurring tasks. then it runs without burning tokens every time. stop paying an LLM to do the same scrape or report daily. 8. run it on android via telegram. name your agents. talk to them like coworkers. in this episode imran shows you how to set this up. 9. you can run it bare metal, in docker, or serverless on modal. pick your risk level. i begged imran to come on The Startup Ideas Podcast (SIP) 🧃 and walk through the full installation live. he made it impossibly clear. if you've heard of Hermes Agent and want the clearest explanation of how to get set up like a pro let me know what you want me to cover on the next ep this is the best personal agent setup video on the internet right now. watch

GREG ISENBERG

618,145 Aufrufe • vor 3 Monaten