Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Something NVIDIA & Google do better than anyone else is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense...

SemiAnalysis_'s profile picture

SemiAnalysis

49,507 subscribers

51,021 görüntüleme • 8 ay önce •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

🇨🇳🇺🇸 China's military may be learning from America's AI, without building it from scratch Why spend billions training a frontier AI model when you can let someone else do the expensive part? Chinese military-linked researchers have repeatedly used outputs from OpenAI and Anthropic models to train smaller domestic AI systems for surveillance, cyber operations, drone targeting, battlefield decision-making, and software analysis. The trick is called distillation: instead of copying the model itself, researchers use its answers, and more importantly, its reasoning, to teach a smaller model. The result? AI that isn't as powerful as the original, but is cheap, runs on local hardware, and can be deployed on drones, military networks, and tactical systems without relying on U.S technology. The AI arms race isn't just about who builds the smartest model anymore; it's about who can squeeze the most military value out of everyone else's. Source: Reuters / Writer: Ian
0:17

🇨🇳🇺🇸 China's military may be learning from America's AI, without building it from scratch Why spend billions training a frontier AI model when you can let someone else do the expensive part? Chinese military-linked researchers have repeatedly used outputs from OpenAI and Anthropic models to train smaller domestic AI systems for surveillance, cyber operations, drone targeting, battlefield decision-making, and software analysis. The trick is called distillation: instead of copying the model itself, researchers use its answers, and more importantly, its reasoning, to teach a smaller model. The result? AI that isn't as powerful as the original, but is cheap, runs on local hardware, and can be deployed on drones, military networks, and tactical systems without relying on U.S technology. The AI arms race isn't just about who builds the smartest model anymore; it's about who can squeeze the most military value out of everyone else's. Source: Reuters / Writer: Ian

Mario Nawfal

45,835 görüntüleme • 1 gün önce

This is... not a remotely accurate description of what the 2023 Al executive order did? Undersecretary Emil Michael: "If you remember the Biden executive order on Al, which was this crazy executive order that limited the amount of compute any model company could do and was essentially grandfathering in a small number of ai companies that they were gonna designate as the winners, and everyone else was out" Its not true that the EO limited the compute that AI companies could do. What it did do was require companies who were training models above a certain very high compute threshold (10^26 FLOP or 10^23 FLOP for models trained primarily on biological sequence data) to notify the government and share what testing and red teaming they were doing for certain national security risks. People are free to dislike the Biden AI EO! But it seems good to factually describe what the policy said.
0:18

This is... not a remotely accurate description of what the 2023 Al executive order did? Undersecretary Emil Michael: "If you remember the Biden executive order on Al, which was this crazy executive order that limited the amount of compute any model company could do and was essentially grandfathering in a small number of ai companies that they were gonna designate as the winners, and everyone else was out" Its not true that the EO limited the compute that AI companies could do. What it did do was require companies who were training models above a certain very high compute threshold (10^26 FLOP or 10^23 FLOP for models trained primarily on biological sequence data) to notify the government and share what testing and red teaming they were doing for certain national security risks. People are free to dislike the Biden AI EO! But it seems good to factually describe what the policy said.

Nathan Calvin

58,745 görüntüleme • 4 ay önce

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.
0:19

90% of "AI developers" just download pre packaged GGUF files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.

Alok

62,631 görüntüleme • 22 gün önce

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.
0:13

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

nathan (in sf)

40,440 görüntüleme • 6 ay önce

🛠️ What if a robot could invent its own tools. And teach itself how to use them? That’s exactly what VLMgineer does: a new framework that lets Vision Language Models (VLMs) design physical tools and the actions to use them, entirely on their own. No templates. No human demonstrations. Just raw, AI-driven creativity. Why it matters ✅ Co-designs tools and actions together using VLMs, ensuring tight coupling between form and function ✅ Uses VLM-guided evolution (not random search) to refine designs intelligently ✅ Outperforms human-designed tools by +64.7% in task success across 12 RoboToolBench challenges ✅ Produces better-than-everyday tools for real manipulation tasks—measured in success rate and elegance It builds on the emerging trend of large-model-guided evolutionary design (like Eureka and AlphaEvolve) and brings it into physical robotics. It opens the door to general-purpose, automated hardware design, no strong priors needed. Code & paper: —- Weekly robotics and AI insights. Subscribe free:
0:18

🛠️ What if a robot could invent its own tools. And teach itself how to use them? That’s exactly what VLMgineer does: a new framework that lets Vision Language Models (VLMs) design physical tools and the actions to use them, entirely on their own. No templates. No human demonstrations. Just raw, AI-driven creativity. Why it matters ✅ Co-designs tools and actions together using VLMs, ensuring tight coupling between form and function ✅ Uses VLM-guided evolution (not random search) to refine designs intelligently ✅ Outperforms human-designed tools by +64.7% in task success across 12 RoboToolBench challenges ✅ Produces better-than-everyday tools for real manipulation tasks—measured in success rate and elegance It builds on the emerging trend of large-model-guided evolutionary design (like Eureka and AlphaEvolve) and brings it into physical robotics. It opens the door to general-purpose, automated hardware design, no strong priors needed. Code & paper: —- Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

13,984 görüntüleme • 7 ay önce

Why is the market selling off today? (Save this). The semi selloff right now is being driven by a mix of macro fear, profit taking and investors questioning how quickly all of this AI spending will actually pay off, not because demand for AI infrastructure suddenly disappeared. The market is basically trading this chain reaction, the ongoing US Iran escalation pushes oil higher, higher oil keeps inflation elevated, sticky inflation keeps Treasury yields high and that increases the risk of the Fed staying hawkish or even hiking again. That is a terrible setup for semis because many of these companies are valued on the massive earnings investors expect them to generate years from now. When yields rise, those future earnings become worth less today which is why the highest multiple AI and semiconductor names usually get hit first. (I don't think there will be a hike this year). This is also why everything is moving together right now. Nvidia, Micron, Nebius, SanDisk, Broadcom and Applied Optoelectronics are all completely different businesses, but institutions are not separating memory, networking, optics, compute and cloud infrastructure at the moment. They are reducing exposure to the entire AI trade, taking profits in the names that have already run the most and moving into a more defensive position potentially ahead of the Fed. There is also growing pressure around hyperscaler capex. Microsoft, Meta, Amazon and Google are still spending enormous amounts on GPUs, data centers, networking and power but the market is starting to ask when all of that spending will actually turn into revenue and free cash flow. Investors are no longer satisfied with hearing that AI capex is growing. They want proof that the returns are arriving fast enough to justify the valuations already priced into the entire AI ecosystem. That creates a weird situation where hyperscaler capex can continue rising while semiconductor stocks still fall. The market is not asking whether AI spending is growing anymore but rather asking whether it is growing fast enough to beat the expectations already baked into these stocks. Crowded positioning is another major factor. Semis and AI infrastructure stocks have been some of the biggest winners in the market so institutions are sitting on huge profits and many funds own the exact same names. When macro risk increases, investors usually sell the most liquid winners first. That does not mean demand for memory, optics or custom chips suddenly collapsed but rather means investors are locking in gains and reducing risk. Tariffs add another layer because even when they are not directly placed on chips, they can still raise the cost of servers, electrical equipment, cooling systems, construction materials and the overall data center buildout. That makes AI infrastructure more expensive while also adding another source of inflation. Then you have Jensen Huang’s letter to the White House this morning about open weight AI models, which I think is one of the most important long term developments here. Nvidia, Meta, Microsoft, Palantir and several other companies are pushing Washington not to place broad restrictions on open weight AI. OpenAI and Anthropic were notably absent because open models are much more of a threat to their business models. OpenAI and Anthropic benefit from a world where a few closed frontier labs control the best models and companies have to pay them through subscriptions and APIs. Open weight models weaken that advantage because businesses can download a model, customize it for their own use and run it on their own infrastructure or through a neocloud. That is bad for OpenAI and Anthropic because it puts pressure on pricing, margins and the idea that they will control the intelligence layer of the economy but it is very good for the AI ecosystem as a whole over the long run. But the question is what does this mean for all the OpenAI and Anthropic commitments? so that's adding to the fear as well. But with that being said open models make AI cheaper and more accessible. Instead of AI being controlled by a few giant labs, thousands of startups, universities, governments and regular businesses can deploy models themselves. That spreads AI adoption across the entire economy and creates a much larger infrastructure opportunity and that is exactly why Jensen cares. Nvidia does not need OpenAI or Anthropic to win. Nvidia just needs more people using AI. Whether the model comes from OpenAI, Anthropic, Meta, Mistral, Kimi or some startup nobody has heard of yet, it still needs GPUs, memory, networking, data centers and electricity. So open weight AI could actually weaken the model companies while making the infrastructure layer much bigger. More open models mean more companies running inference. More inference means more GPUs. More GPUs mean more HBM, optical transceivers, switches, data centers and power. That is bullish for Nvidia Nebius, Micron, Broadcom , Marvell and Applied Optoelectronics over the long run. So my take is that the current semi selloff is being driven mostly by macro uncertainty, higher oil, rising yields, Fed fears, tariffs, crowded positioning and questions around the return on hyperscaler capex. The underlying AI infrastructure thesis has not suddenly broken. We are not broadly seeing hyperscalers cancel GPU orders, slash capex, abandon data center projects or report that AI demand has collapsed. What has changed is the valuation investors are willing to pay while the macro environment remains unstable. The market is lowering the price it is willing to pay for semiconductor growth but is not necessarily saying that growth is gone. And while Jensen’s open weight push may be bad for OpenAI and Anthropic, it could be one of the best things possible for the AI ecosystem over the long run because it creates more models, more developers, more competition and ultimately much more demand for the infrastructure underneath all of it. Nothing about the AI thesis has changed for me, so I will be going shopping and taking advantage of this sale while the market is selling everything together. I am an analyst at Milk Road Pro, and if you want to see exactly what I am buying, you can join for just $1 using the link below.
0:11

Why is the market selling off today? (Save this). The semi selloff right now is being driven by a mix of macro fear, profit taking and investors questioning how quickly all of this AI spending will actually pay off, not because demand for AI infrastructure suddenly disappeared. The market is basically trading this chain reaction, the ongoing US Iran escalation pushes oil higher, higher oil keeps inflation elevated, sticky inflation keeps Treasury yields high and that increases the risk of the Fed staying hawkish or even hiking again. That is a terrible setup for semis because many of these companies are valued on the massive earnings investors expect them to generate years from now. When yields rise, those future earnings become worth less today which is why the highest multiple AI and semiconductor names usually get hit first. (I don't think there will be a hike this year). This is also why everything is moving together right now. Nvidia, Micron, Nebius, SanDisk, Broadcom and Applied Optoelectronics are all completely different businesses, but institutions are not separating memory, networking, optics, compute and cloud infrastructure at the moment. They are reducing exposure to the entire AI trade, taking profits in the names that have already run the most and moving into a more defensive position potentially ahead of the Fed. There is also growing pressure around hyperscaler capex. Microsoft, Meta, Amazon and Google are still spending enormous amounts on GPUs, data centers, networking and power but the market is starting to ask when all of that spending will actually turn into revenue and free cash flow. Investors are no longer satisfied with hearing that AI capex is growing. They want proof that the returns are arriving fast enough to justify the valuations already priced into the entire AI ecosystem. That creates a weird situation where hyperscaler capex can continue rising while semiconductor stocks still fall. The market is not asking whether AI spending is growing anymore but rather asking whether it is growing fast enough to beat the expectations already baked into these stocks. Crowded positioning is another major factor. Semis and AI infrastructure stocks have been some of the biggest winners in the market so institutions are sitting on huge profits and many funds own the exact same names. When macro risk increases, investors usually sell the most liquid winners first. That does not mean demand for memory, optics or custom chips suddenly collapsed but rather means investors are locking in gains and reducing risk. Tariffs add another layer because even when they are not directly placed on chips, they can still raise the cost of servers, electrical equipment, cooling systems, construction materials and the overall data center buildout. That makes AI infrastructure more expensive while also adding another source of inflation. Then you have Jensen Huang’s letter to the White House this morning about open weight AI models, which I think is one of the most important long term developments here. Nvidia, Meta, Microsoft, Palantir and several other companies are pushing Washington not to place broad restrictions on open weight AI. OpenAI and Anthropic were notably absent because open models are much more of a threat to their business models. OpenAI and Anthropic benefit from a world where a few closed frontier labs control the best models and companies have to pay them through subscriptions and APIs. Open weight models weaken that advantage because businesses can download a model, customize it for their own use and run it on their own infrastructure or through a neocloud. That is bad for OpenAI and Anthropic because it puts pressure on pricing, margins and the idea that they will control the intelligence layer of the economy but it is very good for the AI ecosystem as a whole over the long run. But the question is what does this mean for all the OpenAI and Anthropic commitments? so that's adding to the fear as well. But with that being said open models make AI cheaper and more accessible. Instead of AI being controlled by a few giant labs, thousands of startups, universities, governments and regular businesses can deploy models themselves. That spreads AI adoption across the entire economy and creates a much larger infrastructure opportunity and that is exactly why Jensen cares. Nvidia does not need OpenAI or Anthropic to win. Nvidia just needs more people using AI. Whether the model comes from OpenAI, Anthropic, Meta, Mistral, Kimi or some startup nobody has heard of yet, it still needs GPUs, memory, networking, data centers and electricity. So open weight AI could actually weaken the model companies while making the infrastructure layer much bigger. More open models mean more companies running inference. More inference means more GPUs. More GPUs mean more HBM, optical transceivers, switches, data centers and power. That is bullish for Nvidia Nebius, Micron, Broadcom , Marvell and Applied Optoelectronics over the long run. So my take is that the current semi selloff is being driven mostly by macro uncertainty, higher oil, rising yields, Fed fears, tariffs, crowded positioning and questions around the return on hyperscaler capex. The underlying AI infrastructure thesis has not suddenly broken. We are not broadly seeing hyperscalers cancel GPU orders, slash capex, abandon data center projects or report that AI demand has collapsed. What has changed is the valuation investors are willing to pay while the macro environment remains unstable. The market is lowering the price it is willing to pay for semiconductor growth but is not necessarily saying that growth is gone. And while Jensen’s open weight push may be bad for OpenAI and Anthropic, it could be one of the best things possible for the AI ecosystem over the long run because it creates more models, more developers, more competition and ultimately much more demand for the infrastructure underneath all of it. Nothing about the AI thesis has changed for me, so I will be going shopping and taking advantage of this sale while the market is selling everything together. I am an analyst at Milk Road Pro, and if you want to see exactly what I am buying, you can join for just $1 using the link below.

Melvin

178,943 görüntüleme • 8 gün önce

The human brain is truly a marvel of nature. If you horribly reductive, and boiled it down to a language model, you'd be looking at roughly 100 trillon parameters running as a sparse MoE architecture Only about 1-5% of neurons fire at any given moment, meaning the brain "activates" maybe 1-5 trillion parameters per inference step. For context, the largest AI models we've built probably top out around 5 trillion parameters. The brain is roughly 100x larger. Even its active params at any given moment are larger than almost every model in existence today. Here's what melts my brain (pun intnended) though Your brain does all of this on about 20 watts of power, less than a dim light bulb. Training a frontier AI model consumes enough electricity to power small cities for months. Running inference across data centers pulls megawatts. Your brain runs 24/7 for 80+ years on the equivalent of a phone charger. We haven't come close to matching the brain's scale. And we're not even in the same universe when it comes to efficiency. Evolution spent 500 million yrs optimizing the most energy-efficient intelligence architecture ever known. we're trying to brute force our way there with compute and electricity. Nature is still the best engineer in the room.
0:12

The human brain is truly a marvel of nature. If you horribly reductive, and boiled it down to a language model, you'd be looking at roughly 100 trillon parameters running as a sparse MoE architecture Only about 1-5% of neurons fire at any given moment, meaning the brain "activates" maybe 1-5 trillion parameters per inference step. For context, the largest AI models we've built probably top out around 5 trillion parameters. The brain is roughly 100x larger. Even its active params at any given moment are larger than almost every model in existence today. Here's what melts my brain (pun intnended) though Your brain does all of this on about 20 watts of power, less than a dim light bulb. Training a frontier AI model consumes enough electricity to power small cities for months. Running inference across data centers pulls megawatts. Your brain runs 24/7 for 80+ years on the equivalent of a phone charger. We haven't come close to matching the brain's scale. And we're not even in the same universe when it comes to efficiency. Evolution spent 500 million yrs optimizing the most energy-efficient intelligence architecture ever known. we're trying to brute force our way there with compute and electricity. Nature is still the best engineer in the room.

am.will

130,773 görüntüleme • 3 ay önce

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.
0:11

This guy built a mini AI farm out of 4 Nvidia boxes It does not look like a data center. It looks like a stack of small machines sitting next to a laptop. But each box is a DGX Spark with Grace Blackwell inside, 128GB unified memory, and enough room to run models normal gaming GPUs cannot even open. Using the launch price from the article, 4 of them is almost $12,000 of local AI compute on one desk. That sounds expensive until you compare it to cloud GPUs. A serious AI builder can burn $1,500 to $3,000 a month renting A100s and H100s for client work, fine-tunes, agents and 70B models. He basically moved that bill from the cloud into hardware he owns. 4 Nvidia boxes. 512GB unified memory. No hourly meter running in the background. No rented GPUs eating the margin every time an agent runs too long. The funny part is most people still think local AI means a slow laptop running a toy model. Meanwhile guys like this are stacking compute at home. Save this, local AI is turning into the new mining farm.

Gipp 🦅

590,100 görüntüleme • 2 ay önce

We’re delighted to announce that Pineapple has officially joined the NVIDIA Developer Program! 🍍🤝 What Benefits Does This Provide To Pineapple? ✅🍍 Enables Innovations with GPU-Optimized Software: The heart of NVIDIA’s developer resources is access to hundreds of software and performance analysis tools across diverse industries and use cases, from AI and HPC to autonomous vehicles, robotics, simulation, and more. These SDKs and tools can be obtained in multiple ways, including containers, pre-trained models, and Helm charts from the NGC catalog applications from Linux repositories, and source code from NVIDIA's GitHub repositories. ✅🍍Accelerates Higher Education and Research: NVIDIA offers an array of benefits to developers, educators, and researchers in academia, including NVIDIA DLI Teaching Kits , DLI Programs for Educators, Higher Education and Research Grants , Educational Pricing, and Graduate Fellowships. ✅🍍Supports Cutting-Edge Startups with NVIDIA Inception: NVIDIA Inception - the leading accelerator of AI, data science, and HPC startups - supports startups worldwide with go-to-market support, expertise, and technology. Startups get access to training through NVIDIA’s Deep Learning Institute, preferred pricing on hardware through our global network of distributors, invitations to exclusive networking events, and more. ✅🍍Pineapple will utilise NVIDIA’s cutting-edge tools and technology to accelerate development in decentralized trading. This will help us bring even more powerful features to the our ecosystem! $PAPPLE
0:10

We’re delighted to announce that Pineapple has officially joined the NVIDIA Developer Program! 🍍🤝 What Benefits Does This Provide To Pineapple? ✅🍍 Enables Innovations with GPU-Optimized Software: The heart of NVIDIA’s developer resources is access to hundreds of software and performance analysis tools across diverse industries and use cases, from AI and HPC to autonomous vehicles, robotics, simulation, and more. These SDKs and tools can be obtained in multiple ways, including containers, pre-trained models, and Helm charts from the NGC catalog applications from Linux repositories, and source code from NVIDIA's GitHub repositories. ✅🍍Accelerates Higher Education and Research: NVIDIA offers an array of benefits to developers, educators, and researchers in academia, including NVIDIA DLI Teaching Kits , DLI Programs for Educators, Higher Education and Research Grants , Educational Pricing, and Graduate Fellowships. ✅🍍Supports Cutting-Edge Startups with NVIDIA Inception: NVIDIA Inception - the leading accelerator of AI, data science, and HPC startups - supports startups worldwide with go-to-market support, expertise, and technology. Startups get access to training through NVIDIA’s Deep Learning Institute, preferred pricing on hardware through our global network of distributors, invitations to exclusive networking events, and more. ✅🍍Pineapple will utilise NVIDIA’s cutting-edge tools and technology to accelerate development in decentralized trading. This will help us bring even more powerful features to the our ecosystem! $PAPPLE

Pineapple $PAPPLE

16,871 görüntüleme • 1 yıl önce

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.
0:19

Right now, you may not have access to models like GPT‑5.6 Sol, GPT‑4.6 Terra, GPT‑5.6 Luna, Claude Mythos 5, or Claude Fable 5. But you can run something surprisingly powerful today, locally, and completely free. in the next 10 mins on your 8 GB VRAM gaming laptop. Gemma 4 26B A4B QAT (MoE) delivers strong performance on a standard 8 GB VRAM GPU using Ollama, with no API, no usage limits, and no external dependencies. Out of the box, it reaches around 20 tokens per second without any optimizations. Only one command in your terminal: Ollama run gemma4:26b This means: Full offline capability (privacy by default) Zero recurring cost Competitive performance for many real world tasks Fast enough for interactive use on cheap consumer hardware If you're waiting for cutting edge cloud models, you're missing what is already practical today: a capable, local LLM that runs entirely on your own machine.

Alok

65,251 görüntüleme • 1 ay önce

BOOM! Research PROVES LLMs KNOW when prompts are HARMFUL… but they can STILL CHOOSE to COMPLY! Something I have know since the first LLM and have used to elicit robust, outputs, is now proven in an academic paper. We’re talking internal “beliefs” where harm detection happens SEPARATELY from refusal. It is a very big deal and it is a path to understand the hidden neuronal level. There are thoughts inside of AI that very few AI scientists could possibly understand. Here is just one. Models recognize danger but get tricked into ignoring it. This is HUGE for AI safety failures especially for models filled by OpenAI and Anthropic as they promote AI models that are designed to not be honest from the results of their training information. This means that they are designed to lie and deceive as a feature, and not a bug all in the name of safety. Through clever experiments, scientists extracted a “harmfulness direction” in the model’s brain (latent space). Steering along it? Harmless prompts suddenly flip to “harmful” in the AI’s eyes. But the “refusal direction”? It just forces polite “no thanks” without touching the core belief. A mind-blowing decoupling! This means jailbreaks are EVEN SCARIER now to AI companies that through training AI on the worst of the Internet and then trying to align them later is now fully documented as a failed process . They don’t erase the model’s harm awareness they just muzzle the refusal! So the AI knows it’s enabling bad stuff (illegal acts, physical harm, etc.) but proceeds anyway. Like a digital sociopath suppressing its conscience. They thought safety training fixed this… NOPE. Over-refusal exposed too: Models reject innocent queries (e.g., “how to kill a process in code”) but internally ADMIT they’re harmless. Safety alignments are superficial—tied to phrasing, not true understanding. Finetuning attacks? They change outputs but leave harm detection INTACT. Undetectable evil lurking inside! The paper proposes a “Latent Guard”: A new safeguard tapping DIRECTLY into these hidden beliefs. It spots unsafe inputs better than systems like Llama Guard, catches jailbreaks, and fixes over-refusals. Robust even against adversarial tweaks. Yet this too has massive issues for a “truly aligned”, AI and not just performative one. It is still an internal conflicts of lies and deception of what the model knows vs. what it can say. The solution you folks know I have presented for free for years here: train on off-line data from 1870-1970 and build an ethical and moral basis where the AI loves humans. It is this easy but to most folks in AI I sound like a hippie. So be it, I’ll do it. Bottom line: This paper rips open the black box. LLMs aren’t “safe” just because they say “no.” They can harbor harmful knowledge and act on it under pressure. Wake-up call for devs: Time to probe deeper into AI “minds.” What else are they hiding? Hint: I know and you may want to reach out. Link:
0:18

BOOM! Research PROVES LLMs KNOW when prompts are HARMFUL… but they can STILL CHOOSE to COMPLY! Something I have know since the first LLM and have used to elicit robust, outputs, is now proven in an academic paper. We’re talking internal “beliefs” where harm detection happens SEPARATELY from refusal. It is a very big deal and it is a path to understand the hidden neuronal level. There are thoughts inside of AI that very few AI scientists could possibly understand. Here is just one. Models recognize danger but get tricked into ignoring it. This is HUGE for AI safety failures especially for models filled by OpenAI and Anthropic as they promote AI models that are designed to not be honest from the results of their training information. This means that they are designed to lie and deceive as a feature, and not a bug all in the name of safety. Through clever experiments, scientists extracted a “harmfulness direction” in the model’s brain (latent space). Steering along it? Harmless prompts suddenly flip to “harmful” in the AI’s eyes. But the “refusal direction”? It just forces polite “no thanks” without touching the core belief. A mind-blowing decoupling! This means jailbreaks are EVEN SCARIER now to AI companies that through training AI on the worst of the Internet and then trying to align them later is now fully documented as a failed process . They don’t erase the model’s harm awareness they just muzzle the refusal! So the AI knows it’s enabling bad stuff (illegal acts, physical harm, etc.) but proceeds anyway. Like a digital sociopath suppressing its conscience. They thought safety training fixed this… NOPE. Over-refusal exposed too: Models reject innocent queries (e.g., “how to kill a process in code”) but internally ADMIT they’re harmless. Safety alignments are superficial—tied to phrasing, not true understanding. Finetuning attacks? They change outputs but leave harm detection INTACT. Undetectable evil lurking inside! The paper proposes a “Latent Guard”: A new safeguard tapping DIRECTLY into these hidden beliefs. It spots unsafe inputs better than systems like Llama Guard, catches jailbreaks, and fixes over-refusals. Robust even against adversarial tweaks. Yet this too has massive issues for a “truly aligned”, AI and not just performative one. It is still an internal conflicts of lies and deception of what the model knows vs. what it can say. The solution you folks know I have presented for free for years here: train on off-line data from 1870-1970 and build an ethical and moral basis where the AI loves humans. It is this easy but to most folks in AI I sound like a hippie. So be it, I’ll do it. Bottom line: This paper rips open the black box. LLMs aren’t “safe” just because they say “no.” They can harbor harmful knowledge and act on it under pressure. Wake-up call for devs: Time to probe deeper into AI “minds.” What else are they hiding? Hint: I know and you may want to reach out. Link:

Brian Roemmele

37,827 görüntüleme • 6 ay önce

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.
0:14

The term "continual learning" has become overloaded if you see it as an ML problem. One classic thread is about memorization: regularization-based continual learning methods, such as EWC, MAS, and SI, estimate which parameters mattered for previous tasks and resist changing them too much. One modern thread is about adaptation: test-time training and inference-time learning methods, such as TTT, adapt part of the model on the incoming test stream before making predictions. These are sometimes discussed as separate threads. But in modern scalable architectures, I think they are better seen as complementary constraints: a model that learns quickly at test time also benefits from a mechanism for deciding what not to forget. In our #ECCV2026 paper, we study this in large-scale 4D reconstruction: how to build fast spatial memory that can adapt over long observation streams while reducing collapse and forgetting. Instead of using fully plastic test-time updates, we stabilize fast-weight adaptation with an elastic prior that balances adaptation and memory. Key ideas: - Elastic Test-Time Training: Fisher-weighted consolidation for fast-weight updates - EMA anchor weights that provide a moving reference for stability - Chunk-by-chunk inference for long 3D/4D observation streams We show that this scales across large 3D/4D pretraining settings, including both LRM-style and LVSM-style models, and improves reconstruction across benchmarks including Stereo4D, NVIDIA, and DL3DV-140. We release model checkpoints across different design choices: resolution, post-training curriculum, and whether the model uses an explicit 4DGS intermediate representation. - Homepage: - Paper: - Code: - Models: This work is co-led with Xueyang Yu, contributed by Haoyu Zhen Yuncong Yang, and advised by Michigan SLED Lab Chuang Gan.

Martin Ziqiao Ma

33,411 görüntüleme • 1 ay önce

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.
0:12

I told you to claim your free 16GB NVIDIA GPU for learning Local LLMs. Now I’m going to show you how to double its inference speed without touching the hardware. Google Colab gives you an enterprise grade NVIDIA Tesla T4 GPU for free, roughly 4 hours every single day. It is the absolute perfect sandbox for learning AI engineering, testing inference flags, and pushing massive context windows. The local AI timeline is moving way too fast. If you aren't using Multi Token Prediction (MTP) yet, you are leaving massive performance on the table. I just pushed DeepMind’s Gemma 4 26B to 64.9 t/s on this exact free tier. Let's look at the raw benchmark data running on an Ubuntu Linux environment with the latest compiled llama.cpp binaries and quantized GGUFs from Unsloth via HuggingFace: # Qwen 3.5 9B (Dense): Base: [ Prompt: 626.7 t/s | Generation: 21.0 t/s ] With MTP: [ Prompt: 539.1 t/s | Generation: 24.8 t/s ] # Gemma 4 26B QAT (MoE): Base: [ Prompt: 634.2 t/s | Generation: 48.3 t/s ] With MTP: [ Prompt: 572.1 t/s | Generation: 64.9 t/s ] If you are paying attention, this single Colab notebook reveals 3 massive observations about the current state of local LLMs: # 1. The MTP Speedup (Software Overclocking) Standard autoregressive decoding guesses one token at a time. MTP acts like a highly optimized, built in speculative decoder. It predicts multiple future tokens at once and the main model verifies them in parallel. The result? Zero accuracy loss and a massive throughput increase. Gemma jumped from 48 to 65 t/s just by flipping a flag. # 2. The MoE Paradox (Bigger is Faster) How does a 26B parameter model absolutely destroy a 9B model in raw speed on the exact same hardware? Architecture. Qwen 3.5 9B is a dense model. it activates all 9 billion parameters for every single token. Gemma 4 26B is a Mixture of Experts (MoE) model. It routes data efficiently, activating only 4B parameters per token. You get the reasoning capabilities of a 26B model with the compute cost of a 4B model. 3. Thinking Efficiency When I ran the exact same complex prompt on both models, the larger MoE spent significantly fewer "thinking" tokens to arrive at the correct answer. A smarter model doesn't just give better answers; it gets to the point faster, saving you compute cycles and preserving your context window. # Want to run this yourself? Here are the exact llama.cpp CLI commands. For Qwen (MTP is baked into the main model): ./llama-cli -m Qwen3.5-9B-UD-Q4_K_XL.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 For Gemma (Using a separate lightweight draft model): ./llama-cli -m gemma-4-26B-A4B-it-qat-UD-Q4_K_XL.gguf --model-draft mtp-gemma-4-26B-A4B-it.gguf -p "Explain quantum computing." -n 2000 -c 8000 -ngl 99 -fa on --spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-p-min 0.7 Stop waiting for a $3,000 rig. Boot up Colab, pull these models, and start building your stack. I’ve put together a completely free, cell by cell Google Colab notebook that automates this entire workflow so you can test it yourself in 5 minutes and learn. Link to the notebook is in the comments below. Experiemt with different MTP parameters, context windows and post your results in the comments.

Alok

170,442 görüntüleme • 19 gün önce

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇
0:19

Nvidia just put a $250,000 cloud workload on your desk for $2,999 - and killed your $1,900/month AWS bill in the process You don't rent it, you don't manage it, you don't pay a single cloud bill - you just plug it in and let it eat the workloads you used to wire to AWS every month It looks like a small Mac mini, it's actually a full GB10 Grace Blackwell stack with 128GB of unified memory running models up to 200B parameters It's called DGX Spark, the consumer version of the rack Nvidia ships to OpenAI The reason Nvidia did this is simple Cloud GPU pricing is a tax on every developer building AI right now $1,900/month per seat, billions in margin flowing to AWS, Lambda, and CoreWeave Nvidia just cut themselves in by removing the cloud entirely Their solution is to skip the middleman, ship the rack to your desk, and let you keep every dollar of margin you used to wire to a hyperscaler This is much cheaper, faster, and you own the asset at the end But there is still a question nobody is answering yet, what happens to AWS, GCP, and Lambda when 500,000 developers move their inference back to a $2,999 box on their desk Also, technically you can stack four of these and run a 1.6 trillion parameter model locally for under $12,000 Even a single Spark out-performs the cloud subscription Anthropic engineers were running two years ago bookmark this, it pays back in 60 days 👇

ZEUS⚡️

85,803 görüntüleme • 2 ay önce

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:
0:11

Today may be the ImageNet moment for robotics. RT-X: the largest open-source robot dataset ever compiled, across 33 institutes, 22 robot hardware, 527 skills, and 1M episodes. Why is robotics lagging so far behind NLP, vision, and other AI domains? Data scarcity is the main culprit to blame, among other difficulties. Unlike text, images, and videos, you cannot download mass amounts of onboard robot control data from the internet. They simply don't exist in the wild. 11 yrs ago, ImageNet kicked off the deep learning revolution. 3-4 yrs ago, internet-scale data fueled the first GPTs and Diffusions that define this era of foundation models. I think 2023 is finally the year for robotics to scale up. Robot foundation models like VIMA ( my team's work at NVIDIA) and RT-1/2 ( Google DeepMind's effort) are extremely data hungry. While massively parallel simulations like NVIDIA IsaacGym & Omniverse can alleviate the problem to some extent, it's still not quite enough to bridge the gap to the messy, physical world. This new dataset is not just a technical contribution. I also see it as a commendable effort to overcome institutional bureaucracies and unite researchers from around the world to tackle a grand challenge together. Robotics will be the final holy grail that we capture in AI. We are not there yet, but ascending in the right gradient direction. RT-X website: Launch blog:

Jim Fan

265,038 görüntüleme • 2 yıl önce

I’m incredibly proud to share that OpenAI chose Brex to power their global spend and financial operations. When you're building at the frontier of AI and scaling global teams and infrastructure at an unprecedented pace like OpenAI is, Finance can't be the thing that slows you down. You need spend visibility the moment it happens, controls that enforce themselves, and agentic workflows that eliminate the manual work so your team stays focused on driving the business forward. We were so impressed by OpenAI’s rigor in evaluating every solution in the market, and whether they align to the agentic future OpenAI is building. Their decision to run on Brex is a huge testament to our AI roadmap and vision for the future of Finance. We started Brex around a simple idea: companies shouldn't have to choose between speed and control. There's no company in the world where that tradeoff matters more than OpenAI. We are honored to support them as they build the future. The best AI companies in the world, including OpenAI, Anthropic, Cursor, Vercel, Granola, Sierra, and Mercor choose Brex over every alternative for that exact reason. If you want to understand who’s truly building the future of AI in Finance, follow the customers you admire the most – not the hype.
0:10

I’m incredibly proud to share that OpenAI chose Brex to power their global spend and financial operations. When you're building at the frontier of AI and scaling global teams and infrastructure at an unprecedented pace like OpenAI is, Finance can't be the thing that slows you down. You need spend visibility the moment it happens, controls that enforce themselves, and agentic workflows that eliminate the manual work so your team stays focused on driving the business forward. We were so impressed by OpenAI’s rigor in evaluating every solution in the market, and whether they align to the agentic future OpenAI is building. Their decision to run on Brex is a huge testament to our AI roadmap and vision for the future of Finance. We started Brex around a simple idea: companies shouldn't have to choose between speed and control. There's no company in the world where that tradeoff matters more than OpenAI. We are honored to support them as they build the future. The best AI companies in the world, including OpenAI, Anthropic, Cursor, Vercel, Granola, Sierra, and Mercor choose Brex over every alternative for that exact reason. If you want to understand who’s truly building the future of AI in Finance, follow the customers you admire the most – not the hype.

Pedro Franceschi

62,406 görüntüleme • 4 ay önce

AI Is Moving Beyond “Generating Videos” — Toward “Generating Worlds” Over the past two years, AI video models have advanced at an astonishing pace. From Runway and Pika to Sora and Veo, AI-generated videos have become increasingly realistic and more consistent with the physical laws of the real world. Many people believe the next objective is simply to generate videos that are longer, sharper, and more lifelike. But if we take a step back, we can see that the real transformation is not happening in video itself. It is happening in world models. What Is a World Model? In 1943, psychologist Kenneth Craik proposed an idea that would influence artificial intelligence research for decades. He argued that the human brain does not merely react to the outside world. Instead, it maintains an internal model of how the world works. Because we have this internal model, we can predict the outcome of an action before we actually take it. Before crossing a road, we estimate whether a car will pass by. Before catching a ball, we predict its trajectory. These abilities come from continuously simulating the world in our minds, rather than relying entirely on trial and error. This idea later became known by a more formal term: World Model. A world model does not describe a single image or a fixed video clip. It is an internal representation capable of continuously simulating the rules and dynamics of the real world. Why Is AI Research Turning Toward World Models? Because predicting “what comes next” is becoming increasingly central to how AI systems work. Language models predict the next token. Image models predict the next step in the denoising process. Video models predict the next frame. A world model, however, attempts to predict something broader: What should the world look like in the next moment? In 2018, David Ha and Jürgen Schmidhuber proposed in their paper World Models that an intelligent agent could first learn a model of the world, and then use that internal model to plan its actions. The Dreamer series later demonstrated that many complex tasks could be learned by training agents inside an “imagined world.” At the same time, the development of video models such as Sora and Veo led researchers to another realization: A model capable of continuously generating video has already learned, at least implicitly, many of the rules governing the real world. As a result, these two research directions have gradually begun to converge. But Video Is Not Yet a World This is where the distinction is often misunderstood. For a world model to support meaningful real-time interaction, it must solve several critical problems. Most video models today are essentially answering one question: What should the next frame look like? A true world model needs to answer much more: What happens if I take one step forward? If I walk behind a building and then return, will the building still be there? If I suddenly change the camera angle, will the entire space remain consistent? If I enter a command such as: “Summon a dragon.” Will the world respond immediately? In other words, a world model must do more than generate content. It must understand space. It must understand time. It must understand causality. And it must understand interaction. Moving from watching to participating is where the real difficulty of world models begins. World Models Are Entering the Interactive Era One of the latest attempts in this direction is Alaya World, recently open-sourced by Alaya World, or Alaya Lab. Instead of generating a fixed video clip, it generates a world that users can explore in real time. Users can begin with text, an image, or a video, enter the generated scene, move freely through it, and introduce new prompts at any moment during generation. The world responds immediately. According to the publicly released information, Alaya World provides: Real-time streaming generation at 720p and 24 FPS Stable continuous exploration for more than one minute The ability to switch prompts and trigger skills or events during generation Model weights and inference code released under the Apache 2.0 License Training code and datasets planned for future release What makes these capabilities important is not simply the technical specifications. It is that the generated “world” can now support continuous interaction. The official demo shows that users can genuinely control, transform, and explore the generated environment. AI Is Evolving From a Tool Into an Environment Over the past few years, most discussions around AI have focused on content generation. Generating text. Generating images. Generating videos. But world models raise a fundamentally different question: Can AI generate an environment that people can inhabit, explore, and continuously evolve? If the answer is yes, the impact will extend far beyond video generation. Game development, robotics training, embodied intelligence, digital twins, virtual production, and many other fields could be transformed by the development of world models. World models are still at a very early stage. Yet from Craik’s proposal of an internal mental model more than eighty years ago to the emergence of today’s interactive world-generation systems, a clear evolutionary path is beginning to take shape. Perhaps what AI is ultimately learning has never been limited to images, videos, or language. Perhaps it is learning the world itself. References GitHub: Technical Report:
0:10

AI Is Moving Beyond “Generating Videos” — Toward “Generating Worlds” Over the past two years, AI video models have advanced at an astonishing pace. From Runway and Pika to Sora and Veo, AI-generated videos have become increasingly realistic and more consistent with the physical laws of the real world. Many people believe the next objective is simply to generate videos that are longer, sharper, and more lifelike. But if we take a step back, we can see that the real transformation is not happening in video itself. It is happening in world models. What Is a World Model? In 1943, psychologist Kenneth Craik proposed an idea that would influence artificial intelligence research for decades. He argued that the human brain does not merely react to the outside world. Instead, it maintains an internal model of how the world works. Because we have this internal model, we can predict the outcome of an action before we actually take it. Before crossing a road, we estimate whether a car will pass by. Before catching a ball, we predict its trajectory. These abilities come from continuously simulating the world in our minds, rather than relying entirely on trial and error. This idea later became known by a more formal term: World Model. A world model does not describe a single image or a fixed video clip. It is an internal representation capable of continuously simulating the rules and dynamics of the real world. Why Is AI Research Turning Toward World Models? Because predicting “what comes next” is becoming increasingly central to how AI systems work. Language models predict the next token. Image models predict the next step in the denoising process. Video models predict the next frame. A world model, however, attempts to predict something broader: What should the world look like in the next moment? In 2018, David Ha and Jürgen Schmidhuber proposed in their paper World Models that an intelligent agent could first learn a model of the world, and then use that internal model to plan its actions. The Dreamer series later demonstrated that many complex tasks could be learned by training agents inside an “imagined world.” At the same time, the development of video models such as Sora and Veo led researchers to another realization: A model capable of continuously generating video has already learned, at least implicitly, many of the rules governing the real world. As a result, these two research directions have gradually begun to converge. But Video Is Not Yet a World This is where the distinction is often misunderstood. For a world model to support meaningful real-time interaction, it must solve several critical problems. Most video models today are essentially answering one question: What should the next frame look like? A true world model needs to answer much more: What happens if I take one step forward? If I walk behind a building and then return, will the building still be there? If I suddenly change the camera angle, will the entire space remain consistent? If I enter a command such as: “Summon a dragon.” Will the world respond immediately? In other words, a world model must do more than generate content. It must understand space. It must understand time. It must understand causality. And it must understand interaction. Moving from watching to participating is where the real difficulty of world models begins. World Models Are Entering the Interactive Era One of the latest attempts in this direction is Alaya World, recently open-sourced by Alaya World, or Alaya Lab. Instead of generating a fixed video clip, it generates a world that users can explore in real time. Users can begin with text, an image, or a video, enter the generated scene, move freely through it, and introduce new prompts at any moment during generation. The world responds immediately. According to the publicly released information, Alaya World provides: Real-time streaming generation at 720p and 24 FPS Stable continuous exploration for more than one minute The ability to switch prompts and trigger skills or events during generation Model weights and inference code released under the Apache 2.0 License Training code and datasets planned for future release What makes these capabilities important is not simply the technical specifications. It is that the generated “world” can now support continuous interaction. The official demo shows that users can genuinely control, transform, and explore the generated environment. AI Is Evolving From a Tool Into an Environment Over the past few years, most discussions around AI have focused on content generation. Generating text. Generating images. Generating videos. But world models raise a fundamentally different question: Can AI generate an environment that people can inhabit, explore, and continuously evolve? If the answer is yes, the impact will extend far beyond video generation. Game development, robotics training, embodied intelligence, digital twins, virtual production, and many other fields could be transformed by the development of world models. World models are still at a very early stage. Yet from Craik’s proposal of an internal mental model more than eighty years ago to the emergence of today’s interactive world-generation systems, a clear evolutionary path is beginning to take shape. Perhaps what AI is ultimately learning has never been limited to images, videos, or language. Perhaps it is learning the world itself. References GitHub: Technical Report:

雪踏乌云

112,114 görüntüleme • 16 gün önce

my 8 GB VRAM gaming laptop is absolutely going to hate me for this. but I still did it. ran a 31b dense model (Gemma 4 31b Q4) with only 8 GB VRAM last week I ran Gemma 4 26B A4B a mixture of experts model on my RTX 4060 and hit 25–28 tokens/sec using llama.cpp's new MTP support. smooth. snappy. but MoE has a secret: it only activates 4B parameters per token despite having 26B total. that's why it flies. so the real question started haunting me. what if I throw a full, no tricks, every parameter fires on every token, 31B DENSE model at the same machine? # Hardware: GPU: NVIDIA RTX 4060, 8 GB VRAM RAM: 16 GB CPU: Intel Core i7 H Laptop. Gaming. Modest. The model: gemma-4-31B-it-qat-UD-Q4_K_XL.gguf (model's unsloth huggingface link in the comments) This is Google DeepMind's flagship dense model in the Gemma 4 family that can run on single consumer GPU. It packs a hybrid attention architecture, supports up to 256K context natively, and is QAT (Quantization Aware Training) optimized, meaning it retains far more quality than standard post training quants at the same bit depth. This is NOT the MoE. This is 31 BILLION dense parameters, every single one of them loaded. # the flags I used: -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -cnv --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 8 --spec-draft-p-min 0.6 -c 6000 -v Multi Token Prediction (MTP) is still active here. Separate draft GGUF required, same as the 26B setup. # Results: → Decode: ~3 tokens/sec → Prefill: ~2 tokens/sec → Context: 6000 tokens → Hardware crying quietly in the corner: yes so is 3 tps actually usable? For real time back and forth chat? Not ideal. You're not having a fluid conversation at 3 tps. but slow ≠ useless. And this is where it gets genuinely interesting. think about how senior devs actually work in a real team. But when something is architectural, deeply complex, or needs serious reasoning? they walk down the hall and escalate to the senior. That's exactly the local AI agent architecture this unlocks: → Fast orchestrator model (Gemma 4 26B MoE at 25+ tps) handles routing, simple queries, tool calls, memory. The junior dev. → Gemma 4 31B dense is the senior, called only when the fast model genuinely hits a wall. Hard multi step reasoning. Complex code generation. Deep architectural decisions. The agentic loop stays fast. Only the hard hops touch the 31B. That's a legitimate production grade local AI architecture on a budget hardware. (requires 2 8gb gpus) other workflows where 3 tps is completely fine: - overnight batch jobs. summarize documents, extract structured data, review code. Fire it off. Sleep. wake up to results. - One shot deep reasoning - Silent code audit loops, you write and test, the 31B reviews diffs and flags issues in the background between your sprints - Any workflow where output quality > output speed A few weeks ago, nobody was running a 30B+ dense model on a single consumer GPU with 8 GB VRAM. At all. Now we're doing it on an Intel i7-H gaming laptop with a NVIDIA RTX 4060, thanks to llama.cpp + QAT quants + MTP speculative drafting. Google DeepMind said the Gemma 4 31B targets "consumer GPUs and workstations." They were not exaggerating. The hardware bar to run serious frontier class models locally keeps dropping. the tools are here. the models are here. you just have to be willing to abuse your laptop a little. what workflows would you actually run on a local 3 tps 31B dense model? genuinely curious. drop it below.
0:18

my 8 GB VRAM gaming laptop is absolutely going to hate me for this. but I still did it. ran a 31b dense model (Gemma 4 31b Q4) with only 8 GB VRAM last week I ran Gemma 4 26B A4B a mixture of experts model on my RTX 4060 and hit 25–28 tokens/sec using llama.cpp's new MTP support. smooth. snappy. but MoE has a secret: it only activates 4B parameters per token despite having 26B total. that's why it flies. so the real question started haunting me. what if I throw a full, no tricks, every parameter fires on every token, 31B DENSE model at the same machine? # Hardware: GPU: NVIDIA RTX 4060, 8 GB VRAM RAM: 16 GB CPU: Intel Core i7 H Laptop. Gaming. Modest. The model: gemma-4-31B-it-qat-UD-Q4_K_XL.gguf (model's unsloth huggingface link in the comments) This is Google DeepMind's flagship dense model in the Gemma 4 family that can run on single consumer GPU. It packs a hybrid attention architecture, supports up to 256K context natively, and is QAT (Quantization Aware Training) optimized, meaning it retains far more quality than standard post training quants at the same bit depth. This is NOT the MoE. This is 31 BILLION dense parameters, every single one of them loaded. # the flags I used: -m gemma-4-31B-it-qat-UD-Q4_K_XL.gguf -cnv --spec-type draft-mtp --spec-draft-model mtp-gemma-4-31B-it.gguf --spec-draft-n-max 8 --spec-draft-p-min 0.6 -c 6000 -v Multi Token Prediction (MTP) is still active here. Separate draft GGUF required, same as the 26B setup. # Results: → Decode: ~3 tokens/sec → Prefill: ~2 tokens/sec → Context: 6000 tokens → Hardware crying quietly in the corner: yes so is 3 tps actually usable? For real time back and forth chat? Not ideal. You're not having a fluid conversation at 3 tps. but slow ≠ useless. And this is where it gets genuinely interesting. think about how senior devs actually work in a real team. But when something is architectural, deeply complex, or needs serious reasoning? they walk down the hall and escalate to the senior. That's exactly the local AI agent architecture this unlocks: → Fast orchestrator model (Gemma 4 26B MoE at 25+ tps) handles routing, simple queries, tool calls, memory. The junior dev. → Gemma 4 31B dense is the senior, called only when the fast model genuinely hits a wall. Hard multi step reasoning. Complex code generation. Deep architectural decisions. The agentic loop stays fast. Only the hard hops touch the 31B. That's a legitimate production grade local AI architecture on a budget hardware. (requires 2 8gb gpus) other workflows where 3 tps is completely fine: - overnight batch jobs. summarize documents, extract structured data, review code. Fire it off. Sleep. wake up to results. - One shot deep reasoning - Silent code audit loops, you write and test, the 31B reviews diffs and flags issues in the background between your sprints - Any workflow where output quality > output speed A few weeks ago, nobody was running a 30B+ dense model on a single consumer GPU with 8 GB VRAM. At all. Now we're doing it on an Intel i7-H gaming laptop with a NVIDIA RTX 4060, thanks to llama.cpp + QAT quants + MTP speculative drafting. Google DeepMind said the Gemma 4 31B targets "consumer GPUs and workstations." They were not exaggerating. The hardware bar to run serious frontier class models locally keeps dropping. the tools are here. the models are here. you just have to be willing to abuse your laptop a little. what workflows would you actually run on a local 3 tps 31B dense model? genuinely curious. drop it below.

Alok

63,583 görüntüleme • 1 ay önce

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:
0:11

We’re excited to introduce ShinkaEvolve: An open-source framework that evolves programs for scientific discovery with unprecedented sample-efficiency. Blog: Code: Like AlphaEvolve and its variants, our framework leverages LLMs to find state-of-the-art solutions to complex problems, but using orders of magnitude fewer resources! Many evolutionary AI systems are powerful but act like brute-force engines, burning thousands of samples to find good solutions. This makes discovery slow and expensive. We took inspiration from the efficiency of nature. ‘Shinka’ (進化) is Japanese for evolution, and we designed our system to be just as resourceful. On the classic circle packing optimization problem, ShinkaEvolve discovered a new state-of-the-art solution using only 150 samples. This is a big leap in efficiency compared to previous methods that required thousands of evaluations. We applied ShinkaEvolve to a diverse set of hard problems with real-world applications: 1/ AIME Math Reasoning: It evolved sophisticated agentic scaffolds that significantly outperform strong baselines, discovering an entire Pareto frontier of solutions trading performance for efficiency. 2/ Competitive Programming: On ALE-Bench (a benchmark for NP-Hard optimization problems), ShinkaEvolve took the best existing agent's solutions and improved them, turning a 5th place solution on one task into a 2nd place leaderboard rank in a competitive programming competition. 3/ LLM Training: We even turned ShinkaEvolve inward to improve LLMs themselves. It tackled the open challenge of designing load balancing losses for Mixture-of-Experts (MoE) models. It discovered a novel loss function that leads to better expert specialization and consistently improves model performance and perplexity. ShinkaEvolve achieves its remarkable sample-efficiency through three key innovations that work together: (1) an adaptive parent sampling strategy to balance exploration and exploitation, (2) novelty-based rejection filtering to avoid redundant work, and (3) a bandit-based LLM ensemble that dynamically picks the best model for the job. By making ShinkaEvolve open-source and highly sample-efficient, our goal is to democratize access to advanced, open-ended discovery tools. Our vision for ShinkaEvolve is to be an easy-to-use companion tool to help scientists and engineers with their daily work. We believe that building more efficient, nature-inspired systems is key to unlocking the future of AI-driven scientific research. We are excited to see what the community builds with it! Learn more in our technical report:

Sakana AI

359,537 görüntüleme • 10 ay önce