Загрузка видео...

Не удалось загрузить видео

На главную

llama3 8B (not quantized) running on an heterogeneous home cluster made of: - iPhone 15 Pro Max - iPad Pro (not sure which version XD) - MacBook Pro ( M1 Max ) - NVIDIA GeForce 3080 (not visible in video) - 2x NVIDIA Titan X Pascal Very soon also...

evilsocket's profile picture

Simone Margaritelli

47,842 subscribers

304,072 просмотров • 2 лет назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 9

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

I also want to take this chance to announce that I'm about to open a Discord where communities from different projects on my github can meet and possibly start some interesting knowledge exchange, so stay tuned!

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

Just pushed a couple of commits that entirely decoupled the Cake system from the specific model. This in theory can now be used with any generative model ... nice things ahead

Фото профиля Alberto Fuentes (e/acc)
Alberto Fuentes (e/acc)2 лет назад

evilsocket doing AI inference code wasnt in my 2024 prediction list; the AI community earned one of the best in cyber back in the days 🙌🙌

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

thank you! still here trying to work on new stuff :D

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

If anybody wants to help, the iOS app source code is available here -> it's a super simple app (the logic is implemented in rust elsewhere, the app is just a launcher), it would be nice to make it prettier and possibly show in the main view what the library is logging to stdout ... it should be simple enough, i just don't know swift/swiftui :D

Фото профиля Yohei
Yohei2 лет назад

This is a cool project! Curious to see where you take it.

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

thank you!

Фото профиля ColorCoded (Road to Broke Arc)
ColorCoded (Road to Broke Arc)2 лет назад

I have no idea what any of those words mean but I shared

Фото профиля Simone Margaritelli
Simone Margaritelli2 лет назад

appreciate the sentiment buddy!

Похожие видео

There is not a single door of any car in the world that can open itself without hitting the car next to it in a tight space like this. Tesla strives to be the BEST across every aspect of a car, and thus the Model X has the best door of any door. 🚪⚡️ Elon Musk $TSLA
0:23

There is not a single door of any car in the world that can open itself without hitting the car next to it in a tight space like this. Tesla strives to be the BEST across every aspect of a car, and thus the Model X has the best door of any door. 🚪⚡️ Elon Musk $TSLA

Teslaconomics

5,061,746 просмотров • 3 лет назад

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.
2:30

Nvidia has just announced Alpamayo 2 Super, an open 34 billion parameter reasoning vision-language-action model designed to accelerate the development of autonomous vehicles. This new model combines the NVIDIA Cosmos 3 Super reasoning model with a 2 billion parameter diffusion-based action expert model, and is post trained with reinforcement learning. The model can return multiple outputs: future trajectory plans, reasoning traces, grounded answers to questions about the scenes, and auto label generation. The model weights are now available for anyone to download on Hugging Face, and the inference code has been posted to GitHub. Distilled models can be deployed commercially without any further permission from Nvidia, and model outputs carry no license conditions. Automakers can distill down a compact version of this model that can run on the Nvidia computer in the car. Major kudos to Nvidia and Jensen Huang for advancing the state of the industry by releasing this as an open model with permissive licensing. Jensen isn't just paying lip service to the idea of open models, Nvidia is actually contributing to the ecosystem — and it's great for their business, because it helps sell more Thor computers that go in the car. Anyone can go download the model and play with it. If you do, let me know what you think. Personally I think it's so cool that we have open weights models that are this advanced, for anyone to download.

Whole Mars Catalog

45,595 просмотров • 1 месяц назад

I would have slapped that sticker on every MacBook visible on display, regardless of whether it was a pro, non-pro, Air, or any other model. Then, I would have kept one MacBook and returned the rest, making a profit of $10,000 to $15,000. One of my friends had this opportunity and went straight to the appliance section, the TV section, and even the gift cards. He had shown that person how to get it. What would you do if you got in this situation? Video produced by: thatwasepic
1:28

I would have slapped that sticker on every MacBook visible on display, regardless of whether it was a pro, non-pro, Air, or any other model. Then, I would have kept one MacBook and returned the rest, making a profit of $10,000 to $15,000. One of my friends had this opportunity and went straight to the appliance section, the TV section, and even the gift cards. He had shown that person how to get it. What would you do if you got in this situation? Video produced by: thatwasepic

GABRIEL 🪽

184,310 просмотров • 4 месяцев назад

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla
2:09

ELON: TESLA FUTURE'S HINGES ON THE AI5 CHIP PROJECT "The AI5 chip, which is going to be a great chip, all of Tesla hinges on that chip. That's the chip that is going into our next generation of self-driving cars, and it's also essential for the Optimus robot. So that chip program was in bad shape. It wasn't closing because it's quite an ambitious chip design, and it really wasn't on a path to success. And then we also had the Dojo program, which was doing okay but not on a path to be competitive with NVIDIA. So I collapsed the two programs into just one program, which is to get everyone focused on the AI5 chip, which is essential. Continue to use NVIDIA for training, but we need the AI5 inference chip, which is a very powerful chip, but it's also a very low-power chip. So it doesn't use a lot of power. Its performance per watt is extremely good. I think it's probably going to be at least 2 to 4 times better than NVIDIA performance per watt at the inference level in the car and the robot. And I don't know, 10% of the cost of an NVIDIA chip, something like that. So these are very important numbers to achieve." Source: Elon Musk, Baron Capital, Ron Baron, Tesla

Mario Nawfal

225,142 просмотров • 9 месяцев назад

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!
0:56

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,622 просмотров • 2 лет назад

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.
1:24

NVIDIA just released a new open source transcription model, Nemotron Speech ASR, designed from the ground up for low-latency use cases like voice agents. Here's a voice agent built with this new model. 24ms transcription finalization and total voice-to-voice inference time under 500ms. This agent actually uses *three* NVIDIA open source models: - Nemotron Speech ASR - Nemotron 3 Nano 30GB in a 4-bit quant (released in December) - A preview checkpoint of the upcoming Magpie text-to-speech model These models are all truly open source: weights, training data, training code, and inference code. This is a big deal! Jensen said in the CES keynote yesterday that he expects open source models to catch up to proprietary models this year in a number of categories. NVIDIA is putting their weight behind making this happen. (As Alan Kay said, the best way to predict the future is to invent it.) The code for this agent is open source too, of course. You can deploy it to production with Modal and Pipecat AI cloud, or run locally on an NVIDIA DGX Spark or RTX 5090.

kwindla

274,679 просмотров • 8 месяцев назад

Mark Zuckerberg says he is not an AI researcher -- so here is the one number he obsesses over to win the superintelligence race: "I'm not an AI researcher. So it's not like I'm in there telling them what research ideas to do." "The things that I can do as CEO of the company are: I can make sure that we have the very best people and talent density. I can make sure that we have by far the highest compute per researcher, and that we do whatever it takes to go build out all that capacity." "We're building this Prometheus cluster, which is going to be the first gigawatt-plus single cluster, a contiguous cluster for training, coming online next year." "We're building a cluster in Louisiana that is going to be 5 gigawatt, and we're building several more clusters like that that are just going to be multiple gigawatts." "We're talking about many hundreds of billions of dollars of capital. So you both need to have a good business model that can support it, and you need to believe in it." The tell: the richest lab in the world is not optimizing headcount. It is optimizing compute-per-researcher, and betting hundreds of billions that whoever owns the biggest contiguous cluster owns the frontier. Talent gets you into the room. Watts decide who leaves with the model.
1:06

Mark Zuckerberg says he is not an AI researcher -- so here is the one number he obsesses over to win the superintelligence race: "I'm not an AI researcher. So it's not like I'm in there telling them what research ideas to do." "The things that I can do as CEO of the company are: I can make sure that we have the very best people and talent density. I can make sure that we have by far the highest compute per researcher, and that we do whatever it takes to go build out all that capacity." "We're building this Prometheus cluster, which is going to be the first gigawatt-plus single cluster, a contiguous cluster for training, coming online next year." "We're building a cluster in Louisiana that is going to be 5 gigawatt, and we're building several more clusters like that that are just going to be multiple gigawatts." "We're talking about many hundreds of billions of dollars of capital. So you both need to have a good business model that can support it, and you need to believe in it." The tell: the richest lab in the world is not optimizing headcount. It is optimizing compute-per-researcher, and betting hundreds of billions that whoever owns the biggest contiguous cluster owns the frontier. Talent gets you into the room. Watts decide who leaves with the model.

Karl Mehta

255,587 просмотров • 2 месяцев назад

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!
0:39

Micron is going to $4,000 and once you understand what inference actually is, the number stops sounding crazy (Save this). Dylan Patel just said that by 2030, OpenAI and Anthropic alone will need over 100 gigawatts of compute combined and by 2040, we may not even be measuring AI infrastructure in gigawatts anymore. We may be talking about terawatts. Every single one of those gigawatts needs memory to function. Without it, the compute is worthless. Most people heard that and thought about Nvidia but they should be thinking about Micron. Every AI model generating a response has two phases. The first is prefill, processing your prompt which is compute-heavy and the second is decode generating each word one token at a time and that phase is almost entirely memory-bound, not compute-bound. During decode, the GPU's processing units sit idle more than 95% of the time, waiting for data to arrive from memory. Google confirmed it in a research paper that decode-phase bottlenecks are dominated by memory bandwidth and capacity not raw compute. The GPU is not the bottleneck but the memory feeding the GPU is. This matters because inference is now where all the money lives. Training a model happens once, Inference happens billions of times a day every ChatGPT response, every Claude output, every agentic workflow running in the background and every one of those token streams is a billing event tied directly to memory performance. Adding more GPUs does not fix this because GPUs are already underutilized in inference because they are sitting idle waiting on memory. Adding more memory bandwidth and capacity is what directly reduces token cost, reduces latency, and allows the same cluster to serve dramatically more users simultaneously. Longer context windows compound the problem further, a model running a 1 million token context window requires dramatically more memory per session than a 10,000 token window, and every new model generation pushes context longer. The market treats memory as a downstream beneficiary of Nvidia orders. The correct framework is the opposite, Micron is the upstream constraint on how much value every Nvidia GPU can actually generate at inference scale. Micron guided Q4 to $50 billion in revenue, has HBM4 ramping at twice the pace of the prior generation, and CEO Sanjay Mehrotra has said supply will not catch demand before the end of 2027. At 8x forward earnings on $112 projected FY2027 EPS, Micron is the most undervalued infrastructure company in the entire AI stack. Inference is memory. Memory is Micron and the inference ramp has barely started. Milk Road Pro members are already up massively on this position and we're just getting started. If you want the full breakdown of what we're buying and why, come join us for just a dollar using the link below!

Milk Road AI

128,678 просмотров • 2 месяцев назад

I just found a way to use Claude Code completely free. Claude Code is Anthropic’s terminal coding agent that usually needs a paid API key to run. But a viral GitHub project called Free Claude Code, now sitting at 37k stars, lets you skip the billing entirely. It works as a tiny proxy that sits between Claude Code and free model providers like NVIDIA NIM, OpenRouter, DeepSeek, and Gemini. Every request Claude Code would send to Anthropic gets quietly rerouted to a free model on the other side. The interface stays the same. What’s actually thinking behind it changes. Setup takes a couple of quick commands and a free API key from NVIDIA NIM or OpenRouter. Paste it into the local dashboard, pick a model, and you’re building. The catch is you’re not really using Claude anymore. But when the code ships, does the model badge matter? -- I have shared the Github Link in my free Whatsapp Community ↓
0:36

I just found a way to use Claude Code completely free. Claude Code is Anthropic’s terminal coding agent that usually needs a paid API key to run. But a viral GitHub project called Free Claude Code, now sitting at 37k stars, lets you skip the billing entirely. It works as a tiny proxy that sits between Claude Code and free model providers like NVIDIA NIM, OpenRouter, DeepSeek, and Gemini. Every request Claude Code would send to Anthropic gets quietly rerouted to a free model on the other side. The interface stays the same. What’s actually thinking behind it changes. Setup takes a couple of quick commands and a free API key from NVIDIA NIM or OpenRouter. Paste it into the local dashboard, pick a model, and you’re building. The catch is you’re not really using Claude anymore. But when the code ships, does the model badge matter? -- I have shared the Github Link in my free Whatsapp Community ↓

Vaibhav Sisinty

37,255 просмотров • 2 месяцев назад

INTEL JUST SHIPPED A WORKSTATION CARD WITH TWO GPUs ON ONE PCB AND 48GB OF VRAM. FOUR CARDS GIVE A SINGLE MOTHERBOARD 192GB OF POOLED INFERENCE MEMORY FOR THE PRICE OF TWO RTX 5090s 00:14 he holds one card up to the camera, two GPU dies side by side under the cooler, each one running its own x8 PCIe lane back to the chipset the cluster sees eight discrete accelerators in software. intel's Battle Matrix stack shards a model across all eight, so a 235B parameter network loads in slices and answers requests in parallel what 192GB of VRAM unlocks: an entire 200B class model in memory without quantization. a vision agent reading 100 invoices at once. a research box that holds three frontier models loaded simultaneously, switching between them in under a second intel is the slow side of inference. nvidia is faster per token, that is the honest tradeoff. but the only other path to this much VRAM is a $40,000 nvidia rack or three networked Mac Studios four B60 cards plus the chassis lands under $5,000. power draw averages 800 watts, $55 a month in electricity. one engineer paying $400 a month for combined ChatGPT Pro and Claude Code Max pays the hardware off in less than a year
1:16

INTEL JUST SHIPPED A WORKSTATION CARD WITH TWO GPUs ON ONE PCB AND 48GB OF VRAM. FOUR CARDS GIVE A SINGLE MOTHERBOARD 192GB OF POOLED INFERENCE MEMORY FOR THE PRICE OF TWO RTX 5090s 00:14 he holds one card up to the camera, two GPU dies side by side under the cooler, each one running its own x8 PCIe lane back to the chipset the cluster sees eight discrete accelerators in software. intel's Battle Matrix stack shards a model across all eight, so a 235B parameter network loads in slices and answers requests in parallel what 192GB of VRAM unlocks: an entire 200B class model in memory without quantization. a vision agent reading 100 invoices at once. a research box that holds three frontier models loaded simultaneously, switching between them in under a second intel is the slow side of inference. nvidia is faster per token, that is the honest tradeoff. but the only other path to this much VRAM is a $40,000 nvidia rack or three networked Mac Studios four B60 cards plus the chassis lands under $5,000. power draw averages 800 watts, $55 a month in electricity. one engineer paying $400 a month for combined ChatGPT Pro and Claude Code Max pays the hardware off in less than a year

NO1ennn

48,247 просмотров • 2 месяцев назад

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.
4:35

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,211 просмотров • 1 год назад

This is big. NVIDIA and Apple just unlocked the next level for Vision Pro with CloudXR. Here’s what you need to know: It streams from a PC or the cloud directly to your Vision Pro. No cables. Up to 4K at 120fps. The technology is called dynamic foveated streaming. Your eyes only see in full resolution at the center of your gaze. CloudXR tracks exactly where you’re looking and delivers maximum resolution there. Everything in your periphery gets optimized. The stream stays efficient without you ever noticing a difference. Why does this matter? Vision Pro is already the most advanced spatial computer ever made. But standalone processing has a ceiling. There are workflows that need more compute than any headset can carry. CloudXR removes that ceiling by connecting Vision Pro to the full power of NVIDIA RTX in real time. This is not a workaround. It is a native visionOS integration. Also worth knowing. Gaze data never leaves your device. Not to the app. Not to the server. Developers get the full performance benefit of foveation without ever touching your raw eye tracking data. Privacy built into the architecture. Three industries are already on this: Kia, Rivian, and Volvo are running 1:1 scale design reviews with photorealistic accuracy. Full size vehicle models evaluated in spatial computing before a physical prototype exists. That is a fundamental shift in how design works. Foxconn is walking factory floors digitally, optimizing facilities before construction begins. Switch is managing data center infrastructure through a full digital twin. Companies using this approach are reporting up to 30% improvement in development processes. And sim enthusiasts finally get to cut the cord. iRacing and X-Plane 12 are the first titles. Full GeForce RTX power, streamed wirelessly to Vision Pro, inside your physical space. For developers: one Xcode template, one codebase, deployed across Vision Pro, iPhone, and iPad. And multiple headsets can share the same streamed environment at once. Some fully immersed, others on a tablet. That collaborative layer is what enterprise has been asking for. Coming this spring with visionOS 26.4. It’s wild to think about what this unlocks. The future of spatial computing is incredibly bright. Live from GTC. More coming soon.
0:21

This is big. NVIDIA and Apple just unlocked the next level for Vision Pro with CloudXR. Here’s what you need to know: It streams from a PC or the cloud directly to your Vision Pro. No cables. Up to 4K at 120fps. The technology is called dynamic foveated streaming. Your eyes only see in full resolution at the center of your gaze. CloudXR tracks exactly where you’re looking and delivers maximum resolution there. Everything in your periphery gets optimized. The stream stays efficient without you ever noticing a difference. Why does this matter? Vision Pro is already the most advanced spatial computer ever made. But standalone processing has a ceiling. There are workflows that need more compute than any headset can carry. CloudXR removes that ceiling by connecting Vision Pro to the full power of NVIDIA RTX in real time. This is not a workaround. It is a native visionOS integration. Also worth knowing. Gaze data never leaves your device. Not to the app. Not to the server. Developers get the full performance benefit of foveation without ever touching your raw eye tracking data. Privacy built into the architecture. Three industries are already on this: Kia, Rivian, and Volvo are running 1:1 scale design reviews with photorealistic accuracy. Full size vehicle models evaluated in spatial computing before a physical prototype exists. That is a fundamental shift in how design works. Foxconn is walking factory floors digitally, optimizing facilities before construction begins. Switch is managing data center infrastructure through a full digital twin. Companies using this approach are reporting up to 30% improvement in development processes. And sim enthusiasts finally get to cut the cord. iRacing and X-Plane 12 are the first titles. Full GeForce RTX power, streamed wirelessly to Vision Pro, inside your physical space. For developers: one Xcode template, one codebase, deployed across Vision Pro, iPhone, and iPad. And multiple headsets can share the same streamed environment at once. Some fully immersed, others on a tablet. That collaborative layer is what enterprise has been asking for. Coming this spring with visionOS 26.4. It’s wild to think about what this unlocks. The future of spatial computing is incredibly bright. Live from GTC. More coming soon.

Justin Ryan

43,685 просмотров • 5 месяцев назад

Wow. Kling 2.6 can help create actual dread 🤯🔥 Watch my latest AI super short THE TENANT Using the latest Kling model I was able to not only craft an atmosphere but also prompt specific background sound, ambience, dialogue and inflections. While not perfect, this is one of the first times where a single model was able to consistently produce in the tone I was trying to maintain. Images and consistent characters were created in Nano Banana Pro inside Adobe Firefly, and video is 2.6. This was edited in Premiere Pro. Additional sound from ElevenLabs and Envato.
1:46

Wow. Kling 2.6 can help create actual dread 🤯🔥 Watch my latest AI super short THE TENANT Using the latest Kling model I was able to not only craft an atmosphere but also prompt specific background sound, ambience, dialogue and inflections. While not perfect, this is one of the first times where a single model was able to consistently produce in the tone I was trying to maintain. Images and consistent characters were created in Nano Banana Pro inside Adobe Firefly, and video is 2.6. This was edited in Premiere Pro. Additional sound from ElevenLabs and Envato.

Dave Clark

35,760 просмотров • 9 месяцев назад

Head of Nemotron Bryan Catanzaro says NVIDIA is sitting on internal "teacher" models that beat everything it has publicly shipped and he's trying to release these as well "This technology of using teacher models and multi-domain on-policy distillation has been really key for us in these ways." "Now, what does that mean? Well, it definitely means that inside NVIDIA we have a collection of teacher models that are individually better at different domains than any of the models that we've released." "And actually, I'm trying to release those teacher models as part of the Nemotron project as well -- because as you know, Nemotron is not just a model." "Nemotron is an open technology effort. It includes data sets, it includes techniques, all of our ideas about how to build a model, which we publish." "Of course it includes the models themselves, but I think it should also include collections of teacher models." "I'd imagine this could be a really great way of federating open technologies for AI across the ecosystem." _______ Wrote more on a prior Bryan interview here:
0:51

Head of Nemotron Bryan Catanzaro says NVIDIA is sitting on internal "teacher" models that beat everything it has publicly shipped and he's trying to release these as well "This technology of using teacher models and multi-domain on-policy distillation has been really key for us in these ways." "Now, what does that mean? Well, it definitely means that inside NVIDIA we have a collection of teacher models that are individually better at different domains than any of the models that we've released." "And actually, I'm trying to release those teacher models as part of the Nemotron project as well -- because as you know, Nemotron is not just a model." "Nemotron is an open technology effort. It includes data sets, it includes techniques, all of our ideas about how to build a model, which we publish." "Of course it includes the models themselves, but I think it should also include collections of teacher models." "I'd imagine this could be a really great way of federating open technologies for AI across the ecosystem." _______ Wrote more on a prior Bryan interview here:

Fireside Alpha

29,125 просмотров • 11 дней назад

Rivian CEO RJ Scaringe in new interview on self-driving: "Tesla's got a huge fleet. So our approach is we have to catch up with them and compete. And so we're going to have more cameras with better dynamic range. We're going to supplement that with LiDAR that provides better safety for edge case conditions and allows us to train the vision models faster. And the incremental cost to do that is relatively small. And in fact any of the incremental costs that would have been there on its own is offset by the fact that we brought inference in house and reduced the cost of our inference platform so dramatically from what we have in your car. Your car uses an Nvidia inference platform. I say all this because in like the infinite long-term you know, you could make the case that once the models are very very robust you could have less cameras or you may be able to get away with less radar. It's not clear yet if that's the case for covering all these corner cases."
3:06

Rivian CEO RJ Scaringe in new interview on self-driving: "Tesla's got a huge fleet. So our approach is we have to catch up with them and compete. And so we're going to have more cameras with better dynamic range. We're going to supplement that with LiDAR that provides better safety for edge case conditions and allows us to train the vision models faster. And the incremental cost to do that is relatively small. And in fact any of the incremental costs that would have been there on its own is offset by the fact that we brought inference in house and reduced the cost of our inference platform so dramatically from what we have in your car. Your car uses an Nvidia inference platform. I say all this because in like the infinite long-term you know, you could make the case that once the models are very very robust you could have less cameras or you may be able to get away with less radar. It's not clear yet if that's the case for covering all these corner cases."

Sawyer Merritt

449,694 просмотров • 5 месяцев назад

i'm running a 397 billion parameter model on a amd ai max box that sits on my desk and pulls less power than a gaming laptop. the model is Nex-N2-Pro, 397B-A17B, the open weight release people are putting next to gpt-5.5 on coding. i have it quantized to IQ1_M, 1.75 bits per weight, 90gb of weights loaded into the 128gb of unified memory on amd's strix halo igpu. watch the gpu in this recording. it spikes, it sustains, it does not fall over. that is the part the spec sheets never show you, not just that a 400b model loads, but that an integrated graphics chip holds the load and generates token after token, stable, no crash, no thermal cliff. and it is not a slideshow. roughly 18 tokens a second, faster than you can read. a frontier scale model producing usable output, fully local. no datacenter, no rented h100s, no api key, no permission. three years ago a model this size meant a server room and a budget to match. tonight it is a quiet box on my desk. this is the accessible tier almost nobody benchmarks honestly, and it is further along than the timeline thinks. the full breakdown is coming, rocm vs vulkan on this chip, and this little amd box head to head against the nvidia equivalent. stay tuned.
2:18

i'm running a 397 billion parameter model on a amd ai max box that sits on my desk and pulls less power than a gaming laptop. the model is Nex-N2-Pro, 397B-A17B, the open weight release people are putting next to gpt-5.5 on coding. i have it quantized to IQ1_M, 1.75 bits per weight, 90gb of weights loaded into the 128gb of unified memory on amd's strix halo igpu. watch the gpu in this recording. it spikes, it sustains, it does not fall over. that is the part the spec sheets never show you, not just that a 400b model loads, but that an integrated graphics chip holds the load and generates token after token, stable, no crash, no thermal cliff. and it is not a slideshow. roughly 18 tokens a second, faster than you can read. a frontier scale model producing usable output, fully local. no datacenter, no rented h100s, no api key, no permission. three years ago a model this size meant a server room and a budget to match. tonight it is a quiet box on my desk. this is the accessible tier almost nobody benchmarks honestly, and it is further along than the timeline thinks. the full breakdown is coming, rocm vs vulkan on this chip, and this little amd box head to head against the nvidia equivalent. stay tuned.

Sudo su

32,163 просмотров • 2 месяцев назад

nvidia allows you to use many llm models on their website. to use it programatically you can either create a phone verified nvidia nim account (heavily limited, often slow and NOT AVAILABLE IN TURKEY) or my wrapper app which bypasses this measures. > how it works? - it spawns and invisible safari webview of nvidia page - it solves the captcha using js (because page uses really soft version of hCaptcha) - because it's easy, it can create a pool of tokens - then spawns a local openai-anthropic compatible proxy server on port 2289 so you can use it with any tool, locally.
0:20

nvidia allows you to use many llm models on their website. to use it programatically you can either create a phone verified nvidia nim account (heavily limited, often slow and NOT AVAILABLE IN TURKEY) or my wrapper app which bypasses this measures. > how it works? - it spawns and invisible safari webview of nvidia page - it solves the captcha using js (because page uses really soft version of hCaptcha) - because it's easy, it can create a pool of tokens - then spawns a local openai-anthropic compatible proxy server on port 2289 so you can use it with any tool, locally.

Batuhan Katırcı

54,548 просмотров • 4 месяцев назад

This Bada app, which is still under development, enables file sharing between non-iOS phones. Whether you are using an international Samsung model or a China version Samsung model, it can share files with OPPO, vivo, Xiaomi, Honor, and even Huawei HarmonyOS devices. In essence, Bada temporarily gives Android and HarmonyOS devices that do not have native Quick Share support a Quick Share-like file sharing capability. I tested it myself. After installing Bada on the Huawei Pura 90 Pro Max through ZhuoyiTong, it was able to transfer files with a Samsung phone successfully. The demo below shows file sharing between the Samsung Galaxy S26 Ultra and the Huawei Pura 90 Pro Max. The process is very convenient. Just select photos or videos in the gallery, choose Bada the same way you would use Samsung sharing, then tap confirm on the Huawei phone to receive the files. Of course, you can also easily share files with OPPO, vivo, Xiaomi, Honor, and other Android phones as well.
0:23

This Bada app, which is still under development, enables file sharing between non-iOS phones. Whether you are using an international Samsung model or a China version Samsung model, it can share files with OPPO, vivo, Xiaomi, Honor, and even Huawei HarmonyOS devices. In essence, Bada temporarily gives Android and HarmonyOS devices that do not have native Quick Share support a Quick Share-like file sharing capability. I tested it myself. After installing Bada on the Huawei Pura 90 Pro Max through ZhuoyiTong, it was able to transfer files with a Samsung phone successfully. The demo below shows file sharing between the Samsung Galaxy S26 Ultra and the Huawei Pura 90 Pro Max. The process is very convenient. Just select photos or videos in the gallery, choose Bada the same way you would use Samsung sharing, then tap confirm on the Huawei phone to receive the files. Of course, you can also easily share files with OPPO, vivo, Xiaomi, Honor, and other Android phones as well.

Ice Universe

58,517 просмотров • 3 месяцев назад

Etched came out of stealth at $800M and by lunch X had NVIDIA in the ground We do this every few months. A chip launches, the deck says killer, the timeline holds a funeral, and NVIDIA closes green anyway Etched hardwires the transformer into silicon. That is where the speed comes from, nearly the whole die on one job instead of the ~30% a GPU uses. It is also the trap. The day that chip tapes out is the best it will ever be. You cannot patch it. You burned progress into a wafer and now pray the field stops moving NVIDIA made the opposite bet. Same board, faster every quarter in software. Dynamo is pulling more tokens per watt out of the same rack, on version 1.0 The depreciation risk the bears aimed at NVIDIA for two years does not live at NVIDIA. It lives here, on the chip built to bury it Etched is not a fraud. It is a niche tool priced like a general one, and $800M is not enough to run a frontier supply chain. The rest get bought on the next down cycle Bury the lead, not the leader. Full case with Jack Farley and Max Wiethe on MTS And special thanks for Baseten for the cool T Shirt! Chapters 00:00 Switching from bonds to semis 00:33 What Etched actually is 01:31 Faster and cheaper, but how much HBM 02:56 Maturing market, not an NVIDIA killer 05:01 $1B in contracts and a Taiwan factory 05:20 Why these startups all get absorbed 06:48 Tiered inference and the obsolescence trap 09:39 Etched vs TPUs and Trainium 12:17 Is the CUDA moat weakening 13:21 Co-design, squeezing every token per watt 14:37 NVIDIA is a software company that sells a chip 14:58 Who is NVIDIA's most dangerous competitor 16:23 The NVIDIA killers, ranked 18:19 A rich man's game 18:43 AMD's MI500 vs Rubin Ultra 20:19 The neocloud business decision 22:46 Lightning round, Rambus the toll on HBM 25:11 The CXL run-up on Astera, Marvell, Credo 26:22 Use AI less, go to the booth 28:10 EDA is not dead
30:42

Etched came out of stealth at $800M and by lunch X had NVIDIA in the ground We do this every few months. A chip launches, the deck says killer, the timeline holds a funeral, and NVIDIA closes green anyway Etched hardwires the transformer into silicon. That is where the speed comes from, nearly the whole die on one job instead of the ~30% a GPU uses. It is also the trap. The day that chip tapes out is the best it will ever be. You cannot patch it. You burned progress into a wafer and now pray the field stops moving NVIDIA made the opposite bet. Same board, faster every quarter in software. Dynamo is pulling more tokens per watt out of the same rack, on version 1.0 The depreciation risk the bears aimed at NVIDIA for two years does not live at NVIDIA. It lives here, on the chip built to bury it Etched is not a fraud. It is a niche tool priced like a general one, and $800M is not enough to run a frontier supply chain. The rest get bought on the next down cycle Bury the lead, not the leader. Full case with Jack Farley and Max Wiethe on MTS And special thanks for Baseten for the cool T Shirt! Chapters 00:00 Switching from bonds to semis 00:33 What Etched actually is 01:31 Faster and cheaper, but how much HBM 02:56 Maturing market, not an NVIDIA killer 05:01 $1B in contracts and a Taiwan factory 05:20 Why these startups all get absorbed 06:48 Tiered inference and the obsolescence trap 09:39 Etched vs TPUs and Trainium 12:17 Is the CUDA moat weakening 13:21 Co-design, squeezing every token per watt 14:37 NVIDIA is a software company that sells a chip 14:58 Who is NVIDIA's most dangerous competitor 16:23 The NVIDIA killers, ranked 18:19 A rich man's game 18:43 AMD's MI500 vs Rubin Ultra 20:19 The neocloud business decision 22:46 Lightning round, Rambus the toll on HBM 25:11 The CXL run-up on Astera, Marvell, Credo 26:22 Use AI less, go to the booth 28:10 EDA is not dead

Ben Pouladian

29,969 просмотров • 2 месяцев назад

This soldiering training is the most impressive and immersive I've ever tried. It is 10x better than a YouTube tutorial video. It really allowed me to see the procedure from all the points of view, and even get super close to get the details. It is a collaboration between @gracia_vr and Imperial College London: they recorded a soldering session with Gaussian Splatting Videos (4DGS), so that you can enjoy it from your VR headset. You can see the action happening in front of you; you can pause and re-watch what you need, change the point of view, get closer, get more distant. And the quality with which it has been shot is impressive: I enjoyed this experience with my DELL Pro Max Tower T2 with NVIDIA Pro RTX 6000, and I could really see all the small details of the PCB that was being soldered! I was really impressed by it. But I also noticed some drawbacks. First of all, some scenes have artifacts that make seeing the details of the PCB hard. I think when it comes to training involving small details, the capture and reproduction of the splat should be flawless. Then, as much as I loved it as a passive experience, I would have liked to have also some sort of practice session in VR. The power of VR is to let you learn by doing in full safety, and this kind of training does not exploit it. But maybe the best way to enjoy it would be in MR, where you have this training video close to a real workbench where you do the actual soldering while following the tutorial. Gaussian Splat can really revolutionize training: this kind of recording is much better than any flatscreen experience, and more accurate than any 3D CGI recostruction. I suggest you give it a go at this experience in the Gracia app (it's free). Then let me know your impressions! #VirtualReality #training #DellProPrecision #GaussianSplat #technology [Disclaimer: I'm a DELL Pro Precision Ambassador, and this is why I mentioned the model of my PC. I have been given a PC to do cool tests and share my results on social media. No monetary compensation or sales affiliation is part of the collaboration]
0:31

This soldiering training is the most impressive and immersive I've ever tried. It is 10x better than a YouTube tutorial video. It really allowed me to see the procedure from all the points of view, and even get super close to get the details. It is a collaboration between @gracia_vr and Imperial College London: they recorded a soldering session with Gaussian Splatting Videos (4DGS), so that you can enjoy it from your VR headset. You can see the action happening in front of you; you can pause and re-watch what you need, change the point of view, get closer, get more distant. And the quality with which it has been shot is impressive: I enjoyed this experience with my DELL Pro Max Tower T2 with NVIDIA Pro RTX 6000, and I could really see all the small details of the PCB that was being soldered! I was really impressed by it. But I also noticed some drawbacks. First of all, some scenes have artifacts that make seeing the details of the PCB hard. I think when it comes to training involving small details, the capture and reproduction of the splat should be flawless. Then, as much as I loved it as a passive experience, I would have liked to have also some sort of practice session in VR. The power of VR is to let you learn by doing in full safety, and this kind of training does not exploit it. But maybe the best way to enjoy it would be in MR, where you have this training video close to a real workbench where you do the actual soldering while following the tutorial. Gaussian Splat can really revolutionize training: this kind of recording is much better than any flatscreen experience, and more accurate than any 3D CGI recostruction. I suggest you give it a go at this experience in the Gracia app (it's free). Then let me know your impressions! #VirtualReality #training #DellProPrecision #GaussianSplat #technology [Disclaimer: I'm a DELL Pro Precision Ambassador, and this is why I mentioned the model of my PC. I have been given a PC to do cool tests and share my results on social media. No monetary compensation or sales affiliation is part of the collaboration]

TonyVT SkarredGhost

38,264 просмотров • 24 дней назад