Loading video...
Video Failed to Load
Woow Nvidia has just released a 2.6B open-source world model 🔥 You can turn a single image, text prompt and trajectory into controllable worlds... And on a single GPU! - Code available on GitHub - Paper as well on arxiv You can use it for many things like embodied... show more
174,447 views • 4 months ago •via X (Twitter)
41 Comments
You can learn more about it here: - Project: - GitHub: - Paper: World models are evolving so fast! Soon we'll have a Genie 3 equivalent running at home.
lol. “Everyone” that has a 4000.00 video card…..
Still a consumer GPU. You can rent one online for like $0.40/hr.
@dennis_mcleod How?
@dennis_mcleod Using platforms like vast ai, runpod or novita
@amerukraine Does it have free api calls too 😬
Brand new, so maybe we'll have a cheap API soon if a neo cloud is hosting it.
Super clutch! Thank you !
Nvidia cooking as well in last days big W
They're doing a lot for the open source ecosystem!
Did the release the model? I cannot find it
@llmgram
wow
Interesting release. 2.6B params on single GPU is a meaningful efficiency benchmark for embodied AI research. Curious how the trajectory control generalizes across unseen environments.
Yes exactly. The efficiency is quite incredible. Usually you need powerful cloud capacities to run this kind of model.
@C4ndide Everyone who has 5 or 6 thousand dollars for a GPU
Not for most
Nano banana + this is probably goated
At some pointpoint NVIDIA will go back to targeting prosumers. If the can open source mini models, they could give the big dogs something to worry about
Crazy how these models getting more accessible each day
It looks awesome!
This is great. 2.6B can even run on a 250 dollar 8GB VRAM card
Another day, another world model I'll probably spend a weekend failing to integrate into something useful. My GPU is ready for the abuse. One tiny project closer.
Nothing is released. Where are the Models? Inference Code?
That’s huge
Will it run on RTX 4090?
The models aren’t released yet 😢
This is truly groundbreaking news from Nvidia. Releasing a 2.6B open-source world model that can run efficiently on a single GPU is a massive leap forward for democratization in AI. The potential applications for robotics, immersive simulations, and interactive environments are absolutely immense. Thank you for sharing this exciting update!
This is wild. Single GPU world models just became real
everyone? bro, no one I know has a h100 just sitting around
Running an open-source world model on a single GPU is a huge accessibility moment for AI research
Road to VR immersion
Holodeck confirmed
Image plus text prompt plus trajectory into a controllable world is a sentence that would have sounded like sci-fi two years ago
@machinelearnflx So cool 😎
are you a shill ... "things like embodied AI and robotics research" .... did you ever do any robotic research dude .. no.. you just repeat like a parrot
Mesmerized
Magic time
@grok
This is amazing!
Controllable world generation from a single image on one GPU being open-source removes the biggest simulation bottleneck for robot training. What's the visual fidelity ceiling?
![We made a thing! Very happy to announce sqlcoder-pro and the Defog Alignment Platform. Available to use immediately without a wait-list, weights will be open-sourced very soon. The video does a quick show and tell comparison against ChatGPT (with gpt-4o). Read on for more details! TLDR 💪 equal (or better) performance on text-to-SQL as the most capable Claude-3.5 or GPT-4 models 🤝 You can use it today on a free plan/free trial, without a waitlist 🪽 self-hostable on a single RTX4090, with 2 second median generation times for SQL queries 🔁 exactly the same output every time, give the same prompt 👨🏻🏫 teachable and steerable: show the model what you want it to do 🛞 debuggable – you can understand WTF is going on inside the model, instead of treating it like a black box Let's dig into each of these one-by-one! Performance SQLCoder-8b-pro significantly exceeds the performance of our previous sqlcoder-8b model on Postgres text-to-SQL (from 88.2% to 90.2% accuracy - gpt-4o is at 87.6%, for reference). It is also better at following instructions. This was done via self-merges, hand crafted fine-tuning data, and adapting the training data to fit our tokenizer. Cost You can host this on the model on a single $3,500 RTX4090, and support ~5 requests/second via VLLM. If you're looking to host on the cloud instead, you can run it on a single L4 GPU that costs $300/mo on GCP Repeatability We have a dense 8b model with no MoE shenanigans. For the same prompt with temperature=0, you'll always get the same answer – which is critical in BI. Teachable In our alignment and feedback modes, you can give the model feedback on how it answered certain questions, and it will automatically adapt to the feedback. Debuggable You can use logprobs and attention scores to determine where, exactly is the model paying attention to inside a prompt + what it's getting confused by when generating outputs. Available today You can use Defog on the cloud today by going to docs[dot]defog[dot]ai, and getting an API key. Excited to hear what you think!](https://image.24vids.com/tw-1825943270935113842/ext_tw_video_thumb/1825942610525220864/pu/img/qmpqqvd_3KNdJDGd.jpg)
