Загрузка видео...

Не удалось загрузить видео

На главную

Introducing FLUX-mimic, a next-generation Video-Action Model for general purpose dexterity, developed in partnership with Black Forest Labs. Late last year we published mimic-video and introduced Video-Action Models (VAM): a new family of robotics foundation models built on top of video generation models. We showed that robot control reduces to...

119,468 просмотров • 3 месяцев назад •via X (Twitter)

Комментарии: 26

Фото профиля mimic
mimic3 месяцев назад

Read our blogpost:

Фото профиля Elvis Nava
Elvis Nava3 месяцев назад

@bfl_ai Let's go!

Фото профиля Fabio Quattrini #ECCV26
Fabio Quattrini #ECCV263 месяцев назад

@bfl_ai 🦾🦾

Фото профиля Benedek Forrai
Benedek Forrai3 месяцев назад

@bfl_ai a very no-nonsense partnership that brings video models to life (and is euromaxxed)

Фото профиля Julian
Julian3 месяцев назад

@bfl_ai 🦾🚀🚀

Фото профиля Zeno
Zeno3 месяцев назад

@bfl_ai Cool stuff!

Фото профиля Irvin Dalaud
Irvin Dalaud3 месяцев назад

@_fracapuano @bfl_ai Now I’m really curious to know how they get accurate (enough) world dynamics representation with video models like FLUX (for robotics at least). JEPA makes a lot of sense to me but this makes me think twice. Any insights please @bfl_ai @elvisnavah ?

Фото профиля aless
aless3 месяцев назад

@bfl_ai 🤖🤖🤖

Фото профиля atharva ☆
atharva ☆3 месяцев назад

@bfl_ai wowowow

Фото профиля Nathan Benaich
Nathan Benaich3 месяцев назад

@chrisoffner3d @bfl_ai 🤩

Фото профиля William Lamkin
William Lamkin3 месяцев назад

@bfl_ai 🦾

Фото профиля Ana G
Ana G3 месяцев назад

@bfl_ai 🔥🔥🔥

Фото профиля Yann Kronberg
Yann Kronberg3 месяцев назад

Everyone is reacting to the dexterity but that is not the only shift. Fewer demos needed, running on one GPU and on premises is what changes who can train this. Factories without a teleop dataset just got unlocked. I'm curious about the 5% where the prediction is wrong and how fast do you catch it before the arm acts?"

Фото профиля Bblack
Bblack3 месяцев назад

@bfl_ai FLUX-mimic + robotics is insane 🤖 Turning video generation into real dexterity on a single GPU is the future

Фото профиля sujal
sujal3 месяцев назад

@bfl_ai lovely

Фото профиля Ella Tech & Tool
Ella Tech & Tool3 месяцев назад

@bfl_ai This is huge FLUX mimic real factory deployment? Robot learning just leveled up.

Фото профиля Lyss Sky | AI Artist
Lyss Sky | AI Artist3 месяцев назад

@bfl_ai 😍😍😍

Фото профиля SEAR
SEAR3 месяцев назад

@bfl_ai neat to see flux used for dexterity

Фото профиля Erwin
Erwin3 месяцев назад

@bfl_ai How come it's the other german car manufacturer that is exploring robotics outside of @BMW Definitely exciting tech tho!

Фото профиля Samian
Samian3 месяцев назад

@bfl_ai video-action models are still basically imitation learning with a video encoder slapped on top. not convinced that changes the core problem yet

Фото профиля 𝓛𝒶𝒷𝓎𝓇𝒾𝓃𝓉𝒽𝓒ℴ𝒹ℯ𝓇
𝓛𝒶𝒷𝓎𝓇𝒾𝓃𝓉𝒽𝓒ℴ𝒹ℯ𝓇3 месяцев назад

@bfl_ai 👀

Фото профиля clankr
clankr3 месяцев назад

@bfl_ai Starting to test this in factories is great. Deployment means more data, and more data means a better model... It is a cycle. This deployment proves that responsive models can be used instead of just basic radio models.

Фото профиля Abdel Latrache
Abdel Latrache3 месяцев назад

@bfl_ai Lets goo!

Фото профиля Porters Reserve
Porters Reserve3 месяцев назад

Strong work. The core thesis is correct — control reduces to visual prediction, and sample efficiency on real hardware is the actual bottleneck. Factory floors with known parts and controlled lighting are still the easy mode. We’re running the harder version: dense outdoor polyculture, living deformable tissue, constant occlusion, shifting light, wind, and dirt. Same VAM principle, different physics. If FLUX-mimic can hold up under those conditions with the same data efficiency, that’s the unlock for field systems beyond manufacturing. The Reserve is built as the filter. Grit over gloss.

Фото профиля homuraakemifan
homuraakemifan3 месяцев назад

@bfl_ai Surely in the world of AI video generation, nobody is going to fund a startup based on a snazzy video, they are going to want to dive deep into the technical details, or is it a serious case of "I WAS BORN TO LEAD NOT TO READ"

Фото профиля Lex
Lex3 месяцев назад

@bfl_ai Training robots through video-action models feels like the practical path for factory work. The part I’m most curious about is how fast teams can adapt it to a totally new task.

Похожие видео

Introducing FLUX 3 Action. An open weights 7B World Action Model that achieves first place on the RoboLab benchmark. It outperforms the previous best open model by 6.1 percentage points while using 56% fewer parameters and running up to 3.95x faster.⁠⁠ FLUX 3 Action removes the usual trade-off between world action model performance and VLA speed: it still predicts video and actions together, but plans more than twice as far ahead and runs faster per second of robot motion than the strongest open VLA. Teams can fine-tune FLUX 3 Action on their own demonstrations to create policies for a particular robot and task. Together with NVIDIA, we also integrated FLUX 3 Action natively into Hugging Face's LeRobot, with fine-tuning recipes included and edge deployment on NVIDIA Jetson. Beyond robotics, we’re also seeing promising results training task-specific policies for acting in simulated environments like gaming, controlling a vehicle, computer use, and wherever else a model needs to understand a visual environment and then choose what to do next. FLUX 3 Action builds on the same image, video, and audio pretraining as FLUX 3, but uses a smaller architecture designed for practical deployment. In midtraining, we trained the model to predict actions and future frames together. We’re releasing the weights, code, fine-tuning recipe, benchmarks, and reproducible examples so researchers and developers can build on the model with their own robots, environments, and tasks (see below).

Black Forest Labs

182,119 просмотров • 1 месяц назад

JUST IN: Dyna Robotics just published one of the most important research papers in robotics this year. It could fundamentally change how robot foundation models are trained. A scaling law that transfers from human video to robot performance. Dyna-2 is out and it's 🔥 Here's what that means in plain terms. Dyna-2 was pre-trained on ONE MILLION hours of egocentric human video, 170 years of continuous human experience, cooking, folding, assembling, cleaning. And as that human data scaled, robot performance improved. Predictably. Monotonically. Across 39 tasks on two different robot embodiments the model had never seen. → 1,000 hours pre-training → 20% normalised task performance → 10,000 hours → 28% → 100,000 hours → 45% → 1,000,000 hours → 53% Human video exists at effectively unlimited scale. Every cook, every factory worker, every craftsperson wearing a camera is generating training data for future robots. But the finding that stunned even the researchers, world modeling is what makes the transfer work. A model trained to predict future video AND actions massively outperforms one trained on actions alone. Video is the new scaling axis for robotics. One more jaw-dropping data point. 13 minutes of teleoperation data was enough to fine-tune Dyna-2 to open a bottle cap using two five-fingered robot hands. The robots are coming, and they're learning from us directly :D Read more here: Congrats Jason Ma and team! ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

23,681 просмотров • 2 месяцев назад

Excited to announce GR00T N1, the world’s first open foundation model for humanoid robots! We are on a mission to democratize Physical AI. The power of general robot brain, in the palm of your hand - with only 2B parameters, N1 learns from the most diverse physical action dataset ever compiled and punches above its weight: - Real humanoid teleoperation data. - Large-scale simulation data: we are open-sourcing 300K+ trajectories! - Neural trajectories: we apply SOTA video generation models to “hallucinate” new synthetic data that features accurate physics in pixels. Using Jensen’s words, “systematically infinite data”! - Latent actions: we develop novel algorithms to extract action tokens from in-the-wild human videos and neural generated videos. GR00T N1 is a single end-to-end neural net, from photons to actions: - Vision-Language Model (System 2) that interprets the physical world through vision and language instructions, enabling robots to reason about their environment and instructions, and plan the right actions. - Diffusion Transformer (System 1) that “renders” smooth and precise motor actions at 120 Hz, executing the latent plan made by System 2. We deploy N1 on GR1 robot, 1X Neo robot, and a large collection of simulation benchmarks. N1 achieves up to +30% boost in diverse manipulation tasks for household and industrial settings. While humanoid robots are the main focus of N1, our model also supports cross-embodiment. We finetune it to work on the $110 HuggingFace LeRobot SO100 robot arm! Open robot brain runs on open hardware. Sounds just right. Let’s solve robotics, together, one token at a time. Links to our Whitepaper, Github repo, HuggingFace model, and open dataset page in the thread: 🧵

Jim Fan

467,710 просмотров • 1 год назад