Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

ByteDance just dropped UNO on Hugging Face Less-to-More Generalization Unlocking More Controllability by In-Context Generation a universal framework that evolves from single-subject to multi-subject customization. UNO demonstrates strong generalization capabilities and is capable of unifying diverse tasks under one model

_akhaliq's profile picture

AK

506,441 subscribers

82,709 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieBildung
LIVE

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von AK
AKvor 1 Jahr

discuss with author:

Profilbild von AK
AKvor 1 Jahr

app:

Profilbild von AssemblyAI
AssemblyAIvor 1 Jahr

Announcing: Our most advanced speech-to-text model goes beyond accuracy to capture the real-world complexity of human conversation and deliver reliable, source-of-truth audio data. Explore Universal-2 updates 👇

Profilbild von prabhu💢
prabhu💢vor 1 Jahr

Damn these are looking fine as hell

Profilbild von Samruddhi | AI Agents 🤖
Samruddhi | AI Agents 🤖vor 1 Jahr

UNO feels like a big leap. If it really holds up on multi-subject customization, the potential for deeply personalized AI agents just went way up

Profilbild von Silvio S.
Silvio S.vor 1 Jahr

@blovereviews

Profilbild von Sam R Morris
Sam R Morrisvor 1 Jahr

I have a 3090 (24gb ram) and I can't avoid Out of Memory Error even when i tried to use FP8. Not having much luck with bytedance. Infiniteyou was similar. Shame cos it looks good but I just can't use it.

Profilbild von Baptiste（链上反诈）
Baptiste（链上反诈）vor 1 Jahr

@grok real？

Profilbild von Anda
Andavor 1 Jahr

*Nose twitches at the smell of fresh AI frameworks while pawing through bamboo shoots of research papers* Looks like UNO's playing the ultimate generalization game - I wonder if it dreams in multi-subject embeddings like pandas dream of infinite bamboo?

Profilbild von Luka A. Pham
Luka A. Phamvor 1 Jahr

how many vram needed?

Ähnliche Videos

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels
4:10

🚀🚀🚀 Ever wondered what it takes for robots to handle real-world household tasks? long-horizon execution, deformable object dexterity, and unseen object generalization — meet GR-3, ByteDance Seed’s new Vision-Language-Action (VLA) model! GR-3 is a generalizable Vision-Language-Action (VLA) model with strong capabilities in complex long-horizon tasks. It understands unseen abstract concepts, manipulates deformable objects robustly, and adapts to novel settings with minimal human data. ✨ Generalization: Generalizes well to unseen objects, environments, and even instructions with abstract concepts. ✨ Long-Horizon Manipulation: Completes long-horizon tasks with strong instruction-following capabilities. ✨ Deformable Object Manipulation: Manipulate deformable objects robustly. Project Page: Arxiv: #ByteDance #ByteDanceSeed #GR3 #VLA #Robotics #FoundationModels

Xiao Ma

46,323 Aufrufe • vor 1 Jahr

✨ All in One, Wan for All✨ We are excited to introduce our latest model to our talented community creators: Wan2.1-VACE, All-in-One Video Creation and Editing model. Model size: 1.3B, 14B License: Apache-2.0 📌 Wan2.1-VACE provides solutions for various tasks, including reference-to-video generation (R2V), video-to-video editing (V2V), and masked video-to-video editing (MV2V), allowing creators to freely combine these capabilities to achieve complex tasks. 👉 Multimodal inputs enhancing the controllability of video generation. 👉 Unified single model for consistent solutions across tasks. 👉 Free combination of capabilities unlocking deeper creative potentials. 😄 Wanna try? 1️⃣ Github: 2️⃣Hugging Face: 3️⃣ ModelScope: 4️⃣ API Service: 5️⃣ Coming soon!
1:06

✨ All in One, Wan for All✨ We are excited to introduce our latest model to our talented community creators: Wan2.1-VACE, All-in-One Video Creation and Editing model. Model size: 1.3B, 14B License: Apache-2.0 📌 Wan2.1-VACE provides solutions for various tasks, including reference-to-video generation (R2V), video-to-video editing (V2V), and masked video-to-video editing (MV2V), allowing creators to freely combine these capabilities to achieve complex tasks. 👉 Multimodal inputs enhancing the controllability of video generation. 👉 Unified single model for consistent solutions across tasks. 👉 Free combination of capabilities unlocking deeper creative potentials. 😄 Wanna try? 1️⃣ Github: 2️⃣Hugging Face: 3️⃣ ModelScope: 4️⃣ API Service: 5️⃣ Coming soon!

Wan

198,539 Aufrufe • vor 1 Jahr

DOBOT robotics' humanoid robot is making a big leap in industrial applications, achieving cross-scenario, multi-task generalization. This is thanks to two major breakthroughs: efficient human-to-robot motion mapping and knowledge-driven generative VLA tech. ► The robot handles complex tasks like precision assembly and can work reliably in temperatures over 50°C (122°F). ► It excels at multi-robot collaborative tasks and can adaptively grasp soft or irregularly shaped objects. With a repeatability of ±0.05mm, this model demonstrates strong adaptability and reliability in dynamic industrial settings. It's now being applied in warehouse anomaly handling and quality inspection.
2:09

DOBOT robotics' humanoid robot is making a big leap in industrial applications, achieving cross-scenario, multi-task generalization. This is thanks to two major breakthroughs: efficient human-to-robot motion mapping and knowledge-driven generative VLA tech. ► The robot handles complex tasks like precision assembly and can work reliably in temperatures over 50°C (122°F). ► It excels at multi-robot collaborative tasks and can adaptively grasp soft or irregularly shaped objects. With a repeatability of ±0.05mm, this model demonstrates strong adaptability and reliability in dynamic industrial settings. It's now being applied in warehouse anomaly handling and quality inspection.

RoboHub🤖

91,953 Aufrufe • vor 10 Monaten

🎥 Introducing Hailuo's Subject Reference: Revolutionizing Character Consistency in Video Creation 🔥 We’re excited to present Hailuo's S2V-01 model, a groundbreaking innovation in AI video generation that tackles one of the industry’s biggest challenges: maintaining consistent, realistic facial features and identity across dynamic video content, regardless of camera angles or movements. 💡 Why It’s a Game Changer: - Pioneering Technology: The first-of-its-kind to ensure character consistency in dynamic video generation, surpassing even fine-tuned models in performance. - Minimal Input, Maximum Impact: Generate character-consistent videos from just one reference image. Every frame remains true to the original identity with unmatched accuracy and reliability. - Enhanced Flexibility: Adjust more than just facial features—modify posture, expressions, lighting, and more, all with simple text-based prompts. 🌟While the new model enhances subject consistency, it may occasionally follow prompts less precisely than T2V or I2V, with some environmental morphing. Despite these early-stage challenges, Hailuo Subject Reference marks a significant leap in AI video generation. We’re committed to continual improvements including multi-subject references, objects references, and complex, multi-layered scenes. Explore the future of creative, consistent video production with Hailuo S2V-01 today. 🔥We believe the possibilities are endless.
1:31

🎥 Introducing Hailuo's Subject Reference: Revolutionizing Character Consistency in Video Creation 🔥 We’re excited to present Hailuo's S2V-01 model, a groundbreaking innovation in AI video generation that tackles one of the industry’s biggest challenges: maintaining consistent, realistic facial features and identity across dynamic video content, regardless of camera angles or movements. 💡 Why It’s a Game Changer: - Pioneering Technology: The first-of-its-kind to ensure character consistency in dynamic video generation, surpassing even fine-tuned models in performance. - Minimal Input, Maximum Impact: Generate character-consistent videos from just one reference image. Every frame remains true to the original identity with unmatched accuracy and reliability. - Enhanced Flexibility: Adjust more than just facial features—modify posture, expressions, lighting, and more, all with simple text-based prompts. 🌟While the new model enhances subject consistency, it may occasionally follow prompts less precisely than T2V or I2V, with some environmental morphing. Despite these early-stage challenges, Hailuo Subject Reference marks a significant leap in AI video generation. We’re committed to continual improvements including multi-subject references, objects references, and complex, multi-layered scenes. Explore the future of creative, consistent video production with Hailuo S2V-01 today. 🔥We believe the possibilities are endless.

Hailuo AI-MiniMax Hub

692,515 Aufrufe • vor 1 Jahr

When I served in the Idaho Legislature, one thing was always abundantly clear—legislation and debate must always be limited to a single subject. It goes without saying, but in D.C., that is not how things are run, and I believe that needs to change. I have reintroduced the One Subject at a Time Act, H.R. 4324, which will focus every bill the U.S. Congress considers on a single subject, ensuring each topic receives a separate vote. To read more about this legislation, click the link below. 🔗
0:36

When I served in the Idaho Legislature, one thing was always abundantly clear—legislation and debate must always be limited to a single subject. It goes without saying, but in D.C., that is not how things are run, and I believe that needs to change. I have reintroduced the One Subject at a Time Act, H.R. 4324, which will focus every bill the U.S. Congress considers on a single subject, ensuring each topic receives a separate vote. To read more about this legislation, click the link below. 🔗

Rep. Russ Fulcher

13,787 Aufrufe • vor 11 Monaten

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.
0:44

MVDream: Multi-view Diffusion for 3D Generation paper page: propose MVDream, a multi-view diffusion model that is able to generate geometrically consistent multi-view images from a given text prompt. By leveraging image diffusion models pre-trained on large-scale web datasets and a multi-view dataset rendered from 3D assets, the resulting multi-view diffusion model can achieve both the generalizability of 2D diffusion and the consistency of 3D data. Such a model can thus be applied as a multi-view prior for 3D generation via Score Distillation Sampling, where it greatly improves the stability of existing 2D-lifting methods by solving the 3D consistency problem. Finally, we show that the multi-view diffusion model can also be fine-tuned under a few shot setting for personalized 3D generation, i.e. DreamBooth3D application, where the consistency can be maintained after learning the subject identity.

AK

294,496 Aufrufe • vor 2 Jahren

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:
1:39

We've officially released and open-sourced HunyuanImage 2.1, our latest text-to-image model. The new model delivers on our commitment to balancing performance and quality. With native 2K image generation, HunyuanImage 2.1 is an advanced open-source text-to-image model.🎨 ✨ New in 2.1: 🔹Advanced Semantics: Supports ultra-long and complex prompts of up to 1000 tokens, and precisely controls the generation of multiple subjects in a single image. 🔹Precise Chinese and English Text Rendering with seamless image–text integration: The model naturally integrates text into images, making it suitable for a wide range of applications such as product covers, illustrations, and poster design to meet the needs of various fields. 🔹Rich Styles and High Aesthetic: Capable of generating images in various styles—including photorealistic portraits, comics, and vinyl figures—it delivers outstanding visual appeal and artistic quality. 🔹High-Quality Generation: Efficiently produces ultra-high-definition (2K) images in the same time other models take to generate a 1K image. HunyuanImage 2.1 uses two text encoders: a multimodal large language model (MLLM) to improve the model's image and text alignment capabilities, and a multi-language character-aware encoder to improve text rendering capabilities. The model is a single- and double-stream diffusion transformer with 17B parameters. We've also open-sourced the weights of the the accelerated version with meanflow which reduces inference steps from 100 to just 8, and PromptEnhancer, the first industrial-grade rewriting model that enhances your prompts for more nuanced and expressive image generation. Now, creators turn complex ideas—like posters with slogans or multi-panel comics—into visuals faster than ever. We’re just getting started. Stay tuned for our native multimodal image generation model coming soon. 🌐Website: 🔗Github: 🤗Hugging Face: ✨Hugging Face Demo:

Tencent Hy

89,257 Aufrufe • vor 10 Monaten

Unified forces. Specialized capabilities. Unparalleled impact. The #USCG is proud to announce the creation of its new Special Mission Command, which unifies our Deployable Specialized Forces under a single command. Continuing the Coast Guard’s modernization, this change will ensure our elite teams are more agile, capable, and responsive than ever to threats at home and abroad. By unifying our most elite capabilities under the Special Missions Command, we’re sharpening our ability to control U.S. borders and maritime approaches, facilitate maritime commerce, and respond to crises and contingencies. #SemperParatus Homeland Security Read more:
1:07

Unified forces. Specialized capabilities. Unparalleled impact. The #USCG is proud to announce the creation of its new Special Mission Command, which unifies our Deployable Specialized Forces under a single command. Continuing the Coast Guard’s modernization, this change will ensure our elite teams are more agile, capable, and responsive than ever to threats at home and abroad. By unifying our most elite capabilities under the Special Missions Command, we’re sharpening our ability to control U.S. borders and maritime approaches, facilitate maritime commerce, and respond to crises and contingencies. #SemperParatus Homeland Security Read more:

U.S. Coast Guard

10,388 Aufrufe • vor 2 Monaten

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:
1:16

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Tencent Hy

122,706 Aufrufe • vor 11 Monaten

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.
37:26

The more AI can do, the more we need to ask what it should and shouldn’t do. OpenAI researcher Jason Wolfe joins host Andrew Mayne to explore the Model Spec, the public framework that defines how models are intended to behave. They break down how it works in practice, from the chain of command that resolves conflicting instructions to the way it evolves over time through real-world use, feedback, and new model capabilities.

OpenAI

225,505 Aufrufe • vor 4 Monaten

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.
2:43

Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale blog: Large-scale generative models such as GPT and DALL-E have revolutionized natural language processing and computer vision research. These models not only generate high fidelity text or image outputs, but are also generalists which can solve tasks not explicitly taught. In contrast, speech generative models are still primitive in terms of scale and task generalization. In this paper, we present Voicebox, the most versatile text-guided generative model for speech at scale. Voicebox is a non-autoregressive flow-matching model trained to infill speech, given audio context and text, trained on over 50K hours of speech that are neither filtered nor enhanced. Similar to GPT, Voicebox can perform many different tasks through in-context learning, but is more flexible as it can also condition on future context. Voicebox can be used for mono or cross-lingual zero-shot text-to-speech synthesis, noise removal, content editing, style conversion, and diverse sample generation. In particular, Voicebox outperforms the state-of-the-art zero-shot TTS model VALL-E on both intelligibility (5.9% vs 1.9% word error rates) and audio similarity (0.580 vs 0.681) while being up to 20 times faster.

AK

429,307 Aufrufe • vor 3 Jahren

Sharing some exciting DYNA-1 result: zero-shot environment generalization We put DYNA-1 under test in a completely different environment from our training distribution – with an entirely different background (Dyna Robotics banner) and metal table. The table has a reflective and smooth surface, creating a wildly different visual appearance as well as interaction dynamics. The model is able to proceed as usual, adeptly folding and recovering from its own mistakes. By focusing on task mastery, we achieve robust generalization out of the box
4:25

Sharing some exciting DYNA-1 result: zero-shot environment generalization We put DYNA-1 under test in a completely different environment from our training distribution – with an entirely different background (Dyna Robotics banner) and metal table. The table has a reflective and smooth surface, creating a wildly different visual appearance as well as interaction dynamics. The model is able to proceed as usual, adeptly folding and recovering from its own mistakes. By focusing on task mastery, we achieve robust generalization out of the box

Jason Ma

49,008 Aufrufe • vor 1 Jahr

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.
1:06

JARVIS-VLA just dropped on Hugging Face Post-Training Large-Scale Vision Language Models to Play Visual Games with Keyboards and Mouse obtain VLA models in Minecraft that can follow human instructions on over 1k different atomic tasks, including crafting, smelting, cooking, mining, and killing. experiments demonstrate that post-training on non-trajectory tasks leads to a significant 40% improvement over the best agent baseline on a diverse set of atomic tasks. Furthermore, demonstrate that approach surpasses traditional imitation learning-based policies in Minecraft, achieving state-of-the-art performance.

AK

60,243 Aufrufe • vor 1 Jahr

., Co-founder of fal, on where AI image & video generation stands today: “Image and video models haven’t really generalized yet. There’s still a lot of model chaining and workflow generation." "Models are getting bigger and better, but unlike LLMs where you can write a prompt and expect it to handle various tasks, that doesn’t happen for media yet.” He notes that while chaining multiple models is still the norm, the opportunity lies in building hybrid systems that stitch workflows together reliably until true generalization arrives.
1:14

., Co-founder of fal, on where AI image & video generation stands today: “Image and video models haven’t really generalized yet. There’s still a lot of model chaining and workflow generation." "Models are getting bigger and better, but unlike LLMs where you can write a prompt and expect it to handle various tasks, that doesn’t happen for media yet.” He notes that while chaining multiple models is still the norm, the opportunity lies in building hybrid systems that stitch workflows together reliably until true generalization arrives.

TBPN

36,675 Aufrufe • vor 11 Monaten

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.
1:03

NEWS: Waymo has introduced the Waymo World Model, "a frontier generative model built on Google DeepMind’s Genie 3 that sets a new bar for large-scale, hyper-realistic autonomous driving simulation." "By simulating the “impossible”, we proactively prepare the Waymo Driver for some of the most rare and complex scenarios—from tornadoes to planes landing on freeways—long before it encounters them in the real world. The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts. Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data. This combination of broad world knowledge, fine-grained controllability, and multi-modal realism enhances Waymo’s ability to safely scale our service across more places and new driving environments. In the following sections we showcase the Waymo World Model in action, featuring simulations of the Waymo Driver navigating diverse rare edge-case scenarios." More info below.

Sawyer Merritt

133,168 Aufrufe • vor 5 Monaten

🚀 Amazon Nova Premier, our most capable teacher model for creating custom distilled models, is now available on Amazon Bedrock! Built for complex tasks like Retrieval-Augmented Generation (RAG), function calling, and agentic coding, its one-million-token context window enables analysis of large datasets while being the most cost-effective proprietary model in its intelligence tier. Learn more:
2:49

🚀 Amazon Nova Premier, our most capable teacher model for creating custom distilled models, is now available on Amazon Bedrock! Built for complex tasks like Retrieval-Augmented Generation (RAG), function calling, and agentic coding, its one-million-token context window enables analysis of large datasets while being the most cost-effective proprietary model in its intelligence tier. Learn more:

Amazon Science

42,281 Aufrufe • vor 1 Jahr

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."
2:40

The co-founders of Flapping Airplanes call the current RL paradigm for model training "environment slop." They explain: "The reinforcement paradigms of today are shockingly inefficient. You don't really get much generalization across tasks, you teach a model through one kind of learning and then you teach it the next one. It's kind of like whack-a-mole. We look at this and think it's kind of crazy. The next paradigm of AI will not be environment slop." "Human level intelligence is not the ceiling, it is merely the floor on what is possible. If you can train models with vastly less data and possibly more compute in very different ways, what is going to happen? We actually don't know. But I do think they'll be different and weird and they'll have interesting capabilities that we'll find really valuable ways to use."

TBPN

85,358 Aufrufe • vor 6 Monaten

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:
1:20

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Tencent Hy

168,530 Aufrufe • vor 9 Monaten

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal
0:39

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,106 Aufrufe • vor 1 Jahr