正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Turn any world model from World Labs into a multi-user space for exploration and collaboration with FRAME. Voice chat, embodied AI agents, and tons more you can layer on top of the model foundation. This took all of 10 seconds to get going. The metaverse is so back. 😄

Gabriel Baker

2,783 subscribers

21,608 次观看 • 9 个月前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

LiveKit

126,851 次观看 • 1 年前

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Grass

211,934 次观看 • 2 年前

Dedalus Labs (Dedalus Labs) is building the infrastructure layer for agentic AI—the simplest way to build and deploy complex agents across any model and any tool. Congrats on the launch, Cathy Di & Windsor Nguyễn!

Dedalus Labs (Dedalus Labs) is building the infrastructure layer for agentic AI—the simplest way to build and deploy complex agents across any model and any tool. Congrats on the launch, Cathy Di & Windsor Nguyễn!

Y Combinator

138,607 次观看 • 8 个月前

✨ I got [ 🌐 Make 3d world ] working now with World Labs 🏡 Interior AI is the first app to integrate World Labs groundbreaking (and still experimental) world model 😊 You can now redesign any part of your home and then instantly walk through it in 3d Now LIVE for everyone

✨ I got [ 🌐 Make 3d world ] working now with World Labs 🏡 Interior AI is the first app to integrate World Labs groundbreaking (and still experimental) world model 😊 You can now redesign any part of your home and then instantly walk through it in 3d Now LIVE for everyone

@levelsio

171,277 次观看 • 8 个月前

Underrated use case for the new World Labs model: transforming existing spaces. I took a picture of the lobby at a16z NYC, turned it into a "neon nightclub" with Seedream Edit, and then loaded it into World Labs. Fun to step through a familiar space in a new style 😎

Underrated use case for the new World Labs model: transforming existing spaces. I took a picture of the lobby at a16z NYC, turned it into a "neon nightclub" with Seedream Edit, and then loaded it into World Labs. Fun to step through a familiar space in a new style 😎

Justine Moore

43,472 次观看 • 9 个月前

This is not for people who build World Models. I get an exclusive look inside one of the best AI labs in the world in Silicon Valley, Institute of Foundation Models, and talk with Hector Liu, Hector Liu, who leads World Model research there. These will be the “brains” of humanoid robots, and the key tech for all sorts of other things. I met with him to try to figure out how far away a truly generalized robot is. More on this lab and what they do at:

This is not for people who build World Models. I get an exclusive look inside one of the best AI labs in the world in Silicon Valley, Institute of Foundation Models, and talk with Hector Liu, Hector Liu, who leads World Model research there. These will be the “brains” of humanoid robots, and the key tech for all sorts of other things. I met with him to try to figure out how far away a truly generalized robot is. More on this lab and what they do at:

Robert Scoble

28,188 次观看 • 6 天前

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

AI at Meta

129,055 次观看 • 1 年前

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,303 次观看 • 2 年前

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang

a16z

111,918 次观看 • 1 个月前

Bio-inspired swarm intelligence for AI music composition: MusicSwarm instantiates many identical, frozen foundation-model agents that coordinate only via peer-to-peer feedback and pheromone-like signals. Without any weight updates, these agents spontaneously self-organize into differentiated roles and produce compositions with higher local novelty, richer rhythmic diversity, and more human-like small-world structure than centrally critiqued multi-agent or single-shot baselines. We observe swarm dynamics that converge toward Nash-like equilibria in the space of agent behaviors, while the continual emergence of new motifs and long-range links realizes a Gödelian perspective: interacting agents plus a shared external world model behave as a meta-system whose creative trajectories go beyond those of any single, monolithic model.

Bio-inspired swarm intelligence for AI music composition: MusicSwarm instantiates many identical, frozen foundation-model agents that coordinate only via peer-to-peer feedback and pheromone-like signals. Without any weight updates, these agents spontaneously self-organize into differentiated roles and produce compositions with higher local novelty, richer rhythmic diversity, and more human-like small-world structure than centrally critiqued multi-agent or single-shot baselines. We observe swarm dynamics that converge toward Nash-like equilibria in the space of agent behaviors, while the continual emergence of new motifs and long-range links realizes a Gödelian perspective: interacting agents plus a shared external world model behave as a meta-system whose creative trajectories go beyond those of any single, monolithic model.

Markus J. Buehler

24,478 次观看 • 7 个月前

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.

reactor

12,733,152 次观看 • 1 个月前

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO

The Humanoid Hub

68,453 次观看 • 5 个月前

You can now step into any world as a video game ✨ All you need is an image edit model (this is Flux 2) + video model to animate (this is Veo 3.1). And you get to control the plot, characters, and aesthetic. Happy to do a longer tutorial if there's interest...

You can now step into any world as a video game ✨ All you need is an image edit model (this is Flux 2) + video model to animate (this is Veo 3.1). And you get to control the plot, characters, and aesthetic. Happy to do a longer tutorial if there's interest...

Justine Moore

175,278 次观看 • 7 个月前

Introducing Project Eden, a world model research preview from VAST AI Research Project Eden is a persistent, multiplayer world model that fundamentally breaks from existing paradigms by decoupling the underlying world state from visual rendering. Instead of treating the world as a sequence of transient frames, Eden treats it as a structured, evolving environment that runs continuously, can be modified by user actions, and can be consistently observed from any viewpoint.

Introducing Project Eden, a world model research preview from VAST AI Research Project Eden is a persistent, multiplayer world model that fundamentally breaks from existing paradigms by decoupling the underlying world state from visual rendering. Instead of treating the world as a sequence of transient frames, Eden treats it as a structured, evolving environment that runs continuously, can be modified by user actions, and can be consistently observed from any viewpoint.

Tripo

3,362,831 次观看 • 1 个月前

Ingest data from hundreds of sources into our multi-model database with the Fivetran connector for SurrealDB. There’s zero manual overhead and no code, so you can spend more time building your AI apps. Learn more. 👉

Ingest data from hundreds of sources into our multi-model database with the Fivetran connector for SurrealDB. There’s zero manual overhead and no code, so you can spend more time building your AI apps. Learn more. 👉

SurrealDB

123,992 次观看 • 1 年前

India is working on all 5 layers of the AI architecture: application, model, chip, infra and energy. We are building the foundation for global AI services. 📍 World Economic Forum, Davos

India is working on all 5 layers of the AI architecture: application, model, chip, infra and energy. We are building the foundation for global AI services. 📍 World Economic Forum, Davos

Ashwini Vaishnaw

692,652 次观看 • 5 个月前

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

16,591 次观看 • 5 个月前

We’re excited to introduce Taste Labs. Our mission is to end AI slop. We’re building the data and infrastructure layer to give AI models and agents taste. And today we’re coming out of stealth, announcing our $18.5M seed funding, co-led by CRV and Amplify Partners AI has nailed objective domains and made it easy to generate anything. But it still feels off. Now, the challenge is judgement. What fits, what feels like you, what’s GREAT. This requires turning a fuzzy, subjective domain into something we can measure and codify. We’re starting with design. There are two sides to cracking this, the foundation model layer and the agent layer: - We’ve already been working with the top frontier labs to evaluate and improve their models, crafting the right post-training data and RL environments. - We’ve also been working with app-layer companies to build the context and verification tools for their agents to produce better, more on-brand, more creative outputs. We want a future where AI feels right. If you’re passionate about this mission, join us!

Thais Castello Branco @ICML

2,648,000 次观看 • 23 天前

This tool shows which parts of the page will represent it in Google's AI search: Google takes bits and pieces of text from pages and assembles a small query-specific extractive summary for each grounding source and this forms a broader model context together with personalization, user prompt and any attached media. Apart from your branding this is all you have to influence the model.

This tool shows which parts of the page will represent it in Google's AI search: Google takes bits and pieces of text from pages and assembles a small query-specific extractive summary for each grounding source and this forms a broader model context together with personalization, user prompt and any attached media. Apart from your branding this is all you have to influence the model.

DEJAN

22,101 次观看 • 4 个月前