Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Turn any world model from World Labs into a multi-user space for exploration and collaboration with FRAME. Voice chat, embodied AI agents, and tons more you can layer on top of the model foundation. This took all of 10 seconds to get going. The metaverse is so back. 😄

gabrieljbaker's profile picture

Gabriel Baker

2,807 subscribers

21,630 Aufrufe • vor 11 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):
2:36

Today we’re launching our first homegrown AI model: an open source turn detection model for building voice agents. Instead of relying solely on voice activity detection (VAD), which only considers when a user is speaking, our model also considers what has and is being said in the context of a conversation and predicts when a user is finished expressing their thoughts before the agent responds. Conversations with AI voice agents using this new model flow much more naturally without constant interruptions from the AI— check it out (more videos, details, and code in the thread):

LiveKit

126,941 Aufrufe • vor 1 Jahr

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.
0:40

Data is the foundation of any AI training. All of the GPUs in the world can't train an AI model if they don't have data to train it on. Don't forget: Grass is the data layer of AI.

Grass

211,991 Aufrufe • vor 2 Jahren

Dedalus Labs (Dedalus Labs) is building the infrastructure layer for agentic AI—the simplest way to build and deploy complex agents across any model and any tool. Congrats on the launch, Cathy Di & Windsor Nguyễn!
1:47

Dedalus Labs (Dedalus Labs) is building the infrastructure layer for agentic AI—the simplest way to build and deploy complex agents across any model and any tool. Congrats on the launch, Cathy Di & Windsor Nguyễn!

Y Combinator

138,607 Aufrufe • vor 10 Monaten

✨ I got [ 🌐 Make 3d world ] working now with World Labs 🏡 Interior AI is the first app to integrate World Labs groundbreaking (and still experimental) world model 😊 You can now redesign any part of your home and then instantly walk through it in 3d Now LIVE for everyone
0:52

✨ I got [ 🌐 Make 3d world ] working now with World Labs 🏡 Interior AI is the first app to integrate World Labs groundbreaking (and still experimental) world model 😊 You can now redesign any part of your home and then instantly walk through it in 3d Now LIVE for everyone

@levelsio

171,421 Aufrufe • vor 10 Monaten

Underrated use case for the new World Labs model: transforming existing spaces. I took a picture of the lobby at a16z NYC, turned it into a "neon nightclub" with Seedream Edit, and then loaded it into World Labs. Fun to step through a familiar space in a new style 😎
0:27

Underrated use case for the new World Labs model: transforming existing spaces. I took a picture of the lobby at a16z NYC, turned it into a "neon nightclub" with Seedream Edit, and then loaded it into World Labs. Fun to step through a familiar space in a new style 😎

Justine Moore

43,472 Aufrufe • vor 11 Monaten

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.
1:31

SITUATION EXPLAINED: A new generation of AI labs are betting that world models are the path to general intelligence that can interact with the real world. We had General Intuition CEO Pim de Witte on MTS to explain what world models are, and why intuitively world models offer a clearer path towards general intelligence than LLMs. For humans, text is a technology for compressing information about the unfolding, three-dimensional world around us and communicating it to other people. When LLMs reason about the world, they do so by reconstructing the world through text. World models work by modeling the three-dimensional world at the architecture level and then allowing agents to interact with that world. Unlike an LLM, which understands space and time through text, world models perform spatial and temporal reasoning natively. General Intuition hopes to use world models to create embodied AI—machines that can reason and act in the real world. But other companies are also pushing the frontier in this space. Odyssey ML today announced a multi-agent world model, Agora-1. Their launch video shows four participants interacting with a real-time world model simulation of GoldenEye, a first-person shooter for the N64. A playable preview of the GoldenEye simulation is available on their site.

MTS

11,676 Aufrufe • vor 3 Monaten

This is not for people who build World Models. I get an exclusive look inside one of the best AI labs in the world in Silicon Valley, Institute of Foundation Models, and talk with Hector Liu, Hector Liu, who leads World Model research there. These will be the “brains” of humanoid robots, and the key tech for all sorts of other things. I met with him to try to figure out how far away a truly generalized robot is. More on this lab and what they do at:
59:26

This is not for people who build World Models. I get an exclusive look inside one of the best AI labs in the world in Silicon Valley, Institute of Foundation Models, and talk with Hector Liu, Hector Liu, who leads World Model research there. These will be the “brains” of humanoid robots, and the key tech for all sorts of other things. I met with him to try to figure out how far away a truly generalized robot is. More on this lab and what they do at:

Robert Scoble

33,984 Aufrufe • vor 2 Monaten

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.
1:10

New release from Meta FAIR — Meta Motivo is a first-of-its-kind behavioral foundation model for controlling virtual physics-based humanoid agents for a wide range of complex whole-body tasks. The model is capable of expressing human-like behaviors and achieves performance competitive with task-specific methods and outperforms state-of-the-art unsupervised RL and model-based baselines. Try the demo ➡️ Get the model and code ➡️ We’re excited about how this research could pave the way for fully embodied agents, leading to more lifelike NPCs, democratization of character animation and new types of immersive experiences.

AI at Meta

129,239 Aufrufe • vor 1 Jahr

Real-time world model interaction meets space exploration. Star Junkers is a space scavenger RPG — a drifter girl, a loyal weasel, and a ship barely holding together. Explore uncharted planets, uncover hidden treasure, take on risky jobs, and trade your way to a better ship. The exploration is enhanced by a real-time world model, adding a layer of unpredictability to what you find. With an ever-expanding universe of UGC planets, there's always somewhere new to go. Built on Yoroll
3:59

Real-time world model interaction meets space exploration. Star Junkers is a space scavenger RPG — a drifter girl, a loyal weasel, and a ship barely holding together. Explore uncharted planets, uncover hidden treasure, take on risky jobs, and trade your way to a better ship. The exploration is enhanced by a real-time world model, adding a layer of unpredictability to what you find. With an ever-expanding universe of UGC planets, there's always somewhere new to go. Built on Yoroll

Yoroll

47,443 Aufrufe • vor 5 Monaten

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.
0:27

Google presents Genie Generative Interactive Environments introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future.

AK

684,372 Aufrufe • vor 2 Jahren

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang
2:08

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." Fei-Fei Li at Bloomberg Tech live with Emily Chang

a16z

112,217 Aufrufe • vor 2 Monaten

Bio-inspired swarm intelligence for AI music composition: MusicSwarm instantiates many identical, frozen foundation-model agents that coordinate only via peer-to-peer feedback and pheromone-like signals. Without any weight updates, these agents spontaneously self-organize into differentiated roles and produce compositions with higher local novelty, richer rhythmic diversity, and more human-like small-world structure than centrally critiqued multi-agent or single-shot baselines. We observe swarm dynamics that converge toward Nash-like equilibria in the space of agent behaviors, while the continual emergence of new motifs and long-range links realizes a Gödelian perspective: interacting agents plus a shared external world model behave as a meta-system whose creative trajectories go beyond those of any single, monolithic model.
0:35

Bio-inspired swarm intelligence for AI music composition: MusicSwarm instantiates many identical, frozen foundation-model agents that coordinate only via peer-to-peer feedback and pheromone-like signals. Without any weight updates, these agents spontaneously self-organize into differentiated roles and produce compositions with higher local novelty, richer rhythmic diversity, and more human-like small-world structure than centrally critiqued multi-agent or single-shot baselines. We observe swarm dynamics that converge toward Nash-like equilibria in the space of agent behaviors, while the continual emergence of new motifs and long-range links realizes a Gödelian perspective: interacting agents plus a shared external world model behave as a meta-system whose creative trajectories go beyond those of any single, monolithic model.

Markus J. Buehler

24,478 Aufrufe • vor 9 Monaten

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.
1:52

Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by Lightspeed, with participation from Amplify Partners, wndrco, Sky9 Capital, and FPV Ventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.

reactor

12,737,146 Aufrufe • vor 3 Monaten

Here’s some top tips on how to get the most points on Action Model 👾 Make sure to save this video to come back to later. This Action Model tutorial takes you through some of the many ways you can get LAM points and your stake in the community-owned AI model. Want even extra points? Join our discord and get more rewards while also building connections with like-minded people. Comment any questions you have, and happy AI training builders.
1:34

Here’s some top tips on how to get the most points on Action Model 👾 Make sure to save this video to come back to later. This Action Model tutorial takes you through some of the many ways you can get LAM points and your stake in the community-owned AI model. Want even extra points? Join our discord and get more rewards while also building connections with like-minded people. Comment any questions you have, and happy AI training builders.

Georgia | Action Model

11,261 Aufrufe • vor 1 Monat

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO
1:02

The World Model as NEO's Cognitive Core 1X has revealed a major AI development where the NEO humanoid can translate any natural language prompt into robotic action. It demonstrates this capability even for novel tasks, objects, and environments not found in its robot dataset. - the 1X World Model is trained on internet-scale human interaction videos and fine-tuned with robot data to ground its understanding in physics and in NEO's embodiment - from a simple voice or text prompt, the world model generates a visualization of future actions - a built-in inverse dynamics model then translates these into precise motor movements for NEO

The Humanoid Hub

68,453 Aufrufe • vor 7 Monaten

You can now step into any world as a video game ✨ All you need is an image edit model (this is Flux 2) + video model to animate (this is Veo 3.1). And you get to control the plot, characters, and aesthetic. Happy to do a longer tutorial if there's interest...
0:26

You can now step into any world as a video game ✨ All you need is an image edit model (this is Flux 2) + video model to animate (this is Veo 3.1). And you get to control the plot, characters, and aesthetic. Happy to do a longer tutorial if there's interest...

Justine Moore

175,278 Aufrufe • vor 9 Monaten

Introducing Project Eden, a world model research preview from VAST AI Research Project Eden is a persistent, multiplayer world model that fundamentally breaks from existing paradigms by decoupling the underlying world state from visual rendering. Instead of treating the world as a sequence of transient frames, Eden treats it as a structured, evolving environment that runs continuously, can be modified by user actions, and can be consistently observed from any viewpoint.
2:31

Introducing Project Eden, a world model research preview from VAST AI Research Project Eden is a persistent, multiplayer world model that fundamentally breaks from existing paradigms by decoupling the underlying world state from visual rendering. Instead of treating the world as a sequence of transient frames, Eden treats it as a structured, evolving environment that runs continuously, can be modified by user actions, and can be consistently observed from any viewpoint.

Tripo

3,379,575 Aufrufe • vor 3 Monaten

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.
1:29

AI for robotics has some monumental challenges, like high dimensionality and reasoning with the physical world. Ashok Elluswamy, AI lead at Tesla, thinks the hardest problem is Evaluation. - Loss isn't a perfect indicator of Policy Neural Network's quality. - Testing in the real world for all the long-tail failure cases is not possible. To solve the evaluation challenge, Tesla has developed a World Simulator Model. It is trained on state and action pairs, so for a given current camera feed and current action, the model generates the next frame in the cameras. Once the world simulation is good enough, it can be hooked up to the policy network for a closed-loop simulation. In this setup, the model generates the next video frames, and based on those, the policy network comes up with the next action, which feeds back into the world simulation; this continuous loop creates a coherent simulation. The world simulation neural network is not just applicable to self-driving but also to humanoids, because Tesla trains it on common data. The examples show Optimus navigating or manipulating inside the world simulation, with all the pixels being generated by the world simulation model. The robotic agents can be tested in an accurately reproduced edge case and in other potential variations of the same edge case.

The Humanoid Hub

17,065 Aufrufe • vor 6 Monaten

India is working on all 5 layers of the AI architecture: application, model, chip, infra and energy. We are building the foundation for global AI services. 📍 World Economic Forum, Davos
2:26

India is working on all 5 layers of the AI architecture: application, model, chip, infra and energy. We are building the foundation for global AI services. 📍 World Economic Forum, Davos

Ashwini Vaishnaw

693,131 Aufrufe • vor 7 Monaten