Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from...

JasonSCui's profile picture

Jason Cui

4,454 subscribers

36,205 Aufrufe • vor 2 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New short course Multimodal RAG: Chat with Videos, developed with Intel and taught by vasudevlal! In this course, you’ll work with LLaVA (Large Language and Vision Assistant), a Large Vision Language Model (LVLM) that can process both images and text. For example, given an image of a person doing a handstand on a skateboard at the beach, LLaVA doesn't just caption the scene, it’s able to predict possible outcomes, like the person losing balance or falling off. By understanding not just what's in a video frame, but what might happen next, your application can provide more insightful answers to questions about video. You'll build a full multimodal RAG pipeline that can chat about video content: - Use the BridgeTower model to create joint text-image embeddings in a 512-dimensional multimodal semantic space. - Learn video processing techniques to extract keyframes, generate transcripts using Whisper, and create captions. - Use the LanceDB vector database to store and retrieve high-dimensional multimodal embeddings. - Integrate the LLaVA model, combining CLIP's (Contrastive Language Image Pretraining) vision transformer with Llama, for advanced visual-textual reasoning. Your final system will ingest video data, generate embeddings for frames and text, perform similarity searches for relevant content, and use the retrieved multimodal context to inform LVLM-based response generation. The result is a system capable of answering nuanced questions about video content, effectively chatting about the video it has processed. Please sign up here!
2:40

New short course Multimodal RAG: Chat with Videos, developed with Intel and taught by vasudevlal! In this course, you’ll work with LLaVA (Large Language and Vision Assistant), a Large Vision Language Model (LVLM) that can process both images and text. For example, given an image of a person doing a handstand on a skateboard at the beach, LLaVA doesn't just caption the scene, it’s able to predict possible outcomes, like the person losing balance or falling off. By understanding not just what's in a video frame, but what might happen next, your application can provide more insightful answers to questions about video. You'll build a full multimodal RAG pipeline that can chat about video content: - Use the BridgeTower model to create joint text-image embeddings in a 512-dimensional multimodal semantic space. - Learn video processing techniques to extract keyframes, generate transcripts using Whisper, and create captions. - Use the LanceDB vector database to store and retrieve high-dimensional multimodal embeddings. - Integrate the LLaVA model, combining CLIP's (Contrastive Language Image Pretraining) vision transformer with Llama, for advanced visual-textual reasoning. Your final system will ingest video data, generate embeddings for frames and text, perform similarity searches for relevant content, and use the retrieved multimodal context to inform LVLM-based response generation. The result is a system capable of answering nuanced questions about video content, effectively chatting about the video it has processed. Please sign up here!

Andrew Ng

107,548 Aufrufe • vor 1 Jahr

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks
26:35

Real-time world models represent a fundamental shift in AI. reactor is building the platform for real-time generative video infrastructure, supporting developers who need the tech for use across entertainment, physical AI, and robotics. Co-founders Alberto and Bryce Schmidtchen joined us last week on The Investment Memo, hosted by Partners Bucky Moore and Amber Yang, to talk about the era of world models. The conversation centered around the infrastructure Reactor is building, why real-time models are the edge right now, and current use cases for the product. Alberto and Bryce agreed that world models are shaping the way simulations are created, and that developers need a streamlined platform that can support their ideas. We believe Reactor is positioned to be at the frontier of research into real-time generative models. We look forward to seeing how these models apply across industries. Chapters 00:00 Introduction & Overview of Reactor 01:08 Meet the Hosts & Founders 02:18 The Origin Story: From 3D Assets to World Models 05:07 Real-Time Video Applications Across Industries 06:55 The Open Source World Model Explosion 07:23 Why Infrastructure Is the Opportunity 08:42 Parallels to Past Technology Waves 09:51 Bridging the Research-to-Production Gap 13:13 What Developers Are Building with World Models 16:41 Lessons from Luma AI 18:23 What Apple Vision Pro Taught Bryce About Real-Time Systems 20:48 Company Values & Team Culture 22:40 Series A: What the Capital Unlocks 24:13 Reactor's Five-Year Vision 26:09 Closing Remarks

Lightspeed

144,561 Aufrufe • vor 1 Monat

ELON: THE FUTURE OF AI IS REAL-TIME VIDEO COMPREHENSION AND GENERATION Elon said most human-AI interaction will soon happen through real-time video, both in understanding and creating it. He explained that while text carries higher information density, video already dominates the internet in both data generation and compute demand: “Most interaction is going to be real-time video with AI, real-time video comprehension and real-time video generation. That’s going to be most of the load, and that’s how it is for most of the internet right now. Most of the internet is video. Text is a pretty small percentage, but it tends to be higher value or more densely compressed information. If you look at the most bits generated and compute spent, it’s certainly going to be video.” Source: Nikhil Kamath
1:05

ELON: THE FUTURE OF AI IS REAL-TIME VIDEO COMPREHENSION AND GENERATION Elon said most human-AI interaction will soon happen through real-time video, both in understanding and creating it. He explained that while text carries higher information density, video already dominates the internet in both data generation and compute demand: “Most interaction is going to be real-time video with AI, real-time video comprehension and real-time video generation. That’s going to be most of the load, and that’s how it is for most of the internet right now. Most of the internet is video. Text is a pretty small percentage, but it tends to be higher value or more densely compressed information. If you look at the most bits generated and compute spent, it’s certainly going to be video.” Source: Nikhil Kamath

Mario Nawfal

15,670 Aufrufe • vor 7 Monaten

Liquid AI (Liquid AI) CEO Ramin Hasani (Ramin) says: "We're bringing the cost of tokens to zero." "The axis was maximizing intelligence at all costs." Foundation models need to optimize across 3 axes: 1.) Intelligence and capability 2.) Efficiency and cost 3.) Substrate: where the intelligence actually runs "Efficiency is not an afterthought. Energy is not abundant." "If you maximize intelligence at all costs, that axis alone is not going to get you to the place that you want to go." "Efficiency and cost of intelligence as a first-class citizen, and not an afterthought." "Where does this intelligence system go? AI is majorly getting hosted in data centers, but you could also bring intelligence on phones, on laptops, on airplanes, on cars." "Imagine if it's not tokens that are actually important. It's just the outcome that actually matters." "Not just thinking about foundation models as the token machines that are generating money and revenue for the foundation model companies that are useless tokens, and getting them into the place where they can actually unlock true value for enterprises."
6:17

Liquid AI (Liquid AI) CEO Ramin Hasani (Ramin) says: "We're bringing the cost of tokens to zero." "The axis was maximizing intelligence at all costs." Foundation models need to optimize across 3 axes: 1.) Intelligence and capability 2.) Efficiency and cost 3.) Substrate: where the intelligence actually runs "Efficiency is not an afterthought. Energy is not abundant." "If you maximize intelligence at all costs, that axis alone is not going to get you to the place that you want to go." "Efficiency and cost of intelligence as a first-class citizen, and not an afterthought." "Where does this intelligence system go? AI is majorly getting hosted in data centers, but you could also bring intelligence on phones, on laptops, on airplanes, on cars." "Imagine if it's not tokens that are actually important. It's just the outcome that actually matters." "Not just thinking about foundation models as the token machines that are generating money and revenue for the foundation model companies that are useless tokens, and getting them into the place where they can actually unlock true value for enterprises."

Molly O’Shea

12,816 Aufrufe • vor 11 Tagen

The infrastructure for working with video data at scale barely exists. Grass Video Search is our first step in changing that. You can now find anything in a video based on the contents of its frames, not just transcripts or tags. Over the last several months, the Grass Foundation has quietly assembled one of the largest video datasets in history: over a billion videos from across the public web, and is segmenting them by scene with VLM annotations of keyframes. This dataset is now searchable, so you can find the exact moment a visual, pattern, or concept appears. Curation has always been the bottleneck in video analysis. We built a system that automates it. This is just the beginning. Over 4 billion additional videos are queued for processing, and the index is growing daily. We’re opening access to a limited set of collaborators. If you’re a researcher, journalist, or nonprofit working with video, archives, or large-scale analysis, we’d love to hear from you.
0:45

The infrastructure for working with video data at scale barely exists. Grass Video Search is our first step in changing that. You can now find anything in a video based on the contents of its frames, not just transcripts or tags. Over the last several months, the Grass Foundation has quietly assembled one of the largest video datasets in history: over a billion videos from across the public web, and is segmenting them by scene with VLM annotations of keyframes. This dataset is now searchable, so you can find the exact moment a visual, pattern, or concept appears. Curation has always been the bottleneck in video analysis. We built a system that automates it. This is just the beginning. Over 4 billion additional videos are queued for processing, and the index is growing daily. We’re opening access to a limited set of collaborators. If you’re a researcher, journalist, or nonprofit working with video, archives, or large-scale analysis, we’d love to hear from you.

Grass

471,103 Aufrufe • vor 1 Jahr

We recently crossed 10,000 GitHub stars on Rerun 😍 That's great and all, but something bigger is happening in robotics and Physical AI right now: The open-source ecosystem has seen real acceleration over the last few years. More teams are sharing real infrastructure: datasets, models, simulators, tooling, and the unglamorous plumbing that makes iteration possible. Jan Florian Maas, Physical Intelligence, Hugging Face are just some of the names driving this forward. Visualization and data infra belong in that category. If your entire data loop depends on being able to query and inspect what your robots are doing, that layer can’t be fragile or vendor-controlled. It needs to be something you can introspect, build on, extend, and trust long-term. Huge thanks to everyone building open-source robotics. The next decade of Physical AI will be built in the open.
0:22

We recently crossed 10,000 GitHub stars on Rerun 😍 That's great and all, but something bigger is happening in robotics and Physical AI right now: The open-source ecosystem has seen real acceleration over the last few years. More teams are sharing real infrastructure: datasets, models, simulators, tooling, and the unglamorous plumbing that makes iteration possible. Jan Florian Maas, Physical Intelligence, Hugging Face are just some of the names driving this forward. Visualization and data infra belong in that category. If your entire data loop depends on being able to query and inspect what your robots are doing, that layer can’t be fragile or vendor-controlled. It needs to be something you can introspect, build on, extend, and trust long-term. Huge thanks to everyone building open-source robotics. The next decade of Physical AI will be built in the open.

Rerun

29,684 Aufrufe • vor 5 Monaten

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models
0:30

Meta announces Movie Gen A Cast of Media Foundation Models We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user’s image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models

AK

62,719 Aufrufe • vor 1 Jahr

We're changing our name. Captions is now Mirage. Today's most popular medium is short-form video. It's the fastest-growing format, it’s what people want, and it’s what brands need to grow. But unlike other forms of AI content like text or image, which can be generated and ready-to-use with a single prompt, generated video still requires a lot of manual work and editing to be ready to share. This is where the real opportunity lies. The way we see it, the real race for AI video hasn’t even begun. The world is waiting for a foundation model for video that can take an idea and produce a fully usable, cohesive output from scratch. So that’s what we’re building. We’re rebranding to Mirage to reflect our expanded vision for video, starting with the short-form format, through frontier AI research and models. Check it out at mirage dot app
0:30

We're changing our name. Captions is now Mirage. Today's most popular medium is short-form video. It's the fastest-growing format, it’s what people want, and it’s what brands need to grow. But unlike other forms of AI content like text or image, which can be generated and ready-to-use with a single prompt, generated video still requires a lot of manual work and editing to be ready to share. This is where the real opportunity lies. The way we see it, the real race for AI video hasn’t even begun. The world is waiting for a foundation model for video that can take an idea and produce a fully usable, cohesive output from scratch. So that’s what we’re building. We’re rebranding to Mirage to reflect our expanded vision for video, starting with the short-form format, through frontier AI research and models. Check it out at mirage dot app

Gaurav Misra

136,480 Aufrufe • vor 10 Monaten

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg
2:12

FRIEDBERG: VIDEO DATA WILL POWER THE NEXT GENERATION OF AI Friedberg broke down the scale shift coming to artificial intelligence, arguing that text based models like GPT are just the beginning, and that the real revolution will come from video-trained systems: “The internet and all these LLMs are language models trained on text from the internet, around 50 billion words total, maybe one to five terabytes of data in their training sets. But if you look at the video data out there, there are hundreds of billions of hours, much of it on YouTube. By some estimates, there’s a thousand exabytes of video data on the internet, about a billion times more than text data. I think we just saw that play out with the new video model that launched yesterday. Google has all this YouTube data, whether or not they’re using it to train, I don’t know. I’ve heard from insiders they’re not allowed to yet and would have to redo the terms of service.” Source: AIFinInsights david friedberg

Mario Nawfal

15,855 Aufrufe • vor 7 Monaten

Jensen Huang explains the AI industry as a five layer cake, and why "model only" companies are missing the real map. "AI is not just a model. AI is really a five layer cake to think about it." "From the lowest level, it requires energy because AI is produced, generated in real time, just like you and I right now. We need calories, we need energy." "The next level is chips. This is where Nvidia is and where a lot of the companies in Taiwan are." "The layer above that is infrastructure: land, power, data center, as well as the cloud service software that turns the chips into a cloud service." "The layer above that is where most people think about when they think about AI, which is models. ChatGPT was revolutionary. Claude Code is now incredibly successful. But AI is much more than just human language." Most people are still debating the model layer. Jensen is describing an industrial stack: energy, chips, infrastructure, models, and applications. If you are building in AI, this is the map that matters. The winning product may look like software, but the bottlenecks are calories, land, power, data centers, chips, and distribution. - Jensen Huang (NVIDIA), CEO of Nvidia, on CNA
1:48

Jensen Huang explains the AI industry as a five layer cake, and why "model only" companies are missing the real map. "AI is not just a model. AI is really a five layer cake to think about it." "From the lowest level, it requires energy because AI is produced, generated in real time, just like you and I right now. We need calories, we need energy." "The next level is chips. This is where Nvidia is and where a lot of the companies in Taiwan are." "The layer above that is infrastructure: land, power, data center, as well as the cloud service software that turns the chips into a cloud service." "The layer above that is where most people think about when they think about AI, which is models. ChatGPT was revolutionary. Claude Code is now incredibly successful. But AI is much more than just human language." Most people are still debating the model layer. Jensen is describing an industrial stack: energy, chips, infrastructure, models, and applications. If you are building in AI, this is the map that matters. The winning product may look like software, but the bottlenecks are calories, land, power, data centers, chips, and distribution. - Jensen Huang (NVIDIA), CEO of Nvidia, on CNA

Karl Mehta

14,946 Aufrufe • vor 13 Tagen

Shaun Maguire on SpaceX IPO: "Starship is the railroad to space." "In 2019, it was very predictable that Starlink would work and produce a massive amount of revenue, and we called that right." "We're now in a very similar point to where Starlink was in 2019 with AI infrastructure and orbital compute." "I think people are really underestimating how far along the orbital compute product is right now." On the $22 trillion TAM: "What do you think is the TAM for intelligence? What's the TAM for all compute infrastructure 50 years from now?" "People are underappreciating how much infrastructure is going to be built in space and how much easier it is to access energy in space." "We're at the beginning of this AI era, and we're going to double or triple the total amount of intelligence that we have access to over the next decade — so yes, I absolutely believe in the TAM, and I would even argue it's an underestimate." How much is the "railroad to space" worth in the era of AI and orbital compute? Now imagine the railroad company also owns the AI and the orbital compute. Elon Musk Shaun Maguire
2:24

Shaun Maguire on SpaceX IPO: "Starship is the railroad to space." "In 2019, it was very predictable that Starlink would work and produce a massive amount of revenue, and we called that right." "We're now in a very similar point to where Starlink was in 2019 with AI infrastructure and orbital compute." "I think people are really underestimating how far along the orbital compute product is right now." On the $22 trillion TAM: "What do you think is the TAM for intelligence? What's the TAM for all compute infrastructure 50 years from now?" "People are underappreciating how much infrastructure is going to be built in space and how much easier it is to access energy in space." "We're at the beginning of this AI era, and we're going to double or triple the total amount of intelligence that we have access to over the next decade — so yes, I absolutely believe in the TAM, and I would even argue it's an underestimate." How much is the "railroad to space" worth in the era of AI and orbital compute? Now imagine the railroad company also owns the AI and the orbital compute. Elon Musk Shaun Maguire

KanekoaTheGreat

39,709 Aufrufe • vor 1 Monat

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).
5:39

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).

Santiago

56,412 Aufrufe • vor 1 Jahr

“Liam was … we spent a lot of time together and we were together a few weeks before he passed away. […] •An acoustic version of “Through the Dark” was what I wanted to do [at the Grammys]. And that was the plan for a little while to do that separate from Immemorium. And then sadly, the artist I had in mind couldn't make it. […] •We like half filmed a video many years ago called ‘Infinity’ and we never finished the video because […] the boys changed their mind and they decided to go with another single [‘History’]. […]So we had these like this half video of like two hour shoot. So for years the fans have always been going, ‘when's the Infinity video?’ ‘We want the Infinity video.’ But it never existed. There never was one. […] We like took a bit from ‘This Is Us’. We took a bit from the Where We Are Tour at San Siro. And then we took a little bit from Wembley Stadium. And then also from that Infinity video that's never been seen. So people could just see like his joy and love for life. There was so many clips, but he's just laughing and like loving life and like enjoying it and looking at the camera. And like, if you look at it, the consistency of every single one of those clips is significant because he's looking over Verona and celebrating with his hands in the air and smiling. And then he's looking at Wembley Stadium, he's smiling, and then San Siro and he's laughing. And then we're doing the Infinity video and he's laughing. And like, that was really beautiful just to show across the period of like a few years, his joy and kindness and love for life.” - Ben Winston about Liam’s Tribute at the Grammys, Rolling Stone Music Now
4:21

“Liam was … we spent a lot of time together and we were together a few weeks before he passed away. […] •An acoustic version of “Through the Dark” was what I wanted to do [at the Grammys]. And that was the plan for a little while to do that separate from Immemorium. And then sadly, the artist I had in mind couldn't make it. […] •We like half filmed a video many years ago called ‘Infinity’ and we never finished the video because […] the boys changed their mind and they decided to go with another single [‘History’]. […]So we had these like this half video of like two hour shoot. So for years the fans have always been going, ‘when's the Infinity video?’ ‘We want the Infinity video.’ But it never existed. There never was one. […] We like took a bit from ‘This Is Us’. We took a bit from the Where We Are Tour at San Siro. And then we took a little bit from Wembley Stadium. And then also from that Infinity video that's never been seen. So people could just see like his joy and love for life. There was so many clips, but he's just laughing and like loving life and like enjoying it and looking at the camera. And like, if you look at it, the consistency of every single one of those clips is significant because he's looking over Verona and celebrating with his hands in the air and smiling. And then he's looking at Wembley Stadium, he's smiling, and then San Siro and he's laughing. And then we're doing the Infinity video and he's laughing. And like, that was really beautiful just to show across the period of like a few years, his joy and kindness and love for life.” - Ben Winston about Liam’s Tribute at the Grammys, Rolling Stone Music Now

HSNews

203,067 Aufrufe • vor 1 Jahr

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior
44:17

How does an AI model actually learn to see? 🤖 Learn about the tech behind native multimodality, how models reason over visual data like documents and video, and the future of proactive AI assistants with Logan Kilpatrick and Gemini Model Behavior Product Lead, Ani Baddepudi. ↓ Timestamps: 01:12 Why Gemini is natively multimodal 02:23 The technology behind multimodal models 05:15 Video understanding with Gemini 2.5 09:25 Deciding what to build next 13:23 Building new product experiences with multimodal AI 17:15 The vision for proactive assistants 24:13 Improving video usability with variable FPS and frame tokenization 27:35 What’s next for Gemini’s multimodal development 31:47 Deep dive on Gemini’s document understanding capabilities 37:56 The teamwork and collaboration behind Gemini 40:56 What’s next with model behavior

Google AI

58,703 Aufrufe • vor 1 Jahr

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Robots might learn better from video than from language! 📼 Most Vision-Language-Action (VLA) models learn what to do from text, but still struggle with how things move in the real world. That makes them data-hungry and slow to train. mimic video takes a different route. Instead of grounding robot control in text, it grounds it in video, using large pre-trained video models that already capture physical motion and dynamics. The idea is straightforward: let the video model handle “what will happen next,” and let a smaller control model focus only on turning that visual plan into robot actions. The result is big gains in practice. Robots trained this way need 10× less data, converge twice as fast, and perform better on both simulated benchmarks and real bimanual manipulation tasks. If robots can “imagine” motion using video, control becomes a much simpler problem. Shoutout to Jonas Pai, Liam Achenbach, Oier Mees, Elvis Nava and the rest of the team! Here's the project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

49,920 Aufrufe • vor 6 Monaten

., Co-founder of fal, on where AI image & video generation stands today: “Image and video models haven’t really generalized yet. There’s still a lot of model chaining and workflow generation." "Models are getting bigger and better, but unlike LLMs where you can write a prompt and expect it to handle various tasks, that doesn’t happen for media yet.” He notes that while chaining multiple models is still the norm, the opportunity lies in building hybrid systems that stitch workflows together reliably until true generalization arrives.
1:14

., Co-founder of fal, on where AI image & video generation stands today: “Image and video models haven’t really generalized yet. There’s still a lot of model chaining and workflow generation." "Models are getting bigger and better, but unlike LLMs where you can write a prompt and expect it to handle various tasks, that doesn’t happen for media yet.” He notes that while chaining multiple models is still the norm, the opportunity lies in building hybrid systems that stitch workflows together reliably until true generalization arrives.

TBPN

36,675 Aufrufe • vor 11 Monaten

Spectre AI’s founder steps in front of the camera for Y Combinator . As part of our Y Combinator Summer Batch 2026 application, our founder Sunny Enzo recorded a founder video walking through the Spectre AI app and the direction of the company. Sunny is doxxed, but this is the first time he is appearing on video to speak directly about Spectre AI, the product, and the market intelligence layer we are building. Due to YC’s time format, the video only shows around 5 to 6 features from an ecosystem with 100+ features already built or in progress. A full Spectre AI walkthrough would need far more time. Still, it gives a clear look at where we are heading: AI market intelligence, on-chain data, social intelligence, real-time context, and a cleaner way to understand financial markets through one platform. Another step in taking Spectre AI from product delivery into serious company building.
3:03

Spectre AI’s founder steps in front of the camera for Y Combinator . As part of our Y Combinator Summer Batch 2026 application, our founder Sunny Enzo recorded a founder video walking through the Spectre AI app and the direction of the company. Sunny is doxxed, but this is the first time he is appearing on video to speak directly about Spectre AI, the product, and the market intelligence layer we are building. Due to YC’s time format, the video only shows around 5 to 6 features from an ecosystem with 100+ features already built or in progress. A full Spectre AI walkthrough would need far more time. Still, it gives a clear look at where we are heading: AI market intelligence, on-chain data, social intelligence, real-time context, and a cleaner way to understand financial markets through one platform. Another step in taking Spectre AI from product delivery into serious company building.

SPECTRE AI

17,501 Aufrufe • vor 2 Monaten

99% of AI applications are cool-looking demos. Impressive, but don't get fooled by the hype. It takes a lot to build enterprise-grade products that deliver real value. I have at least three weekly conversations with companies that want to use a Large Language Model with their data. The demand is huge! Here is one idea about what you can do to help. The use cases that most of these companies want to solve are similar: They have an extensive knowledge base and want to build a simple application that uses that information to answer questions. In other words, they need help building Retrieval Augmented Generation (RAG) applications they can use in many different scenarios: 1. To train new employees 2. To help their support team 3. To search old meetings and documents 4. To help with their research However, building these systems is not straightforward. Yes, there's a lot of information online, but there aren't enough people who know how to create solutions that work. Here is the idea: Today, you can build an enterprise-grade RAG application without writing code. A couple of MIT PhDs with 10+ years of experience building AI applications created . It's a no-code platform for building applications using Large Language Models. They are partnering with me on this post. You can use Stack AI to create, test, and deploy an end-to-end production-ready AI system. It's SOC-2, HIPAA, and GDPR compliant and offers SSO, role management, access control, and on-premise deployments. Of course, you can use the platform with any LLM on the market now. It's the whole nine yards for building AI applications. Check them out here: 2023 was about models. 2024 is about the tools using these models to build production-ready applications. That's where I'd start.
0:31

99% of AI applications are cool-looking demos. Impressive, but don't get fooled by the hype. It takes a lot to build enterprise-grade products that deliver real value. I have at least three weekly conversations with companies that want to use a Large Language Model with their data. The demand is huge! Here is one idea about what you can do to help. The use cases that most of these companies want to solve are similar: They have an extensive knowledge base and want to build a simple application that uses that information to answer questions. In other words, they need help building Retrieval Augmented Generation (RAG) applications they can use in many different scenarios: 1. To train new employees 2. To help their support team 3. To search old meetings and documents 4. To help with their research However, building these systems is not straightforward. Yes, there's a lot of information online, but there aren't enough people who know how to create solutions that work. Here is the idea: Today, you can build an enterprise-grade RAG application without writing code. A couple of MIT PhDs with 10+ years of experience building AI applications created . It's a no-code platform for building applications using Large Language Models. They are partnering with me on this post. You can use Stack AI to create, test, and deploy an end-to-end production-ready AI system. It's SOC-2, HIPAA, and GDPR compliant and offers SSO, role management, access control, and on-premise deployments. Of course, you can use the platform with any LLM on the market now. It's the whole nine yards for building AI applications. Check them out here: 2023 was about models. 2024 is about the tools using these models to build production-ready applications. That's where I'd start.

Santiago

197,675 Aufrufe • vor 2 Jahren

Palantir is playing an integral role in the AI Revolution, and Akshay Krishnaswamy summarized it best⚡️ “Our conviction here at Palantir is that really it constitutes a new category in the AI stack...Of course there's raw energy. You then have chips, you have data centers, you have the frontier models that are increasingly commoditized and available but increasingly powerful as well. And then there's everything you need to get from that to differentiated outcomes, products, and services. And we are maniacally focused on providing this AI infrastructure for developer teams, for enterprises, for everybody looking to achieve value with AI.”
0:37

Palantir is playing an integral role in the AI Revolution, and Akshay Krishnaswamy summarized it best⚡️ “Our conviction here at Palantir is that really it constitutes a new category in the AI stack...Of course there's raw energy. You then have chips, you have data centers, you have the frontier models that are increasingly commoditized and available but increasingly powerful as well. And then there's everything you need to get from that to differentiated outcomes, products, and services. And we are maniacally focused on providing this AI infrastructure for developer teams, for enterprises, for everybody looking to achieve value with AI.”

Eliano A. Younes

11,513 Aufrufe • vor 1 Jahr

Elon Musk: Despite starting from scratch six months ago, Grok Imagine now generates more videos and images than everyone else combined. “My prediction is that most of AI compute is going to be real-time video understanding and real-time video generation. And we expect to be the leaders in that. It's worth emphasizing these points that six months ago we had basically nothing, very weak in video and image generation and editing and went in six months to the number one spot and in fact generating more videos and images than everyone else combined. We're going to do the same thing with coding and we're going to do the same thing with Macrohard and. I think people will be pretty impressed with the Grok 4.2 model that's coming out. It's a significant improvement. That's the small version of our new model. So we'll have a medium and a large version that are even more intelligent.” From: xAI All-Hands, February 10, 2026
0:42

Elon Musk: Despite starting from scratch six months ago, Grok Imagine now generates more videos and images than everyone else combined. “My prediction is that most of AI compute is going to be real-time video understanding and real-time video generation. And we expect to be the leaders in that. It's worth emphasizing these points that six months ago we had basically nothing, very weak in video and image generation and editing and went in six months to the number one spot and in fact generating more videos and images than everyone else combined. We're going to do the same thing with coding and we're going to do the same thing with Macrohard and. I think people will be pretty impressed with the Grok 4.2 model that's coming out. It's a significant improvement. That's the small version of our new model. So we'll have a medium and a large version that are even more intelligent.” From: xAI All-Hands, February 10, 2026

ELON CLIPS

12,007 Aufrufe • vor 5 Monaten