Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Excited to present DrivAerNet++, coming to #NeurIPS24 Datasets & Benchmarks! DrivAerNet++ introduces the first large-scale multimodal dataset for aerodynamic car design with full CFD simulations. Great work by Mohamed Elrefaie, Florin Morar, Faez Ahmed

angelaqdai's profile picture

Angela Dai

9,578 subscribers

17,749 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungWissenschaft & TechnologieNachrichten & Politik#NeurIPS24
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️
0:25

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️

AI at Meta

52,408 Aufrufe • vor 1 Jahr

Excited to present UnScene3D coming to #CVPR24. David Rozenberszki introduces the first fully unsupervised learned 3D instance segmentation! Using graph cuts over self-supervised features, and self-training for robust & dense masks. by David Rozenberszki, w/ Or Litany
0:48

Excited to present UnScene3D coming to #CVPR24. David Rozenberszki introduces the first fully unsupervised learned 3D instance segmentation! Using graph cuts over self-supervised features, and self-training for robust & dense masks. by David Rozenberszki, w/ Or Litany

Angela Dai

11,586 Aufrufe • vor 2 Jahren

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:
1:25

Introducing FoundationMotion. A large-scale, video-derived motion annotation dataset & auto-labeling pipeline + advanced models for motion understanding. Fully open-source: code, datasets, and models, free to use and build on. Understanding motion is core to physical reasoning, yet today’s leading models still struggle with simple spatial actions like “turn right” or “move up” or “flip the toast” - mainly due to the lack of large, fine-grained motion datasets. We present FoundationMotion, a fully automated pipeline that: • detects & tracks objects in videos • extracts trajectories • uses LLMs + frames to generate rich motion captions & QA pairs → creating large-scale, high-quality motion datasets at scale. After fine-tuning the open-source models Qwen and NVILA on our annotations, these models now outperform the closed-source Gemini-3-Flash and GPT-5.1 on spatial understanding tasks across autonomous driving, robotics, and everyday scenarios. 📜Paper: 🌐Webpage: 💻 Code: 🕸️Model: 📊 Dataset: 👉 Interactive Demo: Let’s move research forward together. FoundationMotion is also referred to as Wolf V2 🐺, the second chapter in the Wolf series:

Boyi Li

66,999 Aufrufe • vor 8 Monaten

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.
0:33

PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking paper page: introduce PointOdyssey, a large-scale synthetic dataset, and data generation framework, for the training and evaluation of long-term fine-grained tracking algorithms. Our goal is to advance the state-of-the-art by placing emphasis on long videos with naturalistic motion. Toward the goal of naturalism, we animate deformable characters using real-world motion capture data, we build 3D scenes to match the motion capture environments, and we render camera viewpoints using trajectories mined via structure-from-motion on real videos. We create combinatorial diversity by randomizing character appearance, motion profiles, materials, lighting, 3D assets, and atmospheric effects. Our dataset currently includes 104 videos, averaging 2,000 frames long, with orders of magnitude more correspondence annotations than prior work. We show that existing methods can be trained from scratch in our dataset and outperform the published variants. Finally, we introduce modifications to the PIPs point tracking method, greatly widening its temporal receptive field, which improves its performance on PointOdyssey as well as on two real-world benchmarks.

AK

122,560 Aufrufe • vor 3 Jahren

NEWS: Nvidia has announced that they are partnering with General Motors on full self-driving. Nvidia CEO Jensen Huang: “Today, I’m super excited to announce that GM has selected Nvidia to partner with to build their future self-driving car fleet. We’re looking forward to building with GM on AI for manufacturing so they can revolutionize the way they manufacture, AI for enterprise so they can revolutionize the way they work, design and simulate cars, and also AI for in the car.”
0:49

NEWS: Nvidia has announced that they are partnering with General Motors on full self-driving. Nvidia CEO Jensen Huang: “Today, I’m super excited to announce that GM has selected Nvidia to partner with to build their future self-driving car fleet. We’re looking forward to building with GM on AI for manufacturing so they can revolutionize the way they manufacture, AI for enterprise so they can revolutionize the way they work, design and simulate cars, and also AI for in the car.”

Sawyer Merritt

419,970 Aufrufe • vor 1 Jahr

Progressively Optimized Local Radiance Fields for Robust View Synthesis paper page: present an algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video. The task poses two core challenges. First, most existing radiance field reconstruction approaches rely on accurate pre-estimated camera poses from Structure-from-Motion algorithms, which frequently fail on in-the-wild videos. Second, using a single, global radiance field with finite representational capacity does not scale to longer trajectories in an unbounded scene. For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner. We show that progressive optimization significantly improves the robustness of the reconstruction. For handling large unbounded scenes, we dynamically allocate new local radiance fields trained with frames within a temporal window. This further improves robustness (e.g., performs well even under moderate pose drifts) and allows us to scale to large scenes. Our extensive evaluation on the Tanks and Temples dataset and our collected outdoor dataset, Static Hikes, show that our approach compares favorably with the state-of-the-art.
4:28

Progressively Optimized Local Radiance Fields for Robust View Synthesis paper page: present an algorithm for reconstructing the radiance field of a large-scale scene from a single casually captured video. The task poses two core challenges. First, most existing radiance field reconstruction approaches rely on accurate pre-estimated camera poses from Structure-from-Motion algorithms, which frequently fail on in-the-wild videos. Second, using a single, global radiance field with finite representational capacity does not scale to longer trajectories in an unbounded scene. For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner. We show that progressive optimization significantly improves the robustness of the reconstruction. For handling large unbounded scenes, we dynamically allocate new local radiance fields trained with frames within a temporal window. This further improves robustness (e.g., performs well even under moderate pose drifts) and allows us to scale to large scenes. Our extensive evaluation on the Tanks and Temples dataset and our collected outdoor dataset, Static Hikes, show that our approach compares favorably with the state-of-the-art.

AK

140,624 Aufrufe • vor 3 Jahren

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.
0:22

ViPE: Video Pose Engine for 3D Geometric Perception Contributions: • A robust and efficient framework, ViPE, for estimating camera parameters and dense depth from diverse, in-the-wild videos. • A system design that integrates the strengths of classical SLAM (efficiency, scalability) and learned models (robustness), with key improvements in efficiency, dynamic object handling, and depth quality over prior work. • A large-scale dataset of annotated videos, created using ViPE, to facilitate future research in 3D computer vision.

MrNeRF

42,553 Aufrufe • vor 1 Jahr

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision paper page: We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation.
0:29

DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision paper page: We have witnessed significant progress in deep learning-based 3D vision, ranging from neural radiance field (NeRF) based 3D representation learning to applications in novel view synthesis (NVS). However, existing scene-level datasets for deep learning-based 3D vision, limited to either synthetic environments or a narrow selection of real-world scenes, are quite insufficient. This insufficiency not only hinders a comprehensive benchmark of existing methods but also caps what could be explored in deep learning-based 3D analysis. To address this critical gap, we present DL3DV-10K, a large-scale scene dataset, featuring 51.2 million frames from 10,510 videos captured from 65 types of point-of-interest (POI) locations, covering both bounded and unbounded scenes, with different levels of reflection, transparency, and lighting. We conducted a comprehensive benchmark of recent NVS methods on DL3DV-10K, which revealed valuable insights for future research in NVS. In addition, we have obtained encouraging results in a pilot study to learn generalizable NeRF from DL3DV-10K, which manifests the necessity of a large-scale scene-level dataset to forge a path toward a foundation model for learning 3D representation.

AK

49,920 Aufrufe • vor 2 Jahren

I am pleased to announce the inaugural auction of Dubai's first licensed Phygital Art Car. The Art Car introduces a new pinnacle of prestige—licensed Phygital Art Cars. This milestone redefines status by merging the exclusivity of sought-after license plates with artistic expression blockchain innovation, making it the ultimate Phygital Flex. The first Art Car is adorned with the iconic Amidst The Waves artwork by VESA, and bidding will start at $1,000,000. 🗓 12th January 📍 e& MOTB, Dubai Design District ⏰ 8:00 PM 🔎Register for the auction at 🙌Join the community:
1:52

I am pleased to announce the inaugural auction of Dubai's first licensed Phygital Art Car. The Art Car introduces a new pinnacle of prestige—licensed Phygital Art Cars. This milestone redefines status by merging the exclusivity of sought-after license plates with artistic expression blockchain innovation, making it the ultimate Phygital Flex. The first Art Car is adorned with the iconic Amidst The Waves artwork by VESA, and bidding will start at $1,000,000. 🗓 12th January 📍 e& MOTB, Dubai Design District ⏰ 8:00 PM 🔎Register for the auction at 🙌Join the community:

Marwan Alzarouni

20,978 Aufrufe • vor 1 Jahr

Artist Jeffrey Gibson’s work is visually captivating and sparks important conversations about our responsibility to the planet and its living beings. "The Animal That Therefore I Am," features four unique sculptures that invite viewers to reflect on the deep interconnectedness between all life and the environment. This momentous body of work explores the relationship between humans and nature, embodying the essence of Gibson's artistic philosophy. Through his intricate designs and thoughtful composition, Gibson encourages a dialogue on how we coexist with our natural surroundings. 🗓️ Create your own beadwork design this Saturday, April 11 and hear about the inspiration and creative process behind the large-scale sculptures created for The Met’s facade with artist Jeffrey Gibson. Link in bio to learn more. 📣 Create your own beadwork design this Saturday, April 11 and hear about the inspiration and creative process behind the large-scale sculptures created for The Met’s facade with artist Jeffrey Gibson. Learn more → The exhibition is presented by Genesis.
1:31

Artist Jeffrey Gibson’s work is visually captivating and sparks important conversations about our responsibility to the planet and its living beings. "The Animal That Therefore I Am," features four unique sculptures that invite viewers to reflect on the deep interconnectedness between all life and the environment. This momentous body of work explores the relationship between humans and nature, embodying the essence of Gibson's artistic philosophy. Through his intricate designs and thoughtful composition, Gibson encourages a dialogue on how we coexist with our natural surroundings. 🗓️ Create your own beadwork design this Saturday, April 11 and hear about the inspiration and creative process behind the large-scale sculptures created for The Met’s facade with artist Jeffrey Gibson. Link in bio to learn more. 📣 Create your own beadwork design this Saturday, April 11 and hear about the inspiration and creative process behind the large-scale sculptures created for The Met’s facade with artist Jeffrey Gibson. Learn more → The exhibition is presented by Genesis.

The Metropolitan Museum of Art

15,848 Aufrufe • vor 4 Monaten

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI
3:58

Excited to share our latest work on 🎧spatial audio-driven human motion generation. We aim to tackle a largely underexplored yet important problem of enabling virtual humans to move naturally in response to spatial audio—capturing not just what is heard, but also where the sound is coming from. To this end, we introduce the Spatial Audio-Driven Human Motion (SAM) dataset—the first comprehensive dataset featuring paired high-quality human motion and spatial audio recordings. For benchmarking, we develop a generative framework for human MOtion generation driven by SPAtial audio, termed MOSPA, which learns to synthesize realistic and diverse human motions conditioned on spatial audio input. We hope this research could provide a foundation for future research in spatial perception, virtual characters, and embodied AI. The dataset and model will be open-sourced soon. A big thank you to our intern, Shuyang Xu, for the wonderful collaboration! Congratulations, Shuyang! Project page: Paper: Video: #Animation #CG #CV #AIGC #DL #Deeplearning #Motion #Graphics #AI #GenerativeAI

Zhiyang (Frank) Dou

14,610 Aufrufe • vor 1 Jahr

My first exposure to Cyberman, this day in 1975. Often overshadowed by the Hinchcliffe masterpieces that follow, Revenge of the Cybermen is stark SF with a new take on Cybermen, atmospheric location work, and the Doctor, Sarah & Harry trio on top form. Great finale for Season 12.
0:50

My first exposure to Cyberman, this day in 1975. Often overshadowed by the Hinchcliffe masterpieces that follow, Revenge of the Cybermen is stark SF with a new take on Cybermen, atmospheric location work, and the Doctor, Sarah & Harry trio on top form. Great finale for Season 12.

Chloe 'Cleo' Mortimer

20,978 Aufrufe • vor 3 Monaten

Elon Musk just unveiled SpaceXAI's first AI satellite. • 150 kW peak power / 120 kW sustained compute power • 150 kW solar array using SpaceX-manufactured solar technology • Centralized AI compute payload designed for high-performance AI workloads • 70-meter wingspan when fully deployed • 110 m² deployable liquid radiators to remove waste heat in space • Redundant cooling loops with integrated micrometeoroid shielding • Designed to launch on Starship, enabling the mass-to-orbit needed for large-scale space computing • Uses laser links while avoiding many of the complex communications systems required by Starlink satellites • SpaceX believes future versions can scale far beyond this first design Elon Musk says the path to scaling AI in space requires 3 things: • Massive launch capability (Starship) • Enormous solar power generation • Large radiators to reject heat from AI chips He also suggested that truly large-scale orbital AI could eventually require hundreds of gigawatts to a terawatt of power, implying millions of tons of infrastructure in orbit.
2:38

Elon Musk just unveiled SpaceXAI's first AI satellite. • 150 kW peak power / 120 kW sustained compute power • 150 kW solar array using SpaceX-manufactured solar technology • Centralized AI compute payload designed for high-performance AI workloads • 70-meter wingspan when fully deployed • 110 m² deployable liquid radiators to remove waste heat in space • Redundant cooling loops with integrated micrometeoroid shielding • Designed to launch on Starship, enabling the mass-to-orbit needed for large-scale space computing • Uses laser links while avoiding many of the complex communications systems required by Starlink satellites • SpaceX believes future versions can scale far beyond this first design Elon Musk says the path to scaling AI in space requires 3 things: • Massive launch capability (Starship) • Enormous solar power generation • Large radiators to reject heat from AI chips He also suggested that truly large-scale orbital AI could eventually require hundreds of gigawatts to a terawatt of power, implying millions of tons of infrastructure in orbit.

DogeDesigner

138,295 Aufrufe • vor 2 Monaten

Dear friends, l'm so excited to share our most recent and ambitious project: Large Nature Model / Living Art in collaboration with NVIDIA . This week, before the epic GTC keynote of Jensen Huang, I was able to transform 20 meters high and 40 meters wide 16k resolution media wall into real-time Al Data Sculpture. Artwork fed by real-time Large Nature Model's flora, fauna, and landscapes Al text to video outputs! Together with NVIDIA Al researchers, we finetuned Getty Image's foundational model with our 2.5 million ethically sourced Nature dataset and made a significant scale physical installation with Al scent and Al character as well. We are deeply grateful to the entire NVIDIA team for the collaboration, support, and pushing the boundaries of imagination in the age of Al!
0:58

Dear friends, l'm so excited to share our most recent and ambitious project: Large Nature Model / Living Art in collaboration with NVIDIA . This week, before the epic GTC keynote of Jensen Huang, I was able to transform 20 meters high and 40 meters wide 16k resolution media wall into real-time Al Data Sculpture. Artwork fed by real-time Large Nature Model's flora, fauna, and landscapes Al text to video outputs! Together with NVIDIA Al researchers, we finetuned Getty Image's foundational model with our 2.5 million ethically sourced Nature dataset and made a significant scale physical installation with Al scent and Al character as well. We are deeply grateful to the entire NVIDIA team for the collaboration, support, and pushing the boundaries of imagination in the age of Al!

Refik Anadol

51,355 Aufrufe • vor 2 Jahren

Really fun to hang again with my friend 🃏 Noam Brown (OpenAI research scientist, our first guest ever on No Priors in early 2023) to talk about the implications of large test-time compute, and what happens when models are given $10M budgets to spend on a single task. Topics: 01:23 – Why Benchmarks Are Broken 04:19 – Compute Budgets and Projections 06:48 – How Long Should Models Think? 08:01 – Benchmarkmaxxing 09:48 – Noam's Evals 12:40 – Safety (When Model Capability Scales With Spend) 16:09 – Implications For the Model Release Cycle 18:34 – Latent Model Capability 22:27 – Limits on Recursive Self-Improvement 28:38 – Large-Scale Multi-Agent Coordination 30:39 – Competition at the Frontier 33:19 – Breaking the Benchmark Grid Equilibrium 34:57 – Why Benchmarks Should be Scaled by Cost
36:18

Really fun to hang again with my friend 🃏 Noam Brown (OpenAI research scientist, our first guest ever on No Priors in early 2023) to talk about the implications of large test-time compute, and what happens when models are given $10M budgets to spend on a single task. Topics: 01:23 – Why Benchmarks Are Broken 04:19 – Compute Budgets and Projections 06:48 – How Long Should Models Think? 08:01 – Benchmarkmaxxing 09:48 – Noam's Evals 12:40 – Safety (When Model Capability Scales With Spend) 16:09 – Implications For the Model Release Cycle 18:34 – Latent Model Capability 22:27 – Limits on Recursive Self-Improvement 28:38 – Large-Scale Multi-Agent Coordination 30:39 – Competition at the Frontier 33:19 – Breaking the Benchmark Grid Equilibrium 34:57 – Why Benchmarks Should be Scaled by Cost

sarah guo

87,850 Aufrufe • vor 1 Monat

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.
1:00

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Neuracore

40,620 Aufrufe • vor 8 Monaten

The infrastructure for working with video data at scale barely exists. Grass Video Search is our first step in changing that. You can now find anything in a video based on the contents of its frames, not just transcripts or tags. Over the last several months, the Grass Foundation has quietly assembled one of the largest video datasets in history: over a billion videos from across the public web, and is segmenting them by scene with VLM annotations of keyframes. This dataset is now searchable, so you can find the exact moment a visual, pattern, or concept appears. Curation has always been the bottleneck in video analysis. We built a system that automates it. This is just the beginning. Over 4 billion additional videos are queued for processing, and the index is growing daily. We’re opening access to a limited set of collaborators. If you’re a researcher, journalist, or nonprofit working with video, archives, or large-scale analysis, we’d love to hear from you.
0:45

The infrastructure for working with video data at scale barely exists. Grass Video Search is our first step in changing that. You can now find anything in a video based on the contents of its frames, not just transcripts or tags. Over the last several months, the Grass Foundation has quietly assembled one of the largest video datasets in history: over a billion videos from across the public web, and is segmenting them by scene with VLM annotations of keyframes. This dataset is now searchable, so you can find the exact moment a visual, pattern, or concept appears. Curation has always been the bottleneck in video analysis. We built a system that automates it. This is just the beginning. Over 4 billion additional videos are queued for processing, and the index is growing daily. We’re opening access to a limited set of collaborators. If you’re a researcher, journalist, or nonprofit working with video, archives, or large-scale analysis, we’d love to hear from you.

Grass

471,574 Aufrufe • vor 1 Jahr

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein
1:12

📢 IntrinsiX: High-Quality PBR Generation using Image Priors 📢 From text input, we generate renderable PBR maps! Next to editable image generation, our predictions can be distilled into room-scale scenes using SDS for large-scale PBR texture generation. We first train separate LoRA modules for the intrinsic properties of albedo, rough/metal, normal. Then, we introduce cross-intrinsic attention using a rerendering loss with importance-weighted light sampling to enable coherent PBR generation. Our method outperforms text -> image -> PBR methods both in generalization and quality, since directly generating PBR maps does not suffer from the inherent ambiguity of intrinsic image decomposition. In addition, our design choice facilitates SDS-based PBR texture distillation. 🌍 🎥 Great work by Peter Kocsis, Lukas Höllein

Matthias Niessner

21,908 Aufrufe • vor 1 Jahr

The Beijing Innovation Center of Humanoid Robotics (X-Humanoid) is building its own "data factory" to tackle the biggest bottleneck in Embodied AI, aiming for a world-first 1-million-hour milestone of high-quality real-world data. 🤖 This 53,820-square-foot (5,000 square meters) facility is already operating with a fleet of over 120 robots from various makers, including their own Tien Kung series and platforms like Unitree and UBTECH. By replicating 30+ real-life scenarios—from kitchens to industrial lines—the center is capturing the "physical intuition" that digital simulations simply cannot replicate. Their "factory-style" pipeline ensures a 95% data pass rate through a rigorous three-stage inspection process, moving beyond the era of small-scale, "fine-tuned" lab datasets. With over 2 million downloads of their open-source Robomind dataset, they are quickly becoming the backbone for developers looking to bridge the sim-to-real gap.
1:57

The Beijing Innovation Center of Humanoid Robotics (X-Humanoid) is building its own "data factory" to tackle the biggest bottleneck in Embodied AI, aiming for a world-first 1-million-hour milestone of high-quality real-world data. 🤖 This 53,820-square-foot (5,000 square meters) facility is already operating with a fleet of over 120 robots from various makers, including their own Tien Kung series and platforms like Unitree and UBTECH. By replicating 30+ real-life scenarios—from kitchens to industrial lines—the center is capturing the "physical intuition" that digital simulations simply cannot replicate. Their "factory-style" pipeline ensures a 95% data pass rate through a rigorous three-stage inspection process, moving beyond the era of small-scale, "fine-tuned" lab datasets. With over 2 million downloads of their open-source Robomind dataset, they are quickly becoming the backbone for developers looking to bridge the sim-to-real gap.

RoboHub🤖

23,072 Aufrufe • vor 4 Monaten