Loading video...

Video Failed to Load

Go Home

Today we're releasing WildDet3D—an open model for monocular 3D object detection in the wild. It works with text, clicks, or 2D boxes, and on zero-shot evals it nearly doubles the best prior scores. 🧵

allen_ai's profile picture

Ai2

85,095 subscribers

85,809 views • 3 months ago •via X (Twitter)

EducationScience & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Introducing WildDet3D, a grounding model for monocular 3D object detection in the wild. A question I keep coming back to is: what is the right backbone for robotics foundation models? Should it be a video model, a language model, or perhaps a grounding model? WildDet3D is our first step in exploring that direction.
2:10

Introducing WildDet3D, a grounding model for monocular 3D object detection in the wild. A question I keep coming back to is: what is the right backbone for robotics foundation models? Should it be a video model, a language model, or perhaps a grounding model? WildDet3D is our first step in exploring that direction.

Jiafei Duan

12,072 views • 3 months ago

We're releasing MolmoMotion, a 3D motion forecasting model. Given one or a few video frames, 3D points on an object, & an instruction like "Put the white bowl on the table," MolmoMotion predicts where those points will go over the next few seconds in a shared 3D world frame. 🧵
1:53

We're releasing MolmoMotion, a 3D motion forecasting model. Given one or a few video frames, 3D points on an object, & an instruction like "Put the white bowl on the table," MolmoMotion predicts where those points will go over the next few seconds in a shared 3D world frame. 🧵

Ai2

186,636 views • 22 days ago

Microsoft's new Florence 2 is big for Computer Vision. It's a merge between Text and Vision. With a single prompt you can instruct the model to do CV tasks like captioning, object detection, grounding, and segmentation. The best part, it only uses a single backbone to handle everything. ▸ Excels in zero-shot performance ▸ Unified model for detection, captioning, etc. ▸ FLD-5B dataset: 5B+ annotations, 126M images ▸ New benchmarks (>5.5+) on COCO, ADE20K
0:27

Microsoft's new Florence 2 is big for Computer Vision. It's a merge between Text and Vision. With a single prompt you can instruct the model to do CV tasks like captioning, object detection, grounding, and segmentation. The best part, it only uses a single backbone to handle everything. ▸ Excels in zero-shot performance ▸ Unified model for detection, captioning, etc. ▸ FLD-5B dataset: 5B+ annotations, 126M images ▸ New benchmarks (>5.5+) on COCO, ADE20K

Lior Alexander

186,560 views • 2 years ago

Announcing 𝐕𝐨𝐢𝐜𝐞𝐂𝐫𝐚𝐟𝐭🪄 SotA for both speech editing and zero-shot text-to-speech, Outperforming VALL-E, XTTS-v2, etc. VoiceCraft works on in-the-wild data such as movies, random videos and podcasts We fully open source it at
1:47

Announcing 𝐕𝐨𝐢𝐜𝐞𝐂𝐫𝐚𝐟𝐭🪄 SotA for both speech editing and zero-shot text-to-speech, Outperforming VALL-E, XTTS-v2, etc. VoiceCraft works on in-the-wild data such as movies, random videos and podcasts We fully open source it at

Puyuan Peng

160,410 views • 2 years ago

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)
2:01

A reward model that works, zero-shot, across robots, tasks, and scenes? Introducing Robometer: Scaling general-purpose robotic reward models with 1M+ trajectories. Enables zero-shot: online/offline/model-based RL, data retrieval + IL, automatic failure detection, and more! 🧵 (1/12)

Jesse Zhang @ ICML 2026 ✈️

100,947 views • 4 months ago

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box
1:35

Google announces InseRF Text-Driven Generative Object Insertion in Neural 3D Scenes paper page: InseRF generates an object in a 3D scene via a text prompt and one 2D bounding box

AK

205,987 views • 2 years ago

Today, we're releasing an open-source async inference stack for all models currently hosted on Hugging Face, powering the world's cutest robots, built with love by the team at LeRobot Details in 🧵
0:38

Today, we're releasing an open-source async inference stack for all models currently hosted on Hugging Face, powering the world's cutest robots, built with love by the team at LeRobot Details in 🧵

Francesco Capuano

24,404 views • 1 year ago

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇
1:11

✨ Introducing 𝐎𝐩𝐞𝐧𝐕𝐋𝐀 — an open-source vision-language-action model for robotics! 👐 - SOTA generalist policy - 7B params - outperforms Octo, RT-2-X on zero-shot evals 🦾 - trained on 970k episodes from OpenX dataset 🤖 - fully open: model/code/data all online 🤗 🧵👇

Moo Jin Kim

226,991 views • 2 years ago

⏰As of today, 3D Text auto Translates in-game using Localization Service 🎉 🥳I'm releasing my new plugin: "3D Text Pro" ⚡️Highlight Features: Curve 3D Text (Splines), Auto-Translate, Presets, Emojis Get It Here: 🧵Look at the thread for more information #ROBLOX #RobloxDev
0:32

⏰As of today, 3D Text auto Translates in-game using Localization Service 🎉 🥳I'm releasing my new plugin: "3D Text Pro" ⚡️Highlight Features: Curve 3D Text (Splines), Auto-Translate, Presets, Emojis Get It Here: 🧵Look at the thread for more information #ROBLOX #RobloxDev

KinqAndi

22,900 views • 1 year ago

Today we're releasing ZONOS2, our next-generation real-time TTS model with high-fidelity voice cloning. ZONOS2 is the most expressive open-source TTS model, released under Apache 2.0 and available on Zyphra Cloud on AMD. 🧵
1:30

Today we're releasing ZONOS2, our next-generation real-time TTS model with high-fidelity voice cloning. ZONOS2 is the most expressive open-source TTS model, released under Apache 2.0 and available on Zyphra Cloud on AMD. 🧵

Zyphra

332,112 views • 27 days ago

Introducing Substance 3D Viewer (beta) a new 3D desktop app that can open, view, and render 70+ file formats. In Adobe Photoshop (beta), drag & drop a 3D model on your canvas, and it becomes an editable Smart Object. Download it for free and give it a try!
0:50

Introducing Substance 3D Viewer (beta) a new 3D desktop app that can open, view, and render 70+ file formats. In Adobe Photoshop (beta), drag & drop a 3D model on your canvas, and it becomes an editable Smart Object. Download it for free and give it a try!

Adobe Substance 3D

89,134 views • 1 year ago

NeoVerse Enhancing 4D World Model with in-the-wild Monocular Videos
0:36

NeoVerse Enhancing 4D World Model with in-the-wild Monocular Videos

AK

23,598 views • 6 months ago

Object Cutter Create high-quality HD background removal for ANY object in your image with a text prompt or bounding boxes!
0:24

Object Cutter Create high-quality HD background removal for ANY object in your image with a text prompt or bounding boxes!

Gradio

135,988 views • 1 year ago

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.
0:42

Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields paper page: Editing a local region or a specific object in a 3D scene represented by a NeRF is challenging, mainly due to the implicit nature of the scene representation. Consistently blending a new realistic object into the scene adds an additional level of difficulty. We present Blended-NeRF, a robust and flexible framework for editing a specific region of interest in an existing NeRF scene, based on text prompts or image patches, along with a 3D ROI box. Our method leverages a pretrained language-image model to steer the synthesis towards a user-provided text prompt or image patch, along with a 3D MLP model initialized on an existing NeRF scene to generate the object and blend it into a specified region in the original scene. We allow local editing by localizing a 3D ROI box in the input scene, and seamlessly blend the content synthesized inside the ROI with the existing scene using a novel volumetric blending technique. To obtain natural looking and view-consistent results, we leverage existing and new geometric priors and 3D augmentations for improving the visual fidelity of the final result. We test our framework both qualitatively and quantitatively on a variety of real 3D scenes and text prompts, demonstrating realistic multi-view consistent results with much flexibility and diversity compared to the baselines. Finally, we show the applicability of our framework for several 3D editing applications, including adding new objects to a scene, removing/replacing/altering existing objects, and texture conversion.

AK

62,768 views • 3 years ago

Today, we are releasing Object Storage, an improved logs experience, and CPU + RAM monitoring for Deployments. 🧵
0:23

Today, we are releasing Object Storage, an improved logs experience, and CPU + RAM monitoring for Deployments. 🧵

Replit ⠕

50,594 views • 2 years ago

We pioneered the first ultra-realistic Text to Speech model, and recently launched the world's most accurate Speech to Text model, Scribe. But we're not stopping there. Today, we're taking one small step for man, and one giant leap for man's best friend... with Text to Bark.
1:00

We pioneered the first ultra-realistic Text to Speech model, and recently launched the world's most accurate Speech to Text model, Scribe. But we're not stopping there. Today, we're taking one small step for man, and one giant leap for man's best friend... with Text to Bark.

ElevenLabs

291,233 views • 1 year ago

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵
0:33

Today, a step forward in open robotics - our results show that sim-to-real zero shot transfer for manipulation is possible. MolmoBot is our open model suite for robotics, trained entirely in simulation on MolmoSpaces.🧵

Ai2

64,466 views • 4 months ago

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵
1:27

We’re releasing the RoboArena today!🤖🦾 Fair & scalable evaluation is a major bottleneck for research on generalist policies. We’re hoping that RoboArena can help! We provide data, model code & sim evals for debugging! Submit your policies today and join the leaderboard! :) 🧵

Karl Pertsch

111,325 views • 1 year ago

Today I'm releasing PortalVR 2.0, which lets you play the entire SteamVR ecosystem without wearing a headset. You plug in your Quest, put it on your desk, and play on your monitor. It supports 3D glasses and even 3D monitors! 😎 Get it now: Thread 🧵
0:40

Today I'm releasing PortalVR 2.0, which lets you play the entire SteamVR ecosystem without wearing a headset. You plug in your Quest, put it on your desk, and play on your monitor. It supports 3D glasses and even 3D monitors! 😎 Get it now: Thread 🧵

gfodor.id

2,318,038 views • 6 months ago

Releasing METAGENE-1: In collaboration with researchers from USC, we're open-sourcing a state-of-the-art 7B parameter Metagenomic Foundation Model. Enabling planetary-scale pathogen detection and reducing the risk of pandemics in the age of exponential biology.
2:03

Releasing METAGENE-1: In collaboration with researchers from USC, we're open-sourcing a state-of-the-art 7B parameter Metagenomic Foundation Model. Enabling planetary-scale pathogen detection and reducing the risk of pandemics in the age of exponential biology.

Prime Intellect

620,098 views • 1 year ago