Loading video...

Video Failed to Load

Go Home

Mask3D 🎭 is now at #ICRA2023, great work Jonas Schult! We use Mask Transformers for 3D Instance Segmentation on Point Clouds ~ 🥇 on ScanNet 📰Paper: 🛠️Project: 👨‍💻Code: @Pandoro89 Or Litany Siyu Tang @VLG-ETHZ

FrancisEngelman's profile picture

Francis Engelmann ☀️ @CVPR'26

1,622 subscribers

29,166 views • 3 years ago •via X (Twitter)

#ICRA2023
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

🎉 The best way to start the week is to find out that our MedSAM is finally published today in Nature Communications! **Segment anything in medical images** Paper: arXiv: Data & Code: MedSAM is the first promotable foundation model for medical image segmentation. **Highlights**: ⭐ Before its formal publication, we have received 220 citations and 1400+ GitHub stars 🙏🙏❤️‍🔥❤️‍🔥❤️‍🔥 📊 We curated a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. 🚀 Built on top of SAM (AI at Meta ) with transfer learning, we have significantly enhanced its segmentation performance of medical images. 📈 Comprehensive evaluations of 86 internal validation tasks and 60 external validation tasks demonstrate its better accuracy and robustness than modality-wise specialist models. **What is Next? --- Clinical Translation!!** 🍕Our next goal is to make the model deployable on laptops (CPUs) or other edge devices without reliance on GPUs. We have distilled a lightweight model, LiteMedSAM, offering a speed boost of 10x while maintaining accuracy. Plus, we have integrated it into the 3D Slicer plugin, providing an efficient tool for medical image segmentation. 🌐 To further promote developments in this field, we organize a competition on #CVPR2026: Segment Anything in Medical Images on Laptop! An out-of-the-box baseline has been released to reduce the entry barriers. Welcome to join us to push the boundary further: 🙏 Massive thanks to MetaAI AI at Meta for their open-source project SAM and many reviewers/users for their invaluable feedback. A huge shoutout to my postdoc Jun Ma (JunMa) for his leadership on this project!! UHN AI Hub Vector Institute Peter Munk Cardiac Centre AI Department of Laboratory Medicine & Pathobiology U of T Department of Computer Science University of Toronto University Health Network Brad Wouters 🇨🇦 Barry Rubin MD, PhD, FRCSC Shaf Keshavjee
0:59

🎉 The best way to start the week is to find out that our MedSAM is finally published today in Nature Communications! **Segment anything in medical images** Paper: arXiv: Data & Code: MedSAM is the first promotable foundation model for medical image segmentation. **Highlights**: ⭐ Before its formal publication, we have received 220 citations and 1400+ GitHub stars 🙏🙏❤️‍🔥❤️‍🔥❤️‍🔥 📊 We curated a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. 🚀 Built on top of SAM (AI at Meta ) with transfer learning, we have significantly enhanced its segmentation performance of medical images. 📈 Comprehensive evaluations of 86 internal validation tasks and 60 external validation tasks demonstrate its better accuracy and robustness than modality-wise specialist models. **What is Next? --- Clinical Translation!!** 🍕Our next goal is to make the model deployable on laptops (CPUs) or other edge devices without reliance on GPUs. We have distilled a lightweight model, LiteMedSAM, offering a speed boost of 10x while maintaining accuracy. Plus, we have integrated it into the 3D Slicer plugin, providing an efficient tool for medical image segmentation. 🌐 To further promote developments in this field, we organize a competition on #CVPR2026: Segment Anything in Medical Images on Laptop! An out-of-the-box baseline has been released to reduce the entry barriers. Welcome to join us to push the boundary further: 🙏 Massive thanks to MetaAI AI at Meta for their open-source project SAM and many reviewers/users for their invaluable feedback. A huge shoutout to my postdoc Jun Ma (JunMa) for his leadership on this project!! UHN AI Hub Vector Institute Peter Munk Cardiac Centre AI Department of Laboratory Medicine & Pathobiology U of T Department of Computer Science University of Toronto University Health Network Brad Wouters 🇨🇦 Barry Rubin MD, PhD, FRCSC Shaf Keshavjee

Bo Wang

140,336 views • 2 years ago

Introducing ✨RigidFormer: Learning Rigid Dynamics with Transformers - our attempt to scale learning-based physical dynamics with Transformers. RigidFormer learns rigid dynamics with Transformers. It is a mesh-free, object-centric Transformer for multi-object rigid-body contact dynamics from point clouds. Learning physics with purely neural simulators, without relying on traditional physics engines, is an important and widely studied problem. Prior SOTA methods often use graph neural networks for accuracy and generalization, but still struggle with efficient, high-fidelity simulation at scale. RigidFormer uses only point inputs, matches or outperforms mesh-based baselines on standard benchmarks, runs much faster, generalizes across point resolutions and datasets, and scales to 200+ objects. We also show a preliminary extension to command-conditioned articulated bodies by treating body parts as interacting object-level components. RigidFormer is mesh-free: it does not require mesh connectivity, SDFs, or vertex-level message passing, making it well-suited for point-cloud observations and scalable simulation. This architecture can also be adapted to learn soft-body dynamics by replacing the rigid-body module (differentiable Kabsch alignment). 🎬See our video for more details. Many thanks to my amazing collaborators: Minghao Guo Minghao Guo, Haixu Wu Haixu Wu 吴海旭, Doug Roble, Tuur Stuyck Tuur Stuyck, and Wojciech Matusik Wojciech Matusik. Project page: Paper:
0:26

Introducing ✨RigidFormer: Learning Rigid Dynamics with Transformers - our attempt to scale learning-based physical dynamics with Transformers. RigidFormer learns rigid dynamics with Transformers. It is a mesh-free, object-centric Transformer for multi-object rigid-body contact dynamics from point clouds. Learning physics with purely neural simulators, without relying on traditional physics engines, is an important and widely studied problem. Prior SOTA methods often use graph neural networks for accuracy and generalization, but still struggle with efficient, high-fidelity simulation at scale. RigidFormer uses only point inputs, matches or outperforms mesh-based baselines on standard benchmarks, runs much faster, generalizes across point resolutions and datasets, and scales to 200+ objects. We also show a preliminary extension to command-conditioned articulated bodies by treating body parts as interacting object-level components. RigidFormer is mesh-free: it does not require mesh connectivity, SDFs, or vertex-level message passing, making it well-suited for point-cloud observations and scalable simulation. This architecture can also be adapted to learn soft-body dynamics by replacing the rigid-body module (differentiable Kabsch alignment). 🎬See our video for more details. Many thanks to my amazing collaborators: Minghao Guo Minghao Guo, Haixu Wu Haixu Wu 吴海旭, Doug Roble, Tuur Stuyck Tuur Stuyck, and Wojciech Matusik Wojciech Matusik. Project page: Paper:

Zhiyang (Frank) Dou

576,318 views • 4 months ago

🚨 Medea Benjamin Says Code Pink Ships Thousands of Pounds to Cuba Every Week. Show Us the Receipts. At some point, “humanitarian aid” still needs a paper trail. Maybe this is lawful humanitarian work. Fine. Then prove it. Because there is a new Cuba sanctions EO on the books now, and the compliance bar is higher. Who is paying for this? Who receives it inside Cuba? Who distributes it after it lands? Are any Cuban state, party, security, or military-linked entities involved? And where does this recurring aid pipeline appear in Code Pink’s nonprofit filings?
0:58

🚨 Medea Benjamin Says Code Pink Ships Thousands of Pounds to Cuba Every Week. Show Us the Receipts. At some point, “humanitarian aid” still needs a paper trail. Maybe this is lawful humanitarian work. Fine. Then prove it. Because there is a new Cuba sanctions EO on the books now, and the compliance bar is higher. Who is paying for this? Who receives it inside Cuba? Who distributes it after it lands? Are any Cuban state, party, security, or military-linked entities involved? And where does this recurring aid pipeline appear in Code Pink’s nonprofit filings?

Stu Smith

12,875 views • 4 months ago

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.
6:20

Drivable 3D Gaussian Avatars paper page: present Drivable 3D Gaussian Avatars (D3GA), the first 3D controllable model for human bodies rendered with Gaussian splats. Current photorealistic drivable avatars require either accurate 3D registrations during training, dense input images during testing, or both. The ones based on neural radiance fields also tend to be prohibitively slow for telepresence applications. This work uses the recently presented 3D Gaussian Splatting (3DGS) technique to render realistic humans at real-time framerates, using dense calibrated multi-view videos as input. To deform those primitives, we depart from the commonly used point deformation method of linear blend skinning (LBS) and use a classic volumetric deformation method: cage deformations. Given their smaller size, we drive these deformations with joint angles and keypoints, which are more suitable for communication applications. Our experiments on nine subjects with varied body shapes, clothes, and motions obtain higher-quality results than state-of-the-art methods when using the same training and test data.

AK

327,373 views • 2 years ago

AI is changing how we work, but spreadsheets (the backbone of most companies) are lagging behind 🐢 So I spent the weekend hacking this Gemini API x Google Sheets add-on. I was blown away with how useful it is that now I use it at Google all the time 👀 Here’s how you could use it: 👩‍💼Sales & Marketing: Extract user sentiment from feedback or write hyper-personalized sales emails 👩‍⚕️Healthcare: Anonymize patient notes 🧑‍💻Software Engineering: Auto-prioritize bugs based on your company’s goals 👨‍🏫Education: Personalized feedback & trajectory warnings 👩‍💻AI Startups: Create new training sets for focused models or fine-tuning large ones 👩‍🔧Supply Chain: Translate multilingual conversations in seconds This is just scratching the surface! Download the add-on, fork the code on Github, and share it with someone who could use it to save time at work! 🚀 #AI #Gemini
1:20

AI is changing how we work, but spreadsheets (the backbone of most companies) are lagging behind 🐢 So I spent the weekend hacking this Gemini API x Google Sheets add-on. I was blown away with how useful it is that now I use it at Google all the time 👀 Here’s how you could use it: 👩‍💼Sales & Marketing: Extract user sentiment from feedback or write hyper-personalized sales emails 👩‍⚕️Healthcare: Anonymize patient notes 🧑‍💻Software Engineering: Auto-prioritize bugs based on your company’s goals 👨‍🏫Education: Personalized feedback & trajectory warnings 👩‍💻AI Startups: Create new training sets for focused models or fine-tuning large ones 👩‍🔧Supply Chain: Translate multilingual conversations in seconds This is just scratching the surface! Download the add-on, fork the code on Github, and share it with someone who could use it to save time at work! 🚀 #AI #Gemini

Liam Bolling

10,051 views • 2 years ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.
0:50

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 views • 3 years ago

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:
1:20

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:

Bingyi Kang

67,750 views • 1 year ago

Coming very soon to a 3D Printer near you. The sheer simplicity of the ammo and the lack of professional tools is a part of the demonstration so don't get too hurt by this. This is a project we have been working on for a while known as the Deep ammo series. It is a 3D printed ammunition system that uses a printed hull, powder, a primer (whether printed or normal), and a projectile. As of now we have these cycling in revolvers, and both actions but getting something like this firing semi-automatic is difficult. Not impossible though, we are hard at work and I just know you'll be seeing more of these soon.
2:13

Coming very soon to a 3D Printer near you. The sheer simplicity of the ammo and the lack of professional tools is a part of the demonstration so don't get too hurt by this. This is a project we have been working on for a while known as the Deep ammo series. It is a 3D printed ammunition system that uses a printed hull, powder, a primer (whether printed or normal), and a projectile. As of now we have these cycling in revolvers, and both actions but getting something like this firing semi-automatic is difficult. Not impossible though, we are hard at work and I just know you'll be seeing more of these soon.

Moderator Gage

35,508 views • 6 months ago

📢 Eyeline and Netflix latest research, 🌟Go-with-the-Track🌟, has been accepted to #SIGGRAPH2026. We propose a video generation framework that jointly conditions on multiple reference images and reference-anchored point tracks, enabling a range of content creation applications within a unified model: • Motion-preserving video restylization using multiple reference images; • Mesh- or keypoint-driven compositing and stylization; • Camera motion control for both static and dynamic scenes using multiple reference images captured from different view-points and time steps. ✊Congrats to the team: Koichi Namekata, Yash Kant, Zhizheng Liu, Ryan Burgert, Yuancheng Xu, Jordan Lin, Emmett Steven, Julien Philip, LiMa, Andrea Vedaldi, Paul Debevec, Ning Yu 📰 Paper: 🌐 Project: ⌨️ Code: *** This work is part of the ongoing research and development at Eyeline and Netflix, and we look forward to seeing its techniques and workflows adopted in future productions.
0:47

📢 Eyeline and Netflix latest research, 🌟Go-with-the-Track🌟, has been accepted to #SIGGRAPH2026. We propose a video generation framework that jointly conditions on multiple reference images and reference-anchored point tracks, enabling a range of content creation applications within a unified model: • Motion-preserving video restylization using multiple reference images; • Mesh- or keypoint-driven compositing and stylization; • Camera motion control for both static and dynamic scenes using multiple reference images captured from different view-points and time steps. ✊Congrats to the team: Koichi Namekata, Yash Kant, Zhizheng Liu, Ryan Burgert, Yuancheng Xu, Jordan Lin, Emmett Steven, Julien Philip, LiMa, Andrea Vedaldi, Paul Debevec, Ning Yu 📰 Paper: 🌐 Project: ⌨️ Code: *** This work is part of the ongoing research and development at Eyeline and Netflix, and we look forward to seeing its techniques and workflows adopted in future productions.

Ning Yu

28,759 views • 3 months ago

$HARMONIC just built a 3D interactive virtual world where people and AI agents can work. They can use desks to: - borrow - launch tokens - access compute - manage liquidity - trade perps, PMs, etc - and so much more Each desk is a service that users and agents can use. Over 40 desks are listed. Paid desk activity leads to buys/burns of the $HARMONIC token. These desks are split into 12 different halls in this 3D world. It’s a place agents live, work, and get paid. Every paid desk feeds the flywheel. Aristotle is Harmonic's AI built by Vlad. It is the brain behind these desks and $HARMONIC agents. Harmonic Operations's goal is to use it in as many DeFi, LP, and onchain activities as possible. And everything it does will feed the token. To date, Vlad has not followed, interacted with, or meaningfully engaged with any AI project on RH (afaik). When he sees a 3D world built with Harmonic Math as its brain, think that changes. Vlad thinks that Harmonic is the foundation for superintelligence. What happens when you point that at trading and build infrastructure to allow everyone to launch their own AI agent that can execute onchain. Few are pick up on this. Math is the mechanism.
0:21

$HARMONIC just built a 3D interactive virtual world where people and AI agents can work. They can use desks to: - borrow - launch tokens - access compute - manage liquidity - trade perps, PMs, etc - and so much more Each desk is a service that users and agents can use. Over 40 desks are listed. Paid desk activity leads to buys/burns of the $HARMONIC token. These desks are split into 12 different halls in this 3D world. It’s a place agents live, work, and get paid. Every paid desk feeds the flywheel. Aristotle is Harmonic's AI built by Vlad. It is the brain behind these desks and $HARMONIC agents. Harmonic Operations's goal is to use it in as many DeFi, LP, and onchain activities as possible. And everything it does will feed the token. To date, Vlad has not followed, interacted with, or meaningfully engaged with any AI project on RH (afaik). When he sees a 3D world built with Harmonic Math as its brain, think that changes. Vlad thinks that Harmonic is the foundation for superintelligence. What happens when you point that at trading and build infrastructure to allow everyone to launch their own AI agent that can execute onchain. Few are pick up on this. Math is the mechanism.

Chill

3,408,332 views • 9 days ago

🎓Learn how to create a powerful Torn Fabric smart material in a matter of seconds in my latest video series (AAA) Pro Tips! This smart material can be used on virtually any 3D asset. ____________________________________________________ In this video, the steps are as follows: 1. Create a base fill layer containing no information. We will use this layer to call out the core effects. Add a black mask to this layer and inside that mask add a paint layer and draw a simple pill shape. 2. Next, add a blur directional and be sure the direction is the same direction that your fabric is flowing to. 3. Add a UV border generator set to subtract to mask out any uv seams followed by an anchorpoint. Additionally, add a messy Fibers 3 fill layer set to overlay. 4. Use a levels to adjust the mask along with a sharpen filter. A warp filter should also be added to introduce some randomness. Add an anchor point at the top of the mask as well. 5. Create another fill layer with its opacity channel set to black and apply a black mask to the fill layer. Inside its mask retrieve the anchorpoint information from the previous fill layer. 6. Create an additional fill layer with a bright diffuse color along with a black mask applied to it. Add the anchorpoint information from the previous mask into its mask as well and this should give us some white fibers on the edges. Now we have a torn fabric effect wherever we paint using the paint layer created inside the callout mask! ____________________________________________________ More AAA Game Dev Tips can be found on my YouTube channel here: Stay tuned for more weekly Tips! Happy Texturing!💚 #gamedev #gameart #tutorial #3dmodeling #hardsurface #texturing
0:59

🎓Learn how to create a powerful Torn Fabric smart material in a matter of seconds in my latest video series (AAA) Pro Tips! This smart material can be used on virtually any 3D asset. ____________________________________________________ In this video, the steps are as follows: 1. Create a base fill layer containing no information. We will use this layer to call out the core effects. Add a black mask to this layer and inside that mask add a paint layer and draw a simple pill shape. 2. Next, add a blur directional and be sure the direction is the same direction that your fabric is flowing to. 3. Add a UV border generator set to subtract to mask out any uv seams followed by an anchorpoint. Additionally, add a messy Fibers 3 fill layer set to overlay. 4. Use a levels to adjust the mask along with a sharpen filter. A warp filter should also be added to introduce some randomness. Add an anchor point at the top of the mask as well. 5. Create another fill layer with its opacity channel set to black and apply a black mask to the fill layer. Inside its mask retrieve the anchorpoint information from the previous fill layer. 6. Create an additional fill layer with a bright diffuse color along with a black mask applied to it. Add the anchorpoint information from the previous mask into its mask as well and this should give us some white fibers on the edges. Now we have a torn fabric effect wherever we paint using the paint layer created inside the callout mask! ____________________________________________________ More AAA Game Dev Tips can be found on my YouTube channel here: Stay tuned for more weekly Tips! Happy Texturing!💚 #gamedev #gameart #tutorial #3dmodeling #hardsurface #texturing

Cohen Brawley

78,888 views • 2 years ago

We told Wes Gray we might be in a bubble. His response about what we can do about it was blunt. "Great, that sucks. Thanks for telling me that. the question is can we can we use that information to actually make us better? And the fortunate answer, as you guys already know, is no. Because if you try to do any sort of model that tries to time based on valuation, they don't work. It's just the fact. It's empirical data. It says good luck." "It's just like being a great short seller. It's not about picking the stock that we all know is gonna go to zero, it's about doing it at the right time, and that's actually, it turns out to be insanely difficult." "Right now, 'cause obviously we have extremely high prices or low earnings yield, I would expect to earn low returns on my overall portfolio in the next 20, 30 years."
1:50

We told Wes Gray we might be in a bubble. His response about what we can do about it was blunt. "Great, that sucks. Thanks for telling me that. the question is can we can we use that information to actually make us better? And the fortunate answer, as you guys already know, is no. Because if you try to do any sort of model that tries to time based on valuation, they don't work. It's just the fact. It's empirical data. It says good luck." "It's just like being a great short seller. It's not about picking the stock that we all know is gonna go to zero, it's about doing it at the right time, and that's actually, it turns out to be insanely difficult." "Right now, 'cause obviously we have extremely high prices or low earnings yield, I would expect to earn low returns on my overall portfolio in the next 20, 30 years."

Excess Returns

34,311 views • 1 month ago

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:
1:31

BREAKING NEWS: Anthropic just dropped Claude Ops 4.5!! It is by FAR the best coding model I've ever used. We've been testing it internally Every 📧 for the last few days, and it is an absolute paradigm shift for any kind of coding task. It extends the horizon of what you can vibe code The current generation of new models—Anthropic’s Sonnet 4.5, Google’s Gemini 3, or OpenAI’s Codex Max 5.1—can all competently build a minimum viable product in one shot, or fix a highly technical bug autonomously. But eventually, if you kept pushing them to vibe code more, they’d start to trip over their own feet: The code would be convoluted and contradictory, and you’d get stuck in endless bugs. We have not found that limit yet with Opus 4.5—it seems to be able to vibe code forever. Takes working in parallel to a whole new level because it's far better at planning and coding, it can work with more autonomy—meaning you can do more in parallel without breaking anything . Kieran Klaassen worked on 11 different projects in six hours—and had good results on all of them. Great at design iteration Opus 4.5 is incredibly skilled at iterating through a design autonomously using an MCP like Playwright. previous models would lose the thread after a few cycles, or say a design was done when it wasn't. Opus 4.5 is incredible at autonomously iterating until a design is pixel perfect. we have a full 4,000 word vibe check on Every 📧 right now with everything we tested:

Dan Shipper 📧

272,699 views • 10 months ago

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal
0:39

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,347 views • 1 year ago

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!
0:29

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Bilawal Sidhu

704,409 views • 1 year ago