Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

The robot’s “eyes” just received a big upgrade. LingBot-Depth 2.0, a depth-completion model with half the depth error just dropped. 12/16 benchmarks topped. Glass, mirrors, and transparent objects are so easy for us humans, but so hard for robots, because they do not behave like ordinary surfaces in a...

rohanpaul_ai's profile picture

Rohan Paul

152,710 subscribers

25,312 Aufrufe • vor 1 Monat •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

🤖 Ever seen a robot get confused by a glass door? That era is ending. Today, we're open-sourcing LingBot-Depth, a new spatial perception model that lets robots truly *see* the physical world, including transparent & reflective objects. It even surpasses top-tier industrial depth cameras in core metrics. A new sense of sight for embodied AI. #EmbodiedAI #Robotics #ComputerVision #OpenSource #DepthCamera
2:37

🤖 Ever seen a robot get confused by a glass door? That era is ending. Today, we're open-sourcing LingBot-Depth, a new spatial perception model that lets robots truly *see* the physical world, including transparent & reflective objects. It even surpasses top-tier industrial depth cameras in core metrics. A new sense of sight for embodied AI. #EmbodiedAI #Robotics #ComputerVision #OpenSource #DepthCamera

Robbyant

320,159 Aufrufe • vor 7 Monaten

Very excited to share our first public release after I joined Robbyant! We present Lingbot-Depth 👀 — a state-of-the-art depth foundation model trained with RGB-D MAE on millions of real & simulated RGBD pairs. 🔹 Camera depths as natural masks for RGB-D MAE modeling 🔹 Large-scale real + sim RGBD data curation pipeline What surprised us most: ✨ Significant improvement on transparent, reflective surfaces, and thin structures — traditionally the hardest cases for depth models. 👇These are some of hard causal-capture images we tested on. The results are pretty good! #DepthEstimation #FoundationModels #EmbodiedAI #3DVision #Robotics #RobbyAnt
0:34

Very excited to share our first public release after I joined Robbyant! We present Lingbot-Depth 👀 — a state-of-the-art depth foundation model trained with RGB-D MAE on millions of real & simulated RGBD pairs. 🔹 Camera depths as natural masks for RGB-D MAE modeling 🔹 Large-scale real + sim RGBD data curation pipeline What surprised us most: ✨ Significant improvement on transparent, reflective surfaces, and thin structures — traditionally the hardest cases for depth models. 👇These are some of hard causal-capture images we tested on. The results are pretty good! #DepthEstimation #FoundationModels #EmbodiedAI #3DVision #Robotics #RobbyAnt

Yinghao Xu

22,664 Aufrufe • vor 7 Monaten

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion
2:29

In collaboration with Intel, our Depth Fusion showcases the power of our LDM3D diffusion model in generating 360° views from text prompts provided by the user. The LDM3D diffusion model generates a 2D RGB image and its corresponding relative depth map providing a complete RGBD representation corresponding to the text prompt. The LDM 3D model is a specialized version of the stable diffusion V 1.4 model that has been modified to fit both image and depth map data.The model was then fine tuned on a subset of the Laion400M data set - large scale image caption data set. The depth maps used to fine tune our model were generated by the DPTBeiT large 512 depth estimation model that provides highly accurate relative depth estimates for each pixel. We take the generated 2D RGB image and depth map and use them to compute a 360° projection using touchdesigner. Touchdesigner is a versatile platform that allows for the creation of immersive and interactive multimedia experiences. Our application harnesses the power of touchdesigner to bring the generated 360° views to life, providing users with a unique and engaging way to experience their text prompts, whether it’s a description of a tranquil forest, a noisy cityscape or a futuristic sci fi world. Our depth fusion can bring these concepts to life in a vivid and immersive detail. - Scottie Fox, VP Engineering Blockade Labs ScottieFox #AI #VR #3D #gamedev #stablediffusion

Blockade Labs

11,439 Aufrufe • vor 3 Jahren

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:
1:20

Want to use Depth Anything, but need metric depth rather than relative depth? Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution. 👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided. Prompt Depth Anything offers 1⃣A series of models for iPhone lidars. 2⃣4D reconstruction from monocular videos (captured with iPhone). 3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses. 4⃣More detailed depth annotations for the ScanNet++ dataset. The first author is our excellent intern Haotong Lin. Paper: Huggingface: Project Page: Code:

Bingyi Kang

67,706 Aufrufe • vor 1 Jahr

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀 One universal model enables SoTA for: 🔥 Mono Depth Estimation 🔥 Multi-View SfM 🔥 Multi-View Stereo 🔥 Depth Completion 🔥 Registration … and many more possibilities! – plus everything is metric 🎯 We release code for data processing, training, benchmarking & ablations – everything Apache 2.0! Details & Links 👇
1:30

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀 One universal model enables SoTA for: 🔥 Mono Depth Estimation 🔥 Multi-View SfM 🔥 Multi-View Stereo 🔥 Depth Completion 🔥 Registration … and many more possibilities! – plus everything is metric 🎯 We release code for data processing, training, benchmarking & ablations – everything Apache 2.0! Details & Links 👇

Nikhil Keetha

123,125 Aufrufe • vor 11 Monaten

I Just Played PlayStation Games In Spatial Depth on Vision Pro And … WOW🤯 Vision Pro Developers Are Just DIFFERENT 🔥Shoutout Vincent 👏🏾 Asobi: Remote Play just dropped a huge update that adds LIVE spatial depth conversion which means you can play your PlayStation games in 3D🤩😭 No New Game Patches Or Game Updates Are Needed‼️ Asobi: PlayStation Remote Play Does All The Work⚡️ Now every game is like a window into another world on Vision Pro & I’m truly BLOWN AWAY💥 Stop Sleeping On Vision Pro & Check This Out Today 🎮
1:14

I Just Played PlayStation Games In Spatial Depth on Vision Pro And … WOW🤯 Vision Pro Developers Are Just DIFFERENT 🔥Shoutout Vincent 👏🏾 Asobi: Remote Play just dropped a huge update that adds LIVE spatial depth conversion which means you can play your PlayStation games in 3D🤩😭 No New Game Patches Or Game Updates Are Needed‼️ Asobi: PlayStation Remote Play Does All The Work⚡️ Now every game is like a window into another world on Vision Pro & I’m truly BLOWN AWAY💥 Stop Sleeping On Vision Pro & Check This Out Today 🎮

Mr. Getting Things Dun

13,332 Aufrufe • vor 4 Monaten

We just tested what might be the most accurate motion transfer workflow we've seen so far. It's a little more involved than a standard workflow, but the results are pretty impressive. Here's the basic workflow: 1. Start with your reference footage. Create character and environment references. 2. Convert the reference footage into a depth map (with Depth-Anything tool on Fal: 3. (Optional) Layer on additional guidance like colorized character masks (with Meta's Segment Anything: and or skeleton poses (with 4. Feed the depth map into Seedance 2.0 Omni as your motion reference along with your character and environment references. 5. Generate your final video. After testing several variations, the depth map workflow consistently produced the cleanest and most accurate motion transfer. If you're trying to recreate specific performances or complex movement, this is one of the closest AI workflows we've seen to achieving production-quality results.
11:34

We just tested what might be the most accurate motion transfer workflow we've seen so far. It's a little more involved than a standard workflow, but the results are pretty impressive. Here's the basic workflow: 1. Start with your reference footage. Create character and environment references. 2. Convert the reference footage into a depth map (with Depth-Anything tool on Fal: 3. (Optional) Layer on additional guidance like colorized character masks (with Meta's Segment Anything: and or skeleton poses (with 4. Feed the depth map into Seedance 2.0 Omni as your motion reference along with your character and environment references. 5. Generate your final video. After testing several variations, the depth map workflow consistently produced the cleanest and most accurate motion transfer. If you're trying to recreate specific performances or complex movement, this is one of the closest AI workflows we've seen to achieving production-quality results.

Curious Refuge

30,397 Aufrufe • vor 1 Monat

Standards are set to be met.....to guide, and shield us from certain mistakes, and irrational behaviors. As an individual, if you stand for nothing you fall for everything.....so your STANDARD is your WATCHWORD. Allow people stand on the business that pays them. Allow people wear the hair that fits them, the sunglasses that shield them, especially when they go out for "MEDICALS" As beauty is in the eyes of the beholder so is "DEPTH". In a entertainment reality show, where PERSONALITY is the focus, some have decided to mine "DEPTH OIL". Trust me all cards must be dealt, but assassinating the character of a player when you have equal access to play than be a paranoid commentator is crazy work. We went from; No depth Insecure Not all that Dumb girl syndrome And now Sugar baby So to the CEO DEPTH OIL LTD, we plead that you allow the NON-DEPTH breathe!!!!!🥹🥹🥹🥹 And it's NEPO BABY not SUGAR BABY. 🎥: Big Brother Naija MAIN CHARACTER DEDE #DedeAshiogwu #BBNaija
0:55

Standards are set to be met.....to guide, and shield us from certain mistakes, and irrational behaviors. As an individual, if you stand for nothing you fall for everything.....so your STANDARD is your WATCHWORD. Allow people stand on the business that pays them. Allow people wear the hair that fits them, the sunglasses that shield them, especially when they go out for "MEDICALS" As beauty is in the eyes of the beholder so is "DEPTH". In a entertainment reality show, where PERSONALITY is the focus, some have decided to mine "DEPTH OIL". Trust me all cards must be dealt, but assassinating the character of a player when you have equal access to play than be a paranoid commentator is crazy work. We went from; No depth Insecure Not all that Dumb girl syndrome And now Sugar baby So to the CEO DEPTH OIL LTD, we plead that you allow the NON-DEPTH breathe!!!!!🥹🥹🥹🥹 And it's NEPO BABY not SUGAR BABY. 🎥: Big Brother Naija MAIN CHARACTER DEDE #DedeAshiogwu #BBNaija

DEDE ASHIOGWU🌻

305,005 Aufrufe • vor 1 Jahr

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.
0:54

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

Blaine Brown

1,468,678 Aufrufe • vor 2 Jahren

Humanoid Robots Are Entering China’s Community Pharmacies When an online order arrives, the humanoid robot autonomously locates and navigates to the shelf, picks up the medicine, and delivers it to the packing station. The store clerk then packs it for the customer. This is a real deployment at a Guoda Pharmacy in Shanghai’s Pudong district. They’ve introduced Ant’s Robbyant wheeled humanoid robot R2, powered by the newly released LingBot VLA 2.0 + LingBot Depth, and integrated with online ordering platforms such as Meituan, Guoda APP(the brand)… Customers get their medicine faster, while pharmacists and staff have more time for professional medication consultations--especially valuable during night shifts when manpower is limited. Robbyant plans to deploy these robots in 20–50 pharmacies this year and expand to 300 stores by 2027. --- The real-world deployment flywheel --where actual usage generates more data to strengthen the models--is spinning nicely here.
0:57

Humanoid Robots Are Entering China’s Community Pharmacies When an online order arrives, the humanoid robot autonomously locates and navigates to the shelf, picks up the medicine, and delivers it to the packing station. The store clerk then packs it for the customer. This is a real deployment at a Guoda Pharmacy in Shanghai’s Pudong district. They’ve introduced Ant’s Robbyant wheeled humanoid robot R2, powered by the newly released LingBot VLA 2.0 + LingBot Depth, and integrated with online ordering platforms such as Meituan, Guoda APP(the brand)… Customers get their medicine faster, while pharmacists and staff have more time for professional medication consultations--especially valuable during night shifts when manpower is limited. Robbyant plans to deploy these robots in 20–50 pharmacies this year and expand to 300 stores by 2027. --- The real-world deployment flywheel --where actual usage generates more data to strengthen the models--is spinning nicely here.

CyberRobo

14,224 Aufrufe • vor 1 Monat

Six open-source LLMs. One sliding puzzle. A brutal test of long-horizon reasoning and tool calling. Five of them broke. One didn't. I gave each model a move_tile tool and a scrambled 3×3 board, then asked it to solve the puzzle through pure turn-by-turn reasoning. The deeper the scramble, the more brutal the search. Five runs per depth, best run kept. A model fails the round if it exceeds 6x the optimal move count. > Depth 5: Everyone solves it. Yawn. > Depth 10: GLM 5.1 melts down. 43 moves. Cut. > Depth 12: Gemma4 26B loses the plot, shuffling tiles in circles. Gone. > Depth 15: The wall. DeepSeek V4 Flash, out. DeepSeek V4 Pro, out. Gemma4, out again. GLM 5.1, out. Two survivors: Qwen3.6 35B-A3B, and Kimi K2.6 with an 11-move solve that looked like cheating. > Depth 18: Same two. Everyone else hallucinating tiles that weren't there. > Depth 22: Final boss. Kimi, flawless for five rounds, finally cracks. 81 moves. Still scrambled. DeepSeek V4 Pro limps home at 90. Qwen3.6 35B-A3B solves it in 36. The smallest model in the room. ~3B active params. Fits on a single 3090. It beat everything. Kimi was elegant. Qwen3.6 was unstoppable.
2:00

Six open-source LLMs. One sliding puzzle. A brutal test of long-horizon reasoning and tool calling. Five of them broke. One didn't. I gave each model a move_tile tool and a scrambled 3×3 board, then asked it to solve the puzzle through pure turn-by-turn reasoning. The deeper the scramble, the more brutal the search. Five runs per depth, best run kept. A model fails the round if it exceeds 6x the optimal move count. > Depth 5: Everyone solves it. Yawn. > Depth 10: GLM 5.1 melts down. 43 moves. Cut. > Depth 12: Gemma4 26B loses the plot, shuffling tiles in circles. Gone. > Depth 15: The wall. DeepSeek V4 Flash, out. DeepSeek V4 Pro, out. Gemma4, out again. GLM 5.1, out. Two survivors: Qwen3.6 35B-A3B, and Kimi K2.6 with an 11-move solve that looked like cheating. > Depth 18: Same two. Everyone else hallucinating tiles that weren't there. > Depth 22: Final boss. Kimi, flawless for five rounds, finally cracks. 81 moves. Still scrambled. DeepSeek V4 Pro limps home at 90. Qwen3.6 35B-A3B solves it in 36. The smallest model in the room. ~3B active params. Fits on a single 3090. It beat everything. Kimi was elegant. Qwen3.6 was unstoppable.

stevibe

16,434 Aufrufe • vor 3 Monaten

🚨 BREAKING: Big news in the computer vision world! 🎥 Luxonis | Robotic Vision just dropped its new OAK 4 line, and it’s a big upgrade for edge computer vision. Instead of being “just a stereo camera,” OAK 4 is a fully standalone vision computer with 52 TOPS of on-device AI. Models run locally, depth is computed locally, and no external PC or cloud pipeline is required. This is why robotics teams love it: lower latency, lower cost, fewer failure points in the field. The hardware is built for the real-world. IP67, shock-resistant, wide-FOV RGB + stereo pair, IR projection, IMU, audio, and a patent-pending calibration system that keeps depth accurate even when conditions change. But the real move is the platform. With Luxonis Hub, you can deploy models, grab telemetry, push OTA updates, or collect data when performance drifts, all from a unified interface. It turns a single device into an end-to-end edge CV system. Most customers today in robotics are groups who just want something that works: AMRs, bin-picking systems, trailer-loading robots, and ag-tech. 🤖 And they all say the same thing, the appeal isn’t raw TOPS, it’s the all-in-one simplicity that lets them scale without building custom infrastructure. Feels like the direction edge vision has been waiting for: rugged hardware + high-throughput on-device compute + a real management layer. A next step toward “plug-and-deploy” perception for robots. 🔗 Find out more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
1:12

🚨 BREAKING: Big news in the computer vision world! 🎥 Luxonis | Robotic Vision just dropped its new OAK 4 line, and it’s a big upgrade for edge computer vision. Instead of being “just a stereo camera,” OAK 4 is a fully standalone vision computer with 52 TOPS of on-device AI. Models run locally, depth is computed locally, and no external PC or cloud pipeline is required. This is why robotics teams love it: lower latency, lower cost, fewer failure points in the field. The hardware is built for the real-world. IP67, shock-resistant, wide-FOV RGB + stereo pair, IR projection, IMU, audio, and a patent-pending calibration system that keeps depth accurate even when conditions change. But the real move is the platform. With Luxonis Hub, you can deploy models, grab telemetry, push OTA updates, or collect data when performance drifts, all from a unified interface. It turns a single device into an end-to-end edge CV system. Most customers today in robotics are groups who just want something that works: AMRs, bin-picking systems, trailer-loading robots, and ag-tech. 🤖 And they all say the same thing, the appeal isn’t raw TOPS, it’s the all-in-one simplicity that lets them scale without building custom infrastructure. Feels like the direction edge vision has been waiting for: rugged hardware + high-throughput on-device compute + a real management layer. A next step toward “plug-and-deploy” perception for robots. 🔗 Find out more here: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

41,955 Aufrufe • vor 8 Monaten

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.
2:00

Meta presents Sapiens Foundation for Human Vision Models discuss: We present Sapiens, a family of models for four fundamental human-centric vision tasks - 2D pose estimation, body-part segmentation, depth estimation, and surface normal prediction. Our models natively support 1K high-resolution inference and are extremely easy to adapt for individual tasks by simply fine-tuning models pretrained on over 300 million in-the-wild human images. We observe that, given the same computational budget, self-supervised pretraining on a curated dataset of human images significantly boosts the performance for a diverse set of human-centric tasks. The resulting models exhibit remarkable generalization to in-the-wild data, even when labeled data is scarce or entirely synthetic. Our simple model design also brings scalability - model performance across tasks improves as we scale the number of parameters from 0.3 to 2 billion. Sapiens consistently surpasses existing baselines across various human-centric benchmarks. We achieve significant improvements over the prior state-of-the-art on Humans-5K (pose) by 7.6 mAP, Humans-2K (part-seg) by 17.1 mIoU, Hi4D (depth) by 22.4% relative RMSE, and THuman2 (normal) by 53.5% relative angular error.

AK

151,614 Aufrufe • vor 2 Jahren

With all due respect to my friend Doomer, the reason they didn’t use selective focus on the Rocky stair shot is because they couldn’t. The tech wasn’t there yet. Rocky was the first film the use a Steadicam. This was groundbreaking tech and changed cinematic language forever. That's the reason the stair scene is so iconic--because audiences had never seen anything like it before. If we were doing that shot today, we’d have a focus puller (called a 1st AC in the USA) manually focusing the lens on a wireless follow focus. Because there’s no other way to pull focus with a fast moving steadi cam following a fast moving subject. With a shallow depth of field Rocky would have been running in and out of focus. But back then they had no choice. So they used the largest depth of field they could get away with so that Rocky stayed in focus the entire time. If they’d had the tech to do selective focus they 100% would have used it--because cinema is all about selective focus. Steadicam inventor Garret Brown actually shot a test with his GF running on the same stairs used in Rocky (see video). And again, he was forced to use a deep depth of field because there was no way to accurately focus the camera while he was moving.
0:53

With all due respect to my friend Doomer, the reason they didn’t use selective focus on the Rocky stair shot is because they couldn’t. The tech wasn’t there yet. Rocky was the first film the use a Steadicam. This was groundbreaking tech and changed cinematic language forever. That's the reason the stair scene is so iconic--because audiences had never seen anything like it before. If we were doing that shot today, we’d have a focus puller (called a 1st AC in the USA) manually focusing the lens on a wireless follow focus. Because there’s no other way to pull focus with a fast moving steadi cam following a fast moving subject. With a shallow depth of field Rocky would have been running in and out of focus. But back then they had no choice. So they used the largest depth of field they could get away with so that Rocky stayed in focus the entire time. If they’d had the tech to do selective focus they 100% would have used it--because cinema is all about selective focus. Steadicam inventor Garret Brown actually shot a test with his GF running on the same stairs used in Rocky (see video). And again, he was forced to use a deep depth of field because there was no way to accurately focus the camera while he was moving.

Rambo Van Halen

20,578 Aufrufe • vor 9 Monaten

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.
1:17

Most video-action robot models are a content-creation video generator with an action module attached. LingBot-VA 2.0 from Robbyant, a video-action foundation model, throws that starting point out and trains the whole stack natively for control. And it runs closed-loop at a peak 225 Hz. It's so important because A robot cannot move responsively when its controller pauses to imagine the next few frames. LingBot-VA 2.0 predicts during execution, then corrects using each real observation. And it carries only about 13B video parameters while activating roughly 1.9B per token. Bigger robot models usually mean slower reactions, creating a direct conflict between intelligence and control. LingBot-VA 2.0 is trained from scratch for robot control rather than adapted from a video generator built for content creation. Robbyant, an embodied AI company under Ant Group, built it to learn how scenes change under actions, predict what should happen next, and turn those predictions into real-time robot movements. Most video-action systems inherit a tokenizer and video backbone trained mainly to reproduce visual appearance. LingBot-VA 2.0 rebuilds both parts around physical control. Its semantic visual-action tokenizer maps observations toward features from a frozen vision foundation model and learns compact latent actions from frame-to-frame changes using self-supervised inverse and forward dynamics. Unlabeled web video can therefore carry action-relevant training signals without robot action labels. The policy is causal from the start, so every prediction can use only past observations. Its sparse Mixture-of-Experts video backbone has about 13B total parameters, while about 1.9B are active per token, keeping the compute lower during each step. A high-level vision-language planner breaks long tasks into smaller instructions, while the low-level video-action policy handles continuous movement. Foresight Reasoning predicts future visual states while the robot is already acting, then replaces imagined states with every new real observation. Combined with few-step distillation and systems acceleration, the paper reports a peak asynchronous execution frequency of 225 Hz. The model adapts from 10–15 demonstrations, transfers across robot embodiments, and handles some new tasks zero-shot. In the paper’s own evaluations, it reaches 93.6 average on RoboTwin 2.0 and reports stronger real-world results than LingBot-VA and π0.5 across the tested tasks. 🧵 1.

Rohan Paul

11,253 Aufrufe • vor 1 Monat

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:47

Trained a humanoid entirely in a 3D scan of the office. Zero real-world fine-tuning. It just walked in and worked. RL needs hundreds of thousands of attempts, and real robots can't afford to crash. A misjudged gap or a glass door collision breaks hardware and costs hours resetting. So you train in a sim. But sim policies usually train on randomized, untextured geometry; depth is easy to fake. The robot learns structure, not the real world: no materials, no lighting, no idea what anything actually is. RGB cameras carry all of that but training RGB policies in generic fake worlds won’t generalize to the real world. Niantic Spatial 🌎 Scaniverse reconstructs your scan of the real deployment site. One 360° camera walkthrough → photorealistic 3D Gaussian splat at metric scale → collision mesh pulled from the same reconstruction, so vision and physics match exactly. Drops straight into NVIDIA Isaac Sim/Lab, no manual conversion. Flexion simulation-first approach then seamlessly enables the training of RGB-only nav policies inside that reconstruction. With added domain randomization + large image encoders for robustness, this deploys straight to hardware. No real-world fine-tuning. Deployment: months of on-site adaptation → days. Tune into the NVIDIA livestream on 12 August to hear how these companies are closing the sim2real gap: NVIDIA Robotics ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

119,091 Aufrufe • vor 20 Tagen

🎙️ Charles Lee on LaMelo Ball's and Brandon Miller's minute restrictions: "We'll see how they recover after their first back-to-back in a while." "You just have to keep the process, the long-term vision, in mind even when you're in these moments. We want to win this game, obviously—every game we want to win really badly—and those two guys obviously help you do it." "But we do have really good depth, and so you lean on that depth in moments like this, and the maturity of those two guys to understand that there is a bigger picture and trying to maximize their availability. They did a good job of staying engaged even if they had to sit out for longer stretches than they're used to, and then when they came in the game, they had an impact."
1:52

🎙️ Charles Lee on LaMelo Ball's and Brandon Miller's minute restrictions: "We'll see how they recover after their first back-to-back in a while." "You just have to keep the process, the long-term vision, in mind even when you're in these moments. We want to win this game, obviously—every game we want to win really badly—and those two guys obviously help you do it." "But we do have really good depth, and so you lean on that depth in moments like this, and the maturity of those two guys to understand that there is a bigger picture and trying to maximize their availability. They did a good job of staying engaged even if they had to sit out for longer stretches than they're used to, and then when they came in the game, they had an impact."

r/CharlotteHornets

26,573 Aufrufe • vor 9 Monaten

New paper from Apple - Sharp Monocular View Synthesis in Less than a Second Mescheder et al. @ Apple just released a very impressive paper (congrats! 🎉🥳). You give it an image and it generates a really great looking 3d Gaussian representation. Uses depth pro. It's really good. The model is about 3GB, and takes ~5-10s on my M1 Max per image. Single feedforward pass network. The video is using "Metal Splatter" to view the .ply from ml-sharp on Apple Vision Pro. Some really wow moments today trying it on different scenes.
0:22

New paper from Apple - Sharp Monocular View Synthesis in Less than a Second Mescheder et al. @ Apple just released a very impressive paper (congrats! 🎉🥳). You give it an image and it generates a really great looking 3d Gaussian representation. Uses depth pro. It's really good. The model is about 3GB, and takes ~5-10s on my M1 Max per image. Single feedforward pass network. The video is using "Metal Splatter" to view the .ply from ml-sharp on Apple Vision Pro. Some really wow moments today trying it on different scenes.

Tim Davison ᯅ

133,001 Aufrufe • vor 8 Monaten

.Bridgit Mendler on the founding thesis of Northwood. "We really were recognizing the same problem, that networking for the space economy is really what drives its value. And there's a significant bottleneck, and that bottleneck actually is not in space. It's actually on Earth. The way that you actually solve that networking problem for space is through introducing a model that gets you to the scale that you need, not just for one individual company or a mission, but for the industry overall." Full "Valley of Depth" episode in comments 👇
2:30

.Bridgit Mendler on the founding thesis of Northwood. "We really were recognizing the same problem, that networking for the space economy is really what drives its value. And there's a significant bottleneck, and that bottleneck actually is not in space. It's actually on Earth. The way that you actually solve that networking problem for space is through introducing a model that gets you to the scale that you need, not just for one individual company or a mission, but for the industry overall." Full "Valley of Depth" episode in comments 👇

Mo Islam

140,491 Aufrufe • vor 2 Monaten

For the past 3 months, I've been heads down building and extending our multimodal data capture rig for Physical AI training. What you're seeing here is a single synchronized capture from our system — RGB, depth (stereo IR camera) feed; real-time 3D hand reconstruction, and a 23×20 tactile pressure grid from sensor gloves, all aligned and timestamped together. Getting here took a lot of unglamorous iteration. We reworked the pipeline to deliver clean, gap-free depth at scale after identifying early depth frame drops caused by data throughput bottlenecks in our capture box. We built per-channel calibration models that convert raw resistance signals from our flex sensor gloves into accurate, drift-corrected force values — solving the creep behavior that made a steady grip falsely appear to fade over time. Even though we’ve stayed quiet on socials, word of mouth spread — our rigs have already been deployed in data factories, manufacturing lines, and homes across the US, Europe, China, and Southeast Asia by our customers and ourselves. In the next few days, I'll be sharing more about how we're tackling rights-cleared data capture from the hardware level, and our closed-loop data-model flywheel — using model evaluation results to identify gaps and guide what data our customers need. More soon, stay tuned…
0:26

For the past 3 months, I've been heads down building and extending our multimodal data capture rig for Physical AI training. What you're seeing here is a single synchronized capture from our system — RGB, depth (stereo IR camera) feed; real-time 3D hand reconstruction, and a 23×20 tactile pressure grid from sensor gloves, all aligned and timestamped together. Getting here took a lot of unglamorous iteration. We reworked the pipeline to deliver clean, gap-free depth at scale after identifying early depth frame drops caused by data throughput bottlenecks in our capture box. We built per-channel calibration models that convert raw resistance signals from our flex sensor gloves into accurate, drift-corrected force values — solving the creep behavior that made a steady grip falsely appear to fade over time. Even though we’ve stayed quiet on socials, word of mouth spread — our rigs have already been deployed in data factories, manufacturing lines, and homes across the US, Europe, China, and Southeast Asia by our customers and ourselves. In the next few days, I'll be sharing more about how we're tackling rights-cleared data capture from the hardware level, and our closed-loop data-model flywheel — using model evaluation results to identify gaps and guide what data our customers need. More soon, stay tuned…

Grace Zhang

54,115 Aufrufe • vor 2 Monaten