Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

"AgentSTAR: Agentic Shape Tracking and Reconstruction from Monocular Videos" VLM agent+optimization to reconstruct an object geometry+kinematic structure from RGB video. Pose+articulation through time including occlusions, transparent objects. No dense pixel correspondences.

Almorgand's profile picture

Alexandre Morgand

2,343 subscribers

24,762 Aufrufe • vor 6 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Alexandre Morgand
Alexandre Morgandvor 6 Tagen

, @n_karaev, Matthew Chang, @JitendraMalikCV, @notmahi @amazon FAR (Frontier AI and Robotics) Project page: Paper: Code:

Profilbild von Alexandre Morgand
Alexandre Morgandvor 6 Tagen

@n_karaev @JitendraMalikCV @notmahi @amazon original thread:

Profilbild von Prince Kushwaha
Prince Kushwahavor 6 Tagen

Ditching dense point tracking for an agent that scripts articulated primitives and visually verifies them solves the transparent and thin object failure mode.

Ähnliche Videos

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw
0:42

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames - correspondences as nearest neighbors - no motion or deformation estimation Stable geometry and tracking, even under large expressions and viewpoint changes - check out our results! 🌐 ▶️ Great work by Umut Kocasarı, Simon Giebenhain, Richard Shaw

Matthias Niessner

61,881 Aufrufe • vor 5 Monaten

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"
0:53

"near real-time method for 6-DoF tracking of an unknown object from a monocular RGBD video sequence [...] a Neural Object Field that is learned concurrently with a pose graph optimization process in order to robustly accumulate information into a consistent 3D representation"

Fabien Benetou

222,937 Aufrufe • vor 3 Jahren

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.
0:45

HDMI (HumanoiD iMitation for Interaction) is a framework enabling humanoid robots to learn whole-body object interaction skills from monocular RGB human videos. It extracts and retargets human poses and object trajectories using GVHMR and LocoMujoco, building reference datasets with contact annotations. The data is used to train an RL policy via robot-object co-tracking. HDMI achieved 67 consecutive door traversals.

The Humanoid Hub

17,395 Aufrufe • vor 1 Jahr

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.
5:18

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

AK

280,570 Aufrufe • vor 3 Jahren

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.
0:23

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.

MrNeRF

29,974 Aufrufe • vor 1 Jahr

A video is compressed 4D data. 🌍 Today, we’re sharing a preview of our latest research: 4D reconstruction from a single monocular video. 🎬 From one casually captured video, our system reconstructs a dynamic, freely viewable 4D Gaussian scene—recovering geometry, appearance, camera motion, and object motion across space and time. No multi-camera rig. Just video → 4DGS. The world has already been captured in video. The next step is to unlock the 4D structure inside it. Watch the result below. 👇 #4DGS #GaussianSplatting #CVPR #CVPR2026 #AIGC #AI #3DGS
0:49

A video is compressed 4D data. 🌍 Today, we’re sharing a preview of our latest research: 4D reconstruction from a single monocular video. 🎬 From one casually captured video, our system reconstructs a dynamic, freely viewable 4D Gaussian scene—recovering geometry, appearance, camera motion, and object motion across space and time. No multi-camera rig. Just video → 4DGS. The world has already been captured in video. The next step is to unlock the 4D structure inside it. Watch the result below. 👇 #4DGS #GaussianSplatting #CVPR #CVPR2026 #AIGC #AI #3DGS

InSpatio｜影溯

64,567 Aufrufe • vor 2 Monaten

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.
5:19

4DGT: Learning a 4D Gaussian Transformer Using Real-World Monocular Videos Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We introduced a novel density control strategy in training, which allows our 4DGT to handle longer space-time input while maintaining efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can significantly outperform prior Gaussian-based networks in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos.

MrNeRF

34,782 Aufrufe • vor 1 Jahr

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.
0:26

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 Aufrufe • vor 3 Jahren

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:
2:30

Can we scale 4D pretraining to learn general space-time representations that reconstruct an object from a few views at any time to any view at any other time? Introducing 4D-LRM: a Large Space-Time Reconstruction Model that ... 🔹 Predicts 4D Gaussian primitives directly from multi-view tokens (no motion vectors, no HexPlane); 🔹 Uses a clean, minimal Transformer backbone; 🔹 Generalizes with fast, high-quality feedforward rendering at any view and infinite frame rate. Check out more interactive demos and scaling behaviors on our homepage/paper. 👉Website: 👉Paper:

Martin Ziqiao Ma

21,851 Aufrufe • vor 1 Jahr

Achieving human-like dexterity is the next frontier for robotics, and yet dexterity data is often subtly hard to scale. Real-world dexterity data, including things like finger-pose estimates, is often slightly off, making it physically invalid and hard to execute on real hardware and hard to learn from. DO AS I DO is an algorithm for reconstructing and retargeting monocular RGB videos to robot hands, outperforming the state of the art and even working from generated videos. Bhawna Paliwal Haritheja Will Liang and Mahi Shafiullah 🏠🤖 join us to tell us more. Watch Episode 107 of RoboPapers, with Chris Paxton and Jiafei Duan, now!
1:02:31

Achieving human-like dexterity is the next frontier for robotics, and yet dexterity data is often subtly hard to scale. Real-world dexterity data, including things like finger-pose estimates, is often slightly off, making it physically invalid and hard to execute on real hardware and hard to learn from. DO AS I DO is an algorithm for reconstructing and retargeting monocular RGB videos to robot hands, outperforming the state of the art and even working from generated videos. Bhawna Paliwal Haritheja Will Liang and Mahi Shafiullah 🏠🤖 join us to tell us more. Watch Episode 107 of RoboPapers, with Chris Paxton and Jiafei Duan, now!

RoboPapers

23,758 Aufrufe • vor 2 Tagen

CoDeF: Content Deformation Fields for Temporally Consistent Video Processing abs: paper page: present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i.e., rendered from the canonical content field) to each individual frame along the time axis.Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline.We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e.g., the object shape) from the video.With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field.We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training.More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog.
0:21

CoDeF: Content Deformation Fields for Temporally Consistent Video Processing abs: paper page: present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i.e., rendered from the canonical content field) to each individual frame along the time axis.Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline.We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e.g., the object shape) from the video.With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field.We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training.More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog.

AK

153,305 Aufrufe • vor 3 Jahren

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)
0:30

We’ve upgraded Stable Video Diffusion 4D to Stable Video 4D 2.0 (SV4D 2.0), improving the quality of 4D outputs generated from a single object-centric video. While 3D provides a static view of an object’s shape and size; 4D extends this by including time, showing how the object moves. This multi-view video diffusion model generates a 4D output in three steps: 1️⃣ Starts with an input video of a moving person or object 2️⃣ Generates novel views of the subject from unseen angles 3️⃣ Constructs a single dynamic 4D video output with spatial and temporal consistency You can learn more here: (1/4)

Stability AI

36,059 Aufrufe • vor 1 Jahr

Infinite Photorealistic Worlds using Procedural Generation paper page: introduce Infinigen, a procedural generator of photorealistic 3D scenes of the natural world. Infinigen is entirely procedural: every asset, from shape to texture, is generated from scratch via randomized mathematical rules, using no external source and allowing infinite variation and composition. Infinigen offers broad coverage of objects and scenes in the natural world including plants, animals, terrains, and natural phenomena such as fire, cloud, rain, and snow. Infinigen can be used to generate unlimited, diverse training data for a wide range of computer vision tasks including object detection, semantic segmentation, optical flow, and 3D reconstruction. We expect Infinigen to be a useful resource for computer vision research and beyond.
3:03

Infinite Photorealistic Worlds using Procedural Generation paper page: introduce Infinigen, a procedural generator of photorealistic 3D scenes of the natural world. Infinigen is entirely procedural: every asset, from shape to texture, is generated from scratch via randomized mathematical rules, using no external source and allowing infinite variation and composition. Infinigen offers broad coverage of objects and scenes in the natural world including plants, animals, terrains, and natural phenomena such as fire, cloud, rain, and snow. Infinigen can be used to generate unlimited, diverse training data for a wide range of computer vision tasks including object detection, semantic segmentation, optical flow, and 3D reconstruction. We expect Infinigen to be a useful resource for computer vision research and beyond.

AK

275,558 Aufrufe • vor 3 Jahren

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen
0:44

📢 OneCanvas: 3D Scene Understanding via Panoramic Reprojection We extract features from video frames and reproject them into one occlusion-free view of the whole scene that a 2D VLM reads just like a normal image. We can center this view on any viewpoint, including an agent's own pose for situated reasoning. The same projection lets us create spatial training tasks with no human annotation, solvable only by reasoning over the 3D positions of real object features placed on an otherwise empty canvas. The result is a stock 2D VLM that reasons in 3D, setting a new state of the art across spatial benchmarks at far less compute. 🌐 ▶️ Great work by Bartłomiej Baranowski & Dave Zhenyu Chen

Matthias Niessner

25,405 Aufrufe • vor 3 Monaten

INFINIT partners with Google and GoogleCloudTech to bring agentic finance to millions. Anyone can access INFINIT's AI Agents for agentic coordination in their financial apps. This partnership marks the first step towards INFINIT becoming the universal infrastructure for agentic finance. This is the foundation for agentic finance at scale. Proven in DeFi, Built for Global Finance INFINIT has proven sophisticated agent coordination in DeFi: • 559,000+ Wallets • 506,000+ DeFi Conversations • 633,000+ Agent Transactions DeFi was the start. Next is scaling these capabilities to millions of developers building the future of agentic finance. A2A Integration Unlocks Exponential Distribution INFINIT integrates with Agent2Agent (A2A), Google's open standard for AI agent interoperability. This transforms how developers access INFINIT's DeFi capabilities.​ Every application adopting A2A automatically gains access to INFINIT's agent infrastructure, exponentially expanding reach from individual partnerships to ecosystem-wide distribution. Any application can now integrate INFINIT's agentic coordination capabilities: • Wallets requiring intelligent portfolio management • Trading platforms executing cross-chain strategies • Financial services building autonomous yield optimization • Portfolio managers coordinating multi-protocol operations Developers integrate sophisticated agentic coordination in a matter of hours, while users access advanced financial strategies with agentic coordination. Google's AI Infrastructure That Enables Agentic Finance Google Cloud's Vertex AI provides the foundation enabling INFINIT's agent coordination at scale with these capabilities:​ 1. Specialization: Vertex AI's Model Garden lets INFINIT's infrastructure to automatically select the optimal LLM for each natural language query.​ 2. Personalization: Vertex AI's RAG Engine processes massive on-chain and off-chain data, enabling agents to understand user history, market conditions, and protocol details providing complete context to AI agents. 3. Accuracy: Gemini's capability feeds complete instructions to all 30+ agents across multiple blockchains without compromises resulting in zero hallucination in financial execution.​​ The Vision: From DeFi to Payments to Complete Financial Coordination This is only the beginning of INFINIT and Google's collaboration.​ Google recently launched its Agent Payments Protocol (AP2) as an extension of A2A - enabling autonomous commerce across 60+ partners including American Express, Mastercard, PayPal, Coinbase, and Revolut.​ Agents will be able to execute purchases, coordinate bookings, and manage delegated financial tasks autonomously, starting from payments. The next stage entails sophisticated agentic coordination beyond simple transactions. INFINIT provides this through A2A-compatible DeFi infrastructure where agents orchestrate: • Personalized yield optimization • Cross-chain liquidity management • Portfolio rebalancing across protocols • Multi-step strategy execution As the agentic payment ecosystem matures, INFINIT becomes the infrastructure enabling agents to not only spend capital, but strategically manage and grow it.​​ From standalone DeFi agents to the universal infrastructure for global agentic finance.​ This partnership and integration with Google and Google Cloud positions INFINIT as a key building block for agentic finance, helping shape a more transparent, efficient, and accessible financial system. The future of finance is agentic. The foundation is INFINIT.
0:40

INFINIT partners with Google and GoogleCloudTech to bring agentic finance to millions. Anyone can access INFINIT's AI Agents for agentic coordination in their financial apps. This partnership marks the first step towards INFINIT becoming the universal infrastructure for agentic finance. This is the foundation for agentic finance at scale. Proven in DeFi, Built for Global Finance INFINIT has proven sophisticated agent coordination in DeFi: • 559,000+ Wallets • 506,000+ DeFi Conversations • 633,000+ Agent Transactions DeFi was the start. Next is scaling these capabilities to millions of developers building the future of agentic finance. A2A Integration Unlocks Exponential Distribution INFINIT integrates with Agent2Agent (A2A), Google's open standard for AI agent interoperability. This transforms how developers access INFINIT's DeFi capabilities.​ Every application adopting A2A automatically gains access to INFINIT's agent infrastructure, exponentially expanding reach from individual partnerships to ecosystem-wide distribution. Any application can now integrate INFINIT's agentic coordination capabilities: • Wallets requiring intelligent portfolio management • Trading platforms executing cross-chain strategies • Financial services building autonomous yield optimization • Portfolio managers coordinating multi-protocol operations Developers integrate sophisticated agentic coordination in a matter of hours, while users access advanced financial strategies with agentic coordination. Google's AI Infrastructure That Enables Agentic Finance Google Cloud's Vertex AI provides the foundation enabling INFINIT's agent coordination at scale with these capabilities:​ 1. Specialization: Vertex AI's Model Garden lets INFINIT's infrastructure to automatically select the optimal LLM for each natural language query.​ 2. Personalization: Vertex AI's RAG Engine processes massive on-chain and off-chain data, enabling agents to understand user history, market conditions, and protocol details providing complete context to AI agents. 3. Accuracy: Gemini's capability feeds complete instructions to all 30+ agents across multiple blockchains without compromises resulting in zero hallucination in financial execution.​​ The Vision: From DeFi to Payments to Complete Financial Coordination This is only the beginning of INFINIT and Google's collaboration.​ Google recently launched its Agent Payments Protocol (AP2) as an extension of A2A - enabling autonomous commerce across 60+ partners including American Express, Mastercard, PayPal, Coinbase, and Revolut.​ Agents will be able to execute purchases, coordinate bookings, and manage delegated financial tasks autonomously, starting from payments. The next stage entails sophisticated agentic coordination beyond simple transactions. INFINIT provides this through A2A-compatible DeFi infrastructure where agents orchestrate: • Personalized yield optimization • Cross-chain liquidity management • Portfolio rebalancing across protocols • Multi-step strategy execution As the agentic payment ecosystem matures, INFINIT becomes the infrastructure enabling agents to not only spend capital, but strategically manage and grow it.​​ From standalone DeFi agents to the universal infrastructure for global agentic finance.​ This partnership and integration with Google and Google Cloud positions INFINIT as a key building block for agentic finance, helping shape a more transparent, efficient, and accessible financial system. The future of finance is agentic. The foundation is INFINIT.

INFINIT

177,142 Aufrufe • vor 11 Monaten

What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to be annoyingly complex—in both the action and vision space—to even get close to real life. We did an initial attempt: Whole-Body Conditioned Egocentric Video Prediction. In collaboration with @dans_t123 , Amir Bar, Yann LeCun , trevordarrell and Jitendra MALIK. (For more details, check: What we did is very simple: Predict Egocentric Video from human Actions (PEVA) - Given the past video and a future action represented by relative 3D body pose, PEVA predicts how the world looks next—from the first-person view. By conditioning on kinematic pose trajectories, structured by the joint hierarchy of the body, it learns how physical actions shape perception.
3:12

What would a World Model look like if we start from a real embodied agent acting in the real world? It has to have: 1) A real, physically grounded and complex action space—not just abstract control signals. 2) Diverse, real-life scenarios and activities. Or in short: It has to be annoyingly complex—in both the action and vision space—to even get close to real life. We did an initial attempt: Whole-Body Conditioned Egocentric Video Prediction. In collaboration with @dans_t123 , Amir Bar, Yann LeCun , trevordarrell and Jitendra MALIK. (For more details, check: What we did is very simple: Predict Egocentric Video from human Actions (PEVA) - Given the past video and a future action represented by relative 3D body pose, PEVA predicts how the world looks next—from the first-person view. By conditioning on kinematic pose trajectories, structured by the joint hierarchy of the body, it learns how physical actions shape perception.

Yutong Bai

179,696 Aufrufe • vor 1 Jahr

🚨 Paper Alert 🚨 ➡️Paper Title: Articulate3D: Zero-Shot Text-Driven 3D Object Posing 🌟Few pointers from the paper 🎯Authors of this paper proposed a training-free method, “Articulate3D”, to pose a 3D asset through language control. 🎯Despite advances in vision and language models, this task remains surprisingly challenging. 🎯To achieve this goal, they decomposed the problem into two steps. 🎯They modified a powerful image-generator to create target images conditioned on the input image and a text instruction. 🎯They then align the mesh to the target images through a multi-view pose optimisation step. 🎯 In detail, they introduced a self-attention rewiring mechanism (RSActrl) that decouples the source structure from pose within an image generative model, allowing it to maintain a consistent structure across varying poses. 🎯They observed that differentiable rendering is an unreliable signal for articulation optimisation; instead, they used keypoints to establish correspondences between input and target images. 🎯The effectiveness of Articulate3D is demonstrated across a diverse range of 3D objects and free-form text prompts, successfully manipulating poses while maintaining the original identity of the mesh. 🎯Quantitative evaluations and a comparative user study, in which their method was preferred over 85% of the time, confirm its superiority over existing approaches. 🏢Organization: University of Oxford , Google DeepMind 🧙Paper Authors: Oishi Deb, Anjun Hu, Ashkan Khakzar, Philip Torr, Christian Rupprecht 📝 Read the Full Paper here: 🗂️ Project Page: 🎥 Be sure to watch the attached Demo Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️QT and teach your network something new Follow me 👣, naveen manwani , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.
0:46

🚨 Paper Alert 🚨 ➡️Paper Title: Articulate3D: Zero-Shot Text-Driven 3D Object Posing 🌟Few pointers from the paper 🎯Authors of this paper proposed a training-free method, “Articulate3D”, to pose a 3D asset through language control. 🎯Despite advances in vision and language models, this task remains surprisingly challenging. 🎯To achieve this goal, they decomposed the problem into two steps. 🎯They modified a powerful image-generator to create target images conditioned on the input image and a text instruction. 🎯They then align the mesh to the target images through a multi-view pose optimisation step. 🎯 In detail, they introduced a self-attention rewiring mechanism (RSActrl) that decouples the source structure from pose within an image generative model, allowing it to maintain a consistent structure across varying poses. 🎯They observed that differentiable rendering is an unreliable signal for articulation optimisation; instead, they used keypoints to establish correspondences between input and target images. 🎯The effectiveness of Articulate3D is demonstrated across a diverse range of 3D objects and free-form text prompts, successfully manipulating poses while maintaining the original identity of the mesh. 🎯Quantitative evaluations and a comparative user study, in which their method was preferred over 85% of the time, confirm its superiority over existing approaches. 🏢Organization: University of Oxford , Google DeepMind 🧙Paper Authors: Oishi Deb, Anjun Hu, Ashkan Khakzar, Philip Torr, Christian Rupprecht 📝 Read the Full Paper here: 🗂️ Project Page: 🎥 Be sure to watch the attached Demo Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️QT and teach your network something new Follow me 👣, naveen manwani , for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

naveen manwani

14,334 Aufrufe • vor 1 Jahr

Why HTML turned out to be the foundation for agentic video making from Bin Liu: “We’ve been trying to build a video agent. However, we learned the hard way that agents have no visual intelligence. So that’s when we turned to code. HTML is the LLM’s native language. LLMs can express not only information, but also visual aesthetics through HTML, CSS, and JavaScript. Footage, images, assets, SVGs can all sit on top of HTML." 📌 As an example, see how you can make a product video from any website URL here:
0:41

Why HTML turned out to be the foundation for agentic video making from Bin Liu: “We’ve been trying to build a video agent. However, we learned the hard way that agents have no visual intelligence. So that’s when we turned to code. HTML is the LLM’s native language. LLMs can express not only information, but also visual aesthetics through HTML, CSS, and JavaScript. Footage, images, assets, SVGs can all sit on top of HTML." 📌 As an example, see how you can make a product video from any website URL here:

Peter Yang

16,653 Aufrufe • vor 3 Monaten

I kept seeing #GPT-6 Astra modelling results on here and got curious enough to build the thing myself: an agentic pipeline that turns a plain RGB video into 3D assets. The living room for robots MIT CSAIL. Input: one handheld phone walkthrough of our lab kitchen (~20s). Nothing else — no depth sensor, no CAD, no asset library. ~1 day (I actually slept overnight), including human-in-the-loop. Live: It is still not perfect — thin and shiny things are still weak, a few objects are drafts, the room shell needs another pass. The loop, roughly: - Monocular video → metric scan (ViPE): camera poses + depth. This is the measuring instrument, not the output. - GPT-6 lists what should exist as separate objects, then open-vocabulary detection + tracking gives per-object masks; each object is fused and measured in metres. - For every asset, GPT-6 works in its own sandbox with tools: it writes the object as a program in a small Blender DSL (closed solids, PBR colours, hinges/drawers), builds it, renders it over the original video frames, compares it with the scan points in 3D, and iterates. - A separate GPT-6 session is the verifier — it can render the model on any frame it wants and must justify every complaint with a frame. The modeller never grades its own work. - A completeness pass renders the whole modelled scene from the video's own cameras, puts it next to the real frames, and says what's still missing. What the detector keeps missing (a row of identical cabinets) gets placed geometrically instead. - Out comes MJCF/URDF with joints. Now, adding simulation to this env.
0:20

I kept seeing #GPT-6 Astra modelling results on here and got curious enough to build the thing myself: an agentic pipeline that turns a plain RGB video into 3D assets. The living room for robots MIT CSAIL. Input: one handheld phone walkthrough of our lab kitchen (~20s). Nothing else — no depth sensor, no CAD, no asset library. ~1 day (I actually slept overnight), including human-in-the-loop. Live: It is still not perfect — thin and shiny things are still weak, a few objects are drafts, the room shell needs another pass. The loop, roughly: - Monocular video → metric scan (ViPE): camera poses + depth. This is the measuring instrument, not the output. - GPT-6 lists what should exist as separate objects, then open-vocabulary detection + tracking gives per-object masks; each object is fused and measured in metres. - For every asset, GPT-6 works in its own sandbox with tools: it writes the object as a program in a small Blender DSL (closed solids, PBR colours, hinges/drawers), builds it, renders it over the original video frames, compares it with the scan points in 3D, and iterates. - A separate GPT-6 session is the verifier — it can render the model on any frame it wants and must justify every complaint with a frame. The modeller never grades its own work. - A completeness pass renders the whole modelled scene from the video's own cameras, puts it next to the real frames, and says what's still missing. What the detector keeps missing (a row of identical cabinets) gets placed geometrically instead. - Out comes MJCF/URDF with joints. Now, adding simulation to this env.

Zhiyang (Frank) Dou

11,170 Aufrufe • vor 16 Tagen