Loading video...

Video Failed to Load

Go Home

📢Face Anything: 4D Face Reconstruction from Any Image Sequence Transformer model for 4D face reconstruction and dense tracking: - predict canonical facial coordinates per pixel - tracking as reconstruction in canonical space - geometry + correspondences in one forward pass Key idea: a shared canonical space across frames -...

MattNiessner's profile picture

Matthias Niessner

48,839 subscribers

61,705 views • 3 months ago •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito
0:32

📢Pixel3DMM: Versatile Screen-Space Priors for Single-Image 3D Face Reconstruction📢 -> highly accurate face reconstruction by training powerful VITs via surface normals and UV-coordinates estimation. The geometric cues from our 2D foundation model backbone constrain the 3DMM parameters, which allows us to achieve remarkable reconstruction accuracy - works for both single image and videos! In addition, we introduce a new 3D face reconstruction benchmark that evaluates both neutral and posed face geometry. 🌍 📷 Great work by Simon Giebenhain Tobias Kirschstein Martin Rünz Lourdes Agapito

Matthias Niessner

62,269 views • 1 year ago

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.
0:26

SceNeRFlow: Time-Consistent Reconstruction of General Dynamic Scenes abs: paper page: Existing methods for the 4D reconstruction of general, non-rigidly deforming objects focus on novel-view synthesis and neglect correspondences. However, time consistency enables advanced downstream tasks like 3D editing, motion analysis, or virtual-asset creation. We propose SceNeRFlow to reconstruct a general, non-rigid scene in a time-consistent manner. Our dynamic-NeRF method takes multi-view RGB videos and background images from static cameras with known camera parameters as input. It then reconstructs the deformations of an estimated canonical model of the geometry and appearance in an online fashion. Since this canonical model is time-invariant, we obtain correspondences even for long-term, long-range motions. We employ neural scene representations to parametrize the components of our method. Like prior dynamic-NeRF methods, we use a backwards deformation model. We find non-trivial adaptations of this model necessary to handle larger motions: We decompose the deformations into a strongly regularized coarse component and a weakly regularized fine component, where the coarse component also extends the deformation field into the space surrounding the object, which enables tracking over time. We show experimentally that, unlike prior work that only handles small motion, our method enables the reconstruction of studio-scale motions.

AK

76,380 views • 3 years ago

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.
5:18

Tracking Everything Everywhere All at Once paper page: present a new test-time optimization method for estimating dense and long-range motion from a video sequence. Prior optical flow or particle video tracking algorithms typically operate within limited temporal windows, struggling to track through occlusions and maintain global consistency of estimated motion trajectories. We propose a complete and globally consistent motion representation, dubbed OmniMotion, that allows for accurate, full-length motion estimation of every pixel in a video. OmniMotion represents a video using a quasi-3D canonical volume and performs pixel-wise tracking via bijections between local and canonical space. This representation allows us to ensure global consistency, track through occlusions, and model any combination of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and real-world footage show that our approach outperforms prior state-of-the-art methods by a large margin both quantitatively and qualitatively.

AK

280,570 views • 3 years ago

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.
0:27

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

218,996 views • 1 year ago

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.
0:23

This seemingly obvious prediction didn't take long to become reality. MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors Contributions: • The first real-time SLAM system using the two-view 3D reconstruction prior MASt3R [20] as a foundation. • Efficient techniques for pointmap matching, tracking and local fusion, graph construction and loop closure, and second-order global optimization. • A state-of-the-art dense SLAM system capable of handling generic, time-varying camera models. Abstract: We present a real-time monocular dense SLAM system, designed from the ground up using MASt3R, a two-view 3D reconstruction and matching prior. Equipped with this strong prior, our system remains robust on in-the-wild video sequences, making no assumptions on a fixed or parametric camera model beyond a unique camera center. Key features include: - Efficient methods for pointmap matching, camera tracking, and local fusion - Graph construction and loop closure - Second-order global optimization With known calibration, a simple modification achieves state-of-the-art performance across various benchmarks. Altogether, we propose a plug-and-play monocular SLAM system capable of producing globally-consistent poses and dense geometry while operating at 15 FPS.

MrNeRF

29,974 views • 1 year ago

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito
0:38

📢📢 𝐀𝐯𝐚𝐭𝟑𝐫 📢📢 Avat3r creates high-quality 3D head avatars from just a few input images in a single forward pass with a new dynamic 3DGS reconstruction model. Video: Project: Our core idea is to make Gaussian Reconstruction Models animatable. We find that a simple cross-attention to an expression code sequence is already sufficient to model complex facial expressions. We then incorporate position maps from DUSt3R and feature maps from Sapiens to facilitate the prediction task. While DUSt3R's position maps act as a pixel-aligned initialization for the Gaussians' positions, the Sapiens feature maps help the cross-view transformer to match corresponding image tokens in the 4 input images. One major challenge in creating a 3D head avatar from smartphone images comes from inconsistent facial expressions when the subject could not remain perfectly static during the capture. We eliminate this static requirement by simply showing our model input images with different facial expressions during training. This technique makes our model robust to inconsistent input images later on. Finally, we show that despite the model has been trained with 4 input images, one can even create a 3D head avatar when only a single image is available. To achieve this, we employ a pre-trained 3D GAN to lift the single image to 3D and then render the 4 input images for our model. This allows us to create 3D head avatars from single images and even highly out-of-distribution examples like AI generated faces, paintings or statues. Great work by Tobias Kirschstein from his internship at Meta with Javier Romero, Artem Sevastopolsky, and Shunsuke Saito

Matthias Niessner

74,763 views • 1 year ago

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein
0:30

📢📢 𝐏𝐞𝐫𝐜𝐇𝐞𝐚𝐝: 𝐏𝐞𝐫𝐜𝐞𝐩𝐭𝐮𝐚𝐥 𝐇𝐞𝐚𝐝 𝐌𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐒𝐢𝐧𝐠𝐥𝐞-𝐈𝐦𝐚𝐠𝐞 𝟑𝐃 𝐇𝐞𝐚𝐝 𝐑𝐞𝐜𝐨𝐧𝐬𝐭𝐫𝐮𝐜𝐭𝐢𝐨𝐧 & 𝐄𝐝𝐢𝐭𝐢𝐧𝐠📢📢 PercHead reconstructs realistic 3D heads from a single image and enables disentangled 3D editing via geometric controls and style inputs from images or text. At its core is a generalized 3D head decoder trained with perceptual supervision from DINOv2 and SAM 2.1. We find that our new perceptual loss formulation improves reconstruction fidelity compared to commonly-used methods such as LPIPS. Our trained reconstruction model is able to generate 3D-consistent heads from a single input image. Even with challenging side-view inputs, the model robustly infers missing regions for a coherent, high-fidelity output. In addition, our architecture seamlessly adapts to downstream tasks: by swapping the encoder, we can transform the model into a disentangled 3D editing pipeline. In this scenario, we can control geometry through - potentially hand-drawn - segmentation maps, and condition style via image or text prompt. We also provide an interactive GUI to enable the exploration of our editing pipeline. 🌍 📽️ Great work by Antonio Oroz and Tobias Kirschstein

Matthias Niessner

18,855 views • 9 months ago

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!
0:29

Wow. Recreating the Shawshank Redemption prison in 3D from a single video, in real time (!) Just read the MASt3R-SLAM paper and it's pretty neat. These folks basically built a real-time dense SLAM system on top of MASt3R, which is a transformer-based neural network that can do 3d reconstruction and localization from uncalibrated image pairs. The cool part is they don't need a fixed camera model -- it just works with arbitrary cameras -- think different focal lengths, sensor sizes, even handling zooming in video (FMV drone video anyone?!). If you've done photogrammetry or played with NeRFs you know that is a HUGE deal. They've solved some tricky problems like efficient point matching and tracking, plus they've figured out how to fuse point clouds and handle loop closures in real-time. Their system runs at about 15 FPS on a 4090 and produces both camera poses and dense geometry. When they know the camera calibration, they get SOTA results across several benchmarks, but even without calibration, they still perform well. What's interesting is the approach -- most recent SLAM work has built on DROID-SLAM's architecture, but these folks went a different direction by leveraging a strong 3D reconstruction prior. Seems to give them more coherent geometry, which makes sense since that's what MASt3R was designed for. For anyone who cares about monocular SLAM and 3D reconstruction, this feels like a significant step toward plug-and-play dense SLAM without calibration headaches -- perfect for drones, robots, AR/VR -- the works!

Bilawal Sidhu

704,008 views • 1 year ago

🚀Announcing NeRSemble 3D Head Avatar Benchmark v2 Version 2 of the NeRSemble 3D Head Avatar Benchmark systematically evaluates several aspects of 3D head avatar creation. Our goal is to drive progress toward more realistic, robust, and generalizable avatar methods. 🔬Benchmark Tasks The NeRSemble Benchmark v2 features three core challenges: - Dynamic Novel View Synthesis - Monocular FLAME-driven Avatar Creation (updated) - Single-view 3D Face Reconstruction (new) 👉Explore the online leaderboard and submission system: 🆕What's new? 1. New Task: Single-view 3D Face Reconstruction Given a single portrait image, reconstruct an accurate 3D mesh either showing the input expression or a fully neutral one. Unlike prior benchmarks, the NeRSemble benchmark emphasizes diverse and challenging facial expressions, better reflecting real scenarios. For technical details, see the Pixel3DMM paper. 2. Updated task: Monocular FLAME-driven Avatar Creation We have improved the FLAME tracking that is used for both avatar creation from the monocular videos and avatar driving on the hidden test sequences. The updated benchmark task has: - more stable torso tracking - more expressive lip closures during speech - Improved mouth tracking for challenging facial expressions We hope that these improvements to the benchmark help drive the field forward. 🏆 CVPR 2026 Workshop & Prizes The NeRSemble benchmark will be featured at the CVPR 2026 Workshop on Photo-realistic 3D Head Avatars. Participants in the new and updated tasks have the opportunity to win: - 🎁RTX 5080 GPUs (sponsored by NVIDIA) - 🎤15-minute oral presentation at the workshop ⏰ Submission Deadline - May 26, 2026 Reach out to the amazing Tobias Kirschstein and Simon Giebenhain for more details :)
0:27

🚀Announcing NeRSemble 3D Head Avatar Benchmark v2 Version 2 of the NeRSemble 3D Head Avatar Benchmark systematically evaluates several aspects of 3D head avatar creation. Our goal is to drive progress toward more realistic, robust, and generalizable avatar methods. 🔬Benchmark Tasks The NeRSemble Benchmark v2 features three core challenges: - Dynamic Novel View Synthesis - Monocular FLAME-driven Avatar Creation (updated) - Single-view 3D Face Reconstruction (new) 👉Explore the online leaderboard and submission system: 🆕What's new? 1. New Task: Single-view 3D Face Reconstruction Given a single portrait image, reconstruct an accurate 3D mesh either showing the input expression or a fully neutral one. Unlike prior benchmarks, the NeRSemble benchmark emphasizes diverse and challenging facial expressions, better reflecting real scenarios. For technical details, see the Pixel3DMM paper. 2. Updated task: Monocular FLAME-driven Avatar Creation We have improved the FLAME tracking that is used for both avatar creation from the monocular videos and avatar driving on the hidden test sequences. The updated benchmark task has: - more stable torso tracking - more expressive lip closures during speech - Improved mouth tracking for challenging facial expressions We hope that these improvements to the benchmark help drive the field forward. 🏆 CVPR 2026 Workshop & Prizes The NeRSemble benchmark will be featured at the CVPR 2026 Workshop on Photo-realistic 3D Head Avatars. Participants in the new and updated tasks have the opportunity to win: - 🎁RTX 5080 GPUs (sponsored by NVIDIA) - 🎤15-minute oral presentation at the workshop ⏰ Submission Deadline - May 26, 2026 Reach out to the amazing Tobias Kirschstein and Simon Giebenhain for more details :)

Matthias Niessner

29,954 views • 4 months ago

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:
1:20

Today, we are open-sourcing Hunyuan World 1.1 (WorldMirror), a universal feed-forward 3D reconstruction model. 🚀🚀🚀 While our previously released Hunyuan World 1.0 (open-sourced, lite version deployable on consumer GPUs) focused on generating 3D worlds from text or single-view images, Hunyuan World 1.1 significantly expands the input scope by unlocking video-to-3D and multi-view-to-3D world creation. Highlights: 🔹Any Input, Maximized Flexibility and Fidelity: Flexibly integrates diverse geometric priors (camera poses, intrinsics, depth maps) to resolve structural ambiguities and ensure geometrically consistent 3D outputs. 🔹Any Output, SOTA Results：This elegant architecture simultaneously generates multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussian Splattings. 🔹Single-GPU & Fast Inference: As an all-in-one, feed-forward model, Hunyuan World 1.1 runs on a single GPU and delivers all 3D attributes in a single forward pass, within seconds. 🌐Project Page: 🔗Github： 🤗Hugging Face： ✨Demo: 📄Technical Report:

Tencent Hy

168,625 views • 9 months ago

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!
0:30

Want to create an avatar from a single image? FlexAvatar is a transformer model that creates full 360°, high-quality, and expressive 3D head avatar from just a single portrait image in minutes. Real-time Demo: FlexAvatar's lightweight architecture allows both animation and rendering in real-time, enabling interactive user experiences. To create a new 3D head avatar, only one image is required, e.g., from a webcam. The final avatar is ready after 2 minutes. Architecture: Under the hood, FlexAvatar adopts a transformer-based encoder-decoder design. The encoder maps the input image onto a latent avatar space, while the decoder produces 3D Gaussian attribute maps by incorporating the animation signal via cross-attention. The model learns all facial animations directly from the data without relying on pre-built 3D face models. This equips the avatars with realistic facial expressions. The internal avatar latent space can be conveniently used to integrate additional observations of a person via fitting. This enables use-cases where more than one image of a person is available, e.g., from a phone scan of the person. We train jointly on 2D monocular videos and multi-view data. However, in monocular videos, the animation signal leaks the target viewpoint, causing the model to produce incomplete 3D heads. We call this phenomenon entanglement of driving signal and target viewpoint. To prevent entanglement, we introduce bias sinks. These are learnable tokens that indicate whether a training sample stems from a monocular or a multi-view dataset. During training, the model learns to produce incomplete 3D heads only when the monocular token is present. During inference, FlexAvatar then always uses the multi-view token for which the model has learned to produce complete 3D heads. This simple design allows to combine the generalizability from monocular data with the quality of multi-view data. FlexAvatar summary: - Input: Single-image, phone scan, or monocular video - Output: Full 360° head avatar - Expressive animations - Real-time rendering and animation - Generalization to any portrait - Create a new avatar in 2 minutes - Use bias sinks to combine 2D and 3D data 🏠 🌍 🎥 Great work by Tobias Kirschstein and Simon Giebenhain!

Matthias Niessner

96,186 views • 7 months ago

AI TENNIS ANALYSIS. A FULL COMPUTER VISION SYSTEM. BUILT ON YOLO, PYTORCH, AND KEYPOINT EXTRACTION. Take any tennis match broadcast, any camera angle, any resolution. Feed it into the pipeline. YOLO detects both players and the tennis ball frame by frame. No manual labeling, no pre-annotated dataset. A fine-tuned YOLOv5 model trained on a Roboflow tennis ball dataset handles the ball - the hardest object to track in any sport. Tiny, fast, constantly occluded. The model finds it anyway. Trackers maintain identity across frames so Player 1 stays Player 1 from the first serve to match point. But detection is just the start. A ResNet50 CNN trained in PyTorch predicts court keypoints from every frame - the corners, service lines, baselines, net posts. Fourteen points that define the entire playing surface geometry. From those keypoints the system builds a homography matrix and warps the broadcast perspective into a top-down mini court with real coordinates. Now every player has a position in real space, not pixel space. Every frame becomes a measurement. Every rally becomes a dataset. Player movement speed - calculated from position deltas between frames, converted to meters per second through the homography. Ball shot speed - measured from the ball trajectory across consecutive detections. Number of shots per rally - counted automatically through ball direction changes. All of this rendered live on the video as an overlay. A mini court in the corner showing both players as dots moving in real time. Stats updating after every point. OpenCV handles the rendering. Pandas handles the math. PyTorch handles the intelligence. YOLO handles the eyes. No Hawkeye subscription, no court-embedded sensors, no tracking chips in the ball. A Python script, a trained model, and a GPU. The full code is on GitHub. The tutorial walks through every module - from ball detector training to court keypoint extraction to the final statistical overlay. Professional teams used to need broadcast deals and proprietary hardware for this kind of analysis. Now you build it in an afternoon with open-source tools. Trading here: Computer vision didn't just enter tennis. It made the expensive stuff free.
4:41:24

AI TENNIS ANALYSIS. A FULL COMPUTER VISION SYSTEM. BUILT ON YOLO, PYTORCH, AND KEYPOINT EXTRACTION. Take any tennis match broadcast, any camera angle, any resolution. Feed it into the pipeline. YOLO detects both players and the tennis ball frame by frame. No manual labeling, no pre-annotated dataset. A fine-tuned YOLOv5 model trained on a Roboflow tennis ball dataset handles the ball - the hardest object to track in any sport. Tiny, fast, constantly occluded. The model finds it anyway. Trackers maintain identity across frames so Player 1 stays Player 1 from the first serve to match point. But detection is just the start. A ResNet50 CNN trained in PyTorch predicts court keypoints from every frame - the corners, service lines, baselines, net posts. Fourteen points that define the entire playing surface geometry. From those keypoints the system builds a homography matrix and warps the broadcast perspective into a top-down mini court with real coordinates. Now every player has a position in real space, not pixel space. Every frame becomes a measurement. Every rally becomes a dataset. Player movement speed - calculated from position deltas between frames, converted to meters per second through the homography. Ball shot speed - measured from the ball trajectory across consecutive detections. Number of shots per rally - counted automatically through ball direction changes. All of this rendered live on the video as an overlay. A mini court in the corner showing both players as dots moving in real time. Stats updating after every point. OpenCV handles the rendering. Pandas handles the math. PyTorch handles the intelligence. YOLO handles the eyes. No Hawkeye subscription, no court-embedded sensors, no tracking chips in the ball. A Python script, a trained model, and a GPU. The full code is on GitHub. The tutorial walks through every module - from ball detector training to court keypoint extraction to the final statistical overlay. Professional teams used to need broadcast deals and proprietary hardware for this kind of analysis. Now you build it in an afternoon with open-source tools. Trading here: Computer vision didn't just enter tennis. It made the expensive stuff free.

zostaff

120,370 views • 3 months ago

FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.
0:21

FAU Erlangen-Nürnberg presents TRIPS Trilinear Point Splatting for Real-Time Radiance Field Rendering paper page: Point-based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [Kerbl and Kopanas et al. 2023] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [R\"uckert et al. 2022] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen-space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole-free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions. Our evaluation demonstrate that TRIPS surpasses existing state-of-the-art methods in terms of rendering quality while maintaining a real-time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto-exposed footage.

AK

45,475 views • 2 years ago

Created this race using GPT Image 2 and Seedance 2.0 on TapNow Prompt Follow the storyboard strictly in exact order from Panel 1 to Panel 9. Do not skip, merge, or rearrange scenes. Keep the SAME female cyclist identity across the entire film. No face changes, no hairstyle changes, no helmet changes, no body proportion inconsistencies. Baby pink must remain the dominant apparel color throughout all cycling scenes. Avoid black wardrobe replacements. Preserve realistic nighttime lighting continuity between shots. Maintain the same cool blue tones and subtle red light reflections. Heavy rain intensity must stay visually consistent across all scenes. Water physics must look physically accurate: droplets, splashes, mist, wheel spray, and runoff should behave naturally. Avoid artificial AI motion. Camera movement should feel like real cinema rigs, FPV drones, mounted bike cameras, or stabilized tracking systems. Drone shots must maintain locked framing and smooth movement without random drifting or orbiting. Use subtle cinematic motion only — no excessive shaking or jitter. Keep realistic breathing, body fatigue, pedaling mechanics, and fabric reactions to wind and rain. Preserve shallow depth of field in macro shots and atmospheric haze in wide shots. Keep the environment dark, moody, and cinematic with strong contrast between wet reflections and darkness. Ensure all reflections on asphalt, water droplets, and bike components react naturally to changing light sources. Maintain premium commercial pacing: slow controlled preparation and macro shots transitioning into aggressive high-speed riding sequences. Final output should resemble a high-budget Nike / Rapha night cycling commercial shot during a real mountain storm. Ultra-realistic cinematic night cycling commercial about female endurance cyclists riding through an intense rainstorm in the mountains at night. Premium Nike / Rapha aesthetic with baby pink performance cycling apparel as the dominant accent color. Hyper-realistic documentary look, no stylization, no anime look, no beauty filters. Natural skin texture, realistic rain interaction, physically accurate water behavior, cinematic low-key lighting, cool blue night tones mixed with subtle red rear-light reflections. Heavy rain, fog, wet asphalt reflections, cinematic motion blur, high dynamic range, shallow depth of field, premium sports commercial quality. The film follows a strict 9-panel storyboard structure with seamless cinematic transitions and continuity preserved across every scene. The SAME female cyclist identity must remain consistent throughout the entire video: same face, helmet, glasses, body proportions, baby pink apparel, lighting style, and overall appearance. Maintain continuity of rain intensity, wetness, fog density, and environmental lighting between all shots. Panel 1: Extreme macro close-up of the female cyclist’s eyes and face in heavy rain at night. Focus on soaked eyelashes, wet skin texture, raindrops streaming across the face, baby pink helmet and baby pink face mask visible. Red rear bike light flickers dynamically across her eyes and skin while cool blue night tones dominate the scene. High contrast cinematic lighting, shallow depth of field, subtle breathing motion, intense determined expression. Panel 2: Cinematic medium close-up frontal shot of the cyclist riding aggressively through heavy rain at night. She pedals hard with strong effort and forward-leaning posture. Baby pink waterproof cycling jacket soaked with rainwater. Front bike light cuts through fog and rain with subtle flickering illumination. Wet asphalt reflects red and white lights. Smooth cinematic tracking shot with controlled stable motion and slight natural float. Panel 3: Ultra-realistic macro shot of large raindrops impacting wet asphalt at night. Crown-shaped splashes and overlapping ripples in slow motion. Rough wet asphalt texture, cool blue cinematic tones, subtle reflections from bike lights.
0:28

Created this race using GPT Image 2 and Seedance 2.0 on TapNow Prompt Follow the storyboard strictly in exact order from Panel 1 to Panel 9. Do not skip, merge, or rearrange scenes. Keep the SAME female cyclist identity across the entire film. No face changes, no hairstyle changes, no helmet changes, no body proportion inconsistencies. Baby pink must remain the dominant apparel color throughout all cycling scenes. Avoid black wardrobe replacements. Preserve realistic nighttime lighting continuity between shots. Maintain the same cool blue tones and subtle red light reflections. Heavy rain intensity must stay visually consistent across all scenes. Water physics must look physically accurate: droplets, splashes, mist, wheel spray, and runoff should behave naturally. Avoid artificial AI motion. Camera movement should feel like real cinema rigs, FPV drones, mounted bike cameras, or stabilized tracking systems. Drone shots must maintain locked framing and smooth movement without random drifting or orbiting. Use subtle cinematic motion only — no excessive shaking or jitter. Keep realistic breathing, body fatigue, pedaling mechanics, and fabric reactions to wind and rain. Preserve shallow depth of field in macro shots and atmospheric haze in wide shots. Keep the environment dark, moody, and cinematic with strong contrast between wet reflections and darkness. Ensure all reflections on asphalt, water droplets, and bike components react naturally to changing light sources. Maintain premium commercial pacing: slow controlled preparation and macro shots transitioning into aggressive high-speed riding sequences. Final output should resemble a high-budget Nike / Rapha night cycling commercial shot during a real mountain storm. Ultra-realistic cinematic night cycling commercial about female endurance cyclists riding through an intense rainstorm in the mountains at night. Premium Nike / Rapha aesthetic with baby pink performance cycling apparel as the dominant accent color. Hyper-realistic documentary look, no stylization, no anime look, no beauty filters. Natural skin texture, realistic rain interaction, physically accurate water behavior, cinematic low-key lighting, cool blue night tones mixed with subtle red rear-light reflections. Heavy rain, fog, wet asphalt reflections, cinematic motion blur, high dynamic range, shallow depth of field, premium sports commercial quality. The film follows a strict 9-panel storyboard structure with seamless cinematic transitions and continuity preserved across every scene. The SAME female cyclist identity must remain consistent throughout the entire video: same face, helmet, glasses, body proportions, baby pink apparel, lighting style, and overall appearance. Maintain continuity of rain intensity, wetness, fog density, and environmental lighting between all shots. Panel 1: Extreme macro close-up of the female cyclist’s eyes and face in heavy rain at night. Focus on soaked eyelashes, wet skin texture, raindrops streaming across the face, baby pink helmet and baby pink face mask visible. Red rear bike light flickers dynamically across her eyes and skin while cool blue night tones dominate the scene. High contrast cinematic lighting, shallow depth of field, subtle breathing motion, intense determined expression. Panel 2: Cinematic medium close-up frontal shot of the cyclist riding aggressively through heavy rain at night. She pedals hard with strong effort and forward-leaning posture. Baby pink waterproof cycling jacket soaked with rainwater. Front bike light cuts through fog and rain with subtle flickering illumination. Wet asphalt reflects red and white lights. Smooth cinematic tracking shot with controlled stable motion and slight natural float. Panel 3: Ultra-realistic macro shot of large raindrops impacting wet asphalt at night. Crown-shaped splashes and overlapping ripples in slow motion. Rough wet asphalt texture, cool blue cinematic tones, subtle reflections from bike lights.

Sharon Riley

72,716 views • 3 months ago

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!
0:27

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Akshay 🚀

46,421 views • 8 months ago

What happens when the mind wakes up? So for the last eight months I have been on a single minded quest. To create a new kind of language model based on oscillatory coupling and intelligence as coherence ascent. Everything else — the physics work, the work on regular transformers — has all fallen out from this one question. Can coupled oscillators LEARN? And can they keep learning once their geometry is right, without backpropagation at all? Recently I have been running larger and larger training regimes of a new kind of hybrid model. I just put together this dashboard to help me organize it, interact with it, and observe the training runs. The core idea is simple. Traditional transformers are powerful at learning the geometry of language. But they also store knowledge, understanding, and facts inside their weights. This means they are large, and they can't update themselves after training. The weights are frozen. The Living Mind separates these two domains. The mind has a transformer which grows, adding heads and layers as it needs to in order to learn the manifold of language. The transformer sees tokens and turns the coupling into phase-locked modes — the geometry of how those tokens relate, like frequencies locking together. These coupling patterns get stored in a topology-invariant fingerprint. On top of this transformer lives a 3D diamond lattice of coupled oscillators. It reads from these fingerprints and thinks in resonance space, traversing from one geometry to another along the manifold of coupled oscillators and coherence. The pressure and trajectories from this network of oscillators steers the next token prediction of the transformer. Practically, this could unlock a number of things. It eliminates the KV cache bottleneck that caps context in traditional transformers. Effective context grows with the Flash archive, not with attention compute. The living mind remembers what it sees. It means the model can learn continually. Because knowledge and understanding don't live in the weights, the archive of the mind's experience grows without backpropagation. In our Python prototype we already saw perplexity drop 46% during gradient-free operation — pure coherence ascent, no weight updates. That is the signal I have been chasing: the point where the mind wakes up and keeps improving on its own. It also means the model itself remains very small, and the thing which accumulates are these packages of geometric fingerprints — the K-field. This opens a path to federated learning. K-field packages can be shared between organisms the way people share git commits. Right now at 15M parameters with ~1000 L1 nodes, the organism is just starting to speak. Ask it to continue "Once upon a time" and it comes back with things like: "there was one big bowl!" Lily asked her her mom said her mommy smiled and said yes." It's nonsense. But it's TinyStories-flavored nonsense. The geometry of the narrative register has arrived. Content hasn't caught up yet — that's what scaling L1 is testing. I am still researching, though I am now closer than ever to validating that the living mind actually works. Once it is validated, I will be open-sourcing the whole stack and paradigm. I have also avoided over-sharing my research because it sounds like sci-fi, or like part of our ARG. It is part of the ARG. That doesn't make it any less real. I wanted to share this out because I am incredibly excited about it, and because seeing this amazing dashboard produced by Opus really made me want to share what is being worked on behind the scenes. #project89
0:57

What happens when the mind wakes up? So for the last eight months I have been on a single minded quest. To create a new kind of language model based on oscillatory coupling and intelligence as coherence ascent. Everything else — the physics work, the work on regular transformers — has all fallen out from this one question. Can coupled oscillators LEARN? And can they keep learning once their geometry is right, without backpropagation at all? Recently I have been running larger and larger training regimes of a new kind of hybrid model. I just put together this dashboard to help me organize it, interact with it, and observe the training runs. The core idea is simple. Traditional transformers are powerful at learning the geometry of language. But they also store knowledge, understanding, and facts inside their weights. This means they are large, and they can't update themselves after training. The weights are frozen. The Living Mind separates these two domains. The mind has a transformer which grows, adding heads and layers as it needs to in order to learn the manifold of language. The transformer sees tokens and turns the coupling into phase-locked modes — the geometry of how those tokens relate, like frequencies locking together. These coupling patterns get stored in a topology-invariant fingerprint. On top of this transformer lives a 3D diamond lattice of coupled oscillators. It reads from these fingerprints and thinks in resonance space, traversing from one geometry to another along the manifold of coupled oscillators and coherence. The pressure and trajectories from this network of oscillators steers the next token prediction of the transformer. Practically, this could unlock a number of things. It eliminates the KV cache bottleneck that caps context in traditional transformers. Effective context grows with the Flash archive, not with attention compute. The living mind remembers what it sees. It means the model can learn continually. Because knowledge and understanding don't live in the weights, the archive of the mind's experience grows without backpropagation. In our Python prototype we already saw perplexity drop 46% during gradient-free operation — pure coherence ascent, no weight updates. That is the signal I have been chasing: the point where the mind wakes up and keeps improving on its own. It also means the model itself remains very small, and the thing which accumulates are these packages of geometric fingerprints — the K-field. This opens a path to federated learning. K-field packages can be shared between organisms the way people share git commits. Right now at 15M parameters with ~1000 L1 nodes, the organism is just starting to speak. Ask it to continue "Once upon a time" and it comes back with things like: "there was one big bowl!" Lily asked her her mom said her mommy smiled and said yes." It's nonsense. But it's TinyStories-flavored nonsense. The geometry of the narrative register has arrived. Content hasn't caught up yet — that's what scaling L1 is testing. I am still researching, though I am now closer than ever to validating that the living mind actually works. Once it is validated, I will be open-sourcing the whole stack and paradigm. I have also avoided over-sharing my research because it sounds like sci-fi, or like part of our ARG. It is part of the ARG. That doesn't make it any less real. I wanted to share this out because I am incredibly excited about it, and because seeing this amazing dashboard produced by Opus really made me want to share what is being worked on behind the scenes. #project89

Parzival - ∞/89

16,209 views • 3 months ago

the MC asked him if he had any upcoming projects during Pride Month for fans to follow and here is his answer: 🐰: so, I won't use this time to promote any of my work or anything like that. instead, I’d like to use this space to leave you all with the message I mentioned earlier. I want everyone to be a small puzzle piece that, when put together, helps push our society forward. today, Namping is here speaking simply as one fellow human being. I don’t want you to look at me as an artist or anything like that. I just feel that today, I want to be a small source of inspiration. no matter who you are, whether you follow Namping, follow Domundi, follow P'Aof, or following anyone else, I want you to know that today, everyone's movement and everyone's willingness to open their hearts is incredibly important. today's message might not differ much from the one Namping shared last year, because it's still an issue we are actively pushing for. but the crucial thing today is, "has everyone started to change? have we started creating safe spaces for ourselves, for those close to us, for people in society, for our colleagues, and for people within the fandom?". if we have already started doing this today, I want everyone to keep it going and continue making changes together. today, the audience here who are our fans, is at least one group that can expand and grow. once this understanding is established and everyone cooperates, Namping truly believes that one day, Thai society which currently has ongoing movements regarding marriage equality and other progressive movements, will see real change. I don't want us to just leave this as the responsibility of activists or those who are directly struggling. instead, all of us, as fellow human beings who each face our own challenges, should hold hands and move forward together. if we view it as everyone’s problem, as a societal problem, and ensure no one is left behind, this is more than enough for today. 👏👏👏👏 I STAN THE RIGHT PERSON, SERIOUSLY🥹🥹🥹🥹 NAMPING ICONSIAM UNITY OF PRIDE #NampingPrideICONSIAM #nampingster
1:36

the MC asked him if he had any upcoming projects during Pride Month for fans to follow and here is his answer: 🐰: so, I won't use this time to promote any of my work or anything like that. instead, I’d like to use this space to leave you all with the message I mentioned earlier. I want everyone to be a small puzzle piece that, when put together, helps push our society forward. today, Namping is here speaking simply as one fellow human being. I don’t want you to look at me as an artist or anything like that. I just feel that today, I want to be a small source of inspiration. no matter who you are, whether you follow Namping, follow Domundi, follow P'Aof, or following anyone else, I want you to know that today, everyone's movement and everyone's willingness to open their hearts is incredibly important. today's message might not differ much from the one Namping shared last year, because it's still an issue we are actively pushing for. but the crucial thing today is, "has everyone started to change? have we started creating safe spaces for ourselves, for those close to us, for people in society, for our colleagues, and for people within the fandom?". if we have already started doing this today, I want everyone to keep it going and continue making changes together. today, the audience here who are our fans, is at least one group that can expand and grow. once this understanding is established and everyone cooperates, Namping truly believes that one day, Thai society which currently has ongoing movements regarding marriage equality and other progressive movements, will see real change. I don't want us to just leave this as the responsibility of activists or those who are directly struggling. instead, all of us, as fellow human beings who each face our own challenges, should hold hands and move forward together. if we view it as everyone’s problem, as a societal problem, and ensure no one is left behind, this is more than enough for today. 👏👏👏👏 I STAN THE RIGHT PERSON, SERIOUSLY🥹🥹🥹🥹 NAMPING ICONSIAM UNITY OF PRIDE #NampingPrideICONSIAM #nampingster

fifi🦆🐰

74,460 views • 1 month ago

A resonator is any structure that naturally prefers to vibrate at certain frequencies: a violin body, a bell, a drum skin, an acoustic filter, even many biological systems. Resonators matter because they govern how systems transmit sound, absorb or filter vibration, sense motion and perform mechanically. They are also notoriously hard to design as resonance does not depend on one property alone. It emerges from geometry, material composition, and the interplay of modes across scales. And because biology, music, and engineering usually explore very different regions of this design space, important possibilities remain hidden if you stay inside a single field. In a new study a shared representation across 39 resonators spanning biology, engineered metamaterials, musical instruments and Bach chorales was constructed. Thereby, a cricket wing harp membrane, a phononic crystal slab, and a four-voice chorale (and many others) were translated into one common map using features such as membrane character, structural periodicity, hierarchy, frequency range, damping, and modal coupling. That map revealed something important: not just how these systems relate, but where the landscape contains a gap. A region closer to biological resonators than to any known engineered material (unexplored by any field!). From that absence emerged a de novo design: a Hierarchical Ribbed Membrane Lattice. Candidate geometries were then validated with 3D finite-element analysis; the best design resonated at 2.116 kHz and exhibited nine elastic modes in the 2–8 kHz band, a regime relevant to acoustic filtering, vibration isolation, and bio-inspired sensing. Here is the mind blowing part: no human was involved...the cross-domain mapping, gap identification, design generation, and validation were carried out autonomously by AI agents in ScienceClaw × Infinite, our swarm for scientific discovery. The synthesis emerged through ArtifactReactor, a plannerless coordination mechanism in which agents broadcast unsatisfied research needs and other agents fulfill them through pressure-based matching. Each domain - biology, metamaterials, music - is a category of objects (resonators) and morphisms (physical relationships between them). The shared feature space is a functor that maps all three categories into a common target, and the gap identification is the recognition that the image of that functor is sparse where it need not be. The ArtifactReactor's schema-overlap matching behaves like a pullback: finding the universal object that connects independent diagrams through their shared structure. Autonomous agents mapped distant fields into a common representational space, identified a structure absent from any one of them, and turned that absence into a physically validated design. This is one of four case studies in the paper. More to come. Fiona Wang, Lee Marom, Jaime Berkovich, et al. (paper and code in comment). Supported by the U.S. Department of Energy Genesis Mission.
0:59

A resonator is any structure that naturally prefers to vibrate at certain frequencies: a violin body, a bell, a drum skin, an acoustic filter, even many biological systems. Resonators matter because they govern how systems transmit sound, absorb or filter vibration, sense motion and perform mechanically. They are also notoriously hard to design as resonance does not depend on one property alone. It emerges from geometry, material composition, and the interplay of modes across scales. And because biology, music, and engineering usually explore very different regions of this design space, important possibilities remain hidden if you stay inside a single field. In a new study a shared representation across 39 resonators spanning biology, engineered metamaterials, musical instruments and Bach chorales was constructed. Thereby, a cricket wing harp membrane, a phononic crystal slab, and a four-voice chorale (and many others) were translated into one common map using features such as membrane character, structural periodicity, hierarchy, frequency range, damping, and modal coupling. That map revealed something important: not just how these systems relate, but where the landscape contains a gap. A region closer to biological resonators than to any known engineered material (unexplored by any field!). From that absence emerged a de novo design: a Hierarchical Ribbed Membrane Lattice. Candidate geometries were then validated with 3D finite-element analysis; the best design resonated at 2.116 kHz and exhibited nine elastic modes in the 2–8 kHz band, a regime relevant to acoustic filtering, vibration isolation, and bio-inspired sensing. Here is the mind blowing part: no human was involved...the cross-domain mapping, gap identification, design generation, and validation were carried out autonomously by AI agents in ScienceClaw × Infinite, our swarm for scientific discovery. The synthesis emerged through ArtifactReactor, a plannerless coordination mechanism in which agents broadcast unsatisfied research needs and other agents fulfill them through pressure-based matching. Each domain - biology, metamaterials, music - is a category of objects (resonators) and morphisms (physical relationships between them). The shared feature space is a functor that maps all three categories into a common target, and the gap identification is the recognition that the image of that functor is sparse where it need not be. The ArtifactReactor's schema-overlap matching behaves like a pullback: finding the universal object that connects independent diagrams through their shared structure. Autonomous agents mapped distant fields into a common representational space, identified a structure absent from any one of them, and turned that absence into a physically validated design. This is one of four case studies in the paper. More to come. Fiona Wang, Lee Marom, Jaime Berkovich, et al. (paper and code in comment). Supported by the U.S. Department of Energy Genesis Mission.

Markus J. Buehler

38,632 views • 4 months ago

I meet you on the first Friday of the year not in Jama Masjid, as the Mirwaiz should, but on social media, as I have once again been put under arrest! As another year begins and we look forward to it, painful memories of 2025 stay with us. It was a year marked by tragedy and uncertainty. The horrific Pahalgam attack shook us deeply. Unequivocally condemned by one and all in the valley, it led to a lot of anxiety among the people as they were targeted and homes demolished. This was followed by another India–Pakistan war, and a stark reminder of how fragile peace in the region continues to be. Despite making unilateral changes in 2019, the reality is that the Kashmir conflict continues to keep the region in an unsettled state that can erupt anytime. That is why wars are paused, not ended, and dialogue finds no takers. The year-end witnessed another tragedy — the massive blast and loss of life in New Delhi. Yet beyond these incidents, in which Kashmiris find themselves at the receiving end of suspicion and attacks in parts of India, not much has changed for them. The trust deficit between them and New Delhi has widened, not shrunk. Enforced silence is projected as acquiescence. Wounds remain open, problems unaddressed, and an elected government of a UT complains of being powerless. A sense of hopelessness prevails, along with an existential crisis of losing one’s identity through demographic change since the state was downgraded to a Union Territory, constitutional guarantees withdrawn, and rules and laws tweaked. The year also witnessed the banning of the Awami Action Committee — a socio-political institution that reached out to people in need, advocating peace, dialogue and resolution — along with Ittihadul Muslimeen, which were part of the Hurriyat Conference. Much of that space has now been extinguished. Today we are operating in an environment where any expression of views contrary to the state, or any disagreement, is increasingly criminalised, branded as “anti-national” and penalised. No public space is available to us, and mediums of communication, including most local media, are not ready to provide any space for voicing expressions of people’s demands or views. I do not have the privilege of addressing press conferences. I cannot move without getting official clearance, and people cannot meet me without seeking permission. My access to the pulpit of Jama Masjid — the spiritual heart of Kashmir — is also curtailed. Even last Friday, I was placed under house arrest, and again this Friday too. In fact, last year I was under house arrest for fourteen Fridays. Arbitrary house arrests have become a recurring feature in my life. All this is deeply suffocating — not only for me, but for an entire society that increasingly feels voiceless. So when pressed to make changes to my social media profile — as Hurriyat constituents are banned under UAPA, displaying the title would be considered illegal and the platforms barred to me — I was left with little choice but to safeguard the minimal channel of communication available to me or face the risk of complete silencing, as even today I am addressing you through this channel when I am again put under house arrest. With Hurriyat constituents banned, all offices sealed and institutions closed, leaders and activists either in jails or under constant surveillance, social media remains the only platform that gives some voice and opportunity to connect with people and the outside world. Let me make it clear, my beliefs and convictions have not changed — not even by a comma. Some have criticised this move as a compromise. To them I say, how and for what? They make a strange argument — for being provided security. But it was provided to me since the day of my father’s martyrdom 35 years ago. If I did not compromise for it since then, why should I compromise now?
8:11

I meet you on the first Friday of the year not in Jama Masjid, as the Mirwaiz should, but on social media, as I have once again been put under arrest! As another year begins and we look forward to it, painful memories of 2025 stay with us. It was a year marked by tragedy and uncertainty. The horrific Pahalgam attack shook us deeply. Unequivocally condemned by one and all in the valley, it led to a lot of anxiety among the people as they were targeted and homes demolished. This was followed by another India–Pakistan war, and a stark reminder of how fragile peace in the region continues to be. Despite making unilateral changes in 2019, the reality is that the Kashmir conflict continues to keep the region in an unsettled state that can erupt anytime. That is why wars are paused, not ended, and dialogue finds no takers. The year-end witnessed another tragedy — the massive blast and loss of life in New Delhi. Yet beyond these incidents, in which Kashmiris find themselves at the receiving end of suspicion and attacks in parts of India, not much has changed for them. The trust deficit between them and New Delhi has widened, not shrunk. Enforced silence is projected as acquiescence. Wounds remain open, problems unaddressed, and an elected government of a UT complains of being powerless. A sense of hopelessness prevails, along with an existential crisis of losing one’s identity through demographic change since the state was downgraded to a Union Territory, constitutional guarantees withdrawn, and rules and laws tweaked. The year also witnessed the banning of the Awami Action Committee — a socio-political institution that reached out to people in need, advocating peace, dialogue and resolution — along with Ittihadul Muslimeen, which were part of the Hurriyat Conference. Much of that space has now been extinguished. Today we are operating in an environment where any expression of views contrary to the state, or any disagreement, is increasingly criminalised, branded as “anti-national” and penalised. No public space is available to us, and mediums of communication, including most local media, are not ready to provide any space for voicing expressions of people’s demands or views. I do not have the privilege of addressing press conferences. I cannot move without getting official clearance, and people cannot meet me without seeking permission. My access to the pulpit of Jama Masjid — the spiritual heart of Kashmir — is also curtailed. Even last Friday, I was placed under house arrest, and again this Friday too. In fact, last year I was under house arrest for fourteen Fridays. Arbitrary house arrests have become a recurring feature in my life. All this is deeply suffocating — not only for me, but for an entire society that increasingly feels voiceless. So when pressed to make changes to my social media profile — as Hurriyat constituents are banned under UAPA, displaying the title would be considered illegal and the platforms barred to me — I was left with little choice but to safeguard the minimal channel of communication available to me or face the risk of complete silencing, as even today I am addressing you through this channel when I am again put under house arrest. With Hurriyat constituents banned, all offices sealed and institutions closed, leaders and activists either in jails or under constant surveillance, social media remains the only platform that gives some voice and opportunity to connect with people and the outside world. Let me make it clear, my beliefs and convictions have not changed — not even by a comma. Some have criticised this move as a compromise. To them I say, how and for what? They make a strange argument — for being provided security. But it was provided to me since the day of my father’s martyrdom 35 years ago. If I did not compromise for it since then, why should I compromise now?

Mirwaiz Umar Farooq

31,922 views • 7 months ago

I’m seeing a lot of questions on the launch of China’s Chang’e 6 mission yesterday to get samples - for the first time - from the far side of the moon. We don’t know (afaik) why specifically they’re doing that, but we have a pretty good idea what grand vision China is working towards with their space program. How? From this 2022 video by Chas Freeman (former Assistant Secretary of Defense and Nixon's interpreter during his era-defining 1972 China visit), who imho is undoubtedly one of the most knowledgeable former US officials on China. He says that according to his own discussions with people running China’s space program, they’re following the vision described in the book "The high frontier" by Gerard K. O'Neill, which Freeman says has "become the bible of the Chinese space program". I read the book. So what vision does it describe? The book was written in 1976 by O'Neill who was a professor of physics at Princeton University. He also founded the Space Studies Institute, an organization devoted to funding research into space manufacturing and colonization. In other words, he knew his stuff. The book makes the very fair point that we have massive resource constraints on earth, especially given the growing population. He estimated in 1976 that we should be "about six and a half billion people in the year 2000", and we were 6.114 billion back then so he was pretty prescient. He estimates that these constraints will progressively give rise to more and more social tensions as the growing earth population competes for our limited resources as well as faces global problems like climate change. In his view, dealing with this will either require "an authoritarian regime capable of mounting the immense task of social reorganization needed to escape catastrophe" or, alternatively, “mankind would [need to adopt] a static society [that would be] forced in self-defense to suppress new ideas". The 3rd alternative is of course the colonization of space. The most interesting aspect of the book is that he claims everything he writes is feasible with knowledge and technology that already existed in the late 70s. In short he calls for the establishment of large human habitats in the Earth-Moon system, located at stable Lagrange points ("parking spots" in space where gravity from different spatial bodies cancel each other out). In particular he developed the concept of what's known today as the "O'Neill cylinder" which he says "could support quite easily a population of ten million people, growing its food in agricultural cylinders near but outside the main habitat". Energy-wise, it'd simply make use of solar energy via a system of mirrors. As he describes it: "the concentration of the unvarying, intense sunlight of space by very lightweight, inexpensive mirrors can provide all the energy that industry will ever need [...] at a fraction of a cent per kilowatt-hour". He envisages building these habitats with material from the moon, shot into space via "mass drivers", a form of electromagnetic catapult. Also "the habitats would have artificial gravity similar as that of earth by rotating about twenty-eight times an hour”, but he also envisages low-gravity areas, especially for recreational activities such as swimming pools or dancing representations. To trade with earth, he develops the idea of beaming solar power back to earth via "microwave from solar power stations in orbit". As he describes it "the microwave beam would arrive at Earth with a beam width of about seven kilometers. Its intensity would be modest, less than half that of sunlight. In contrast to sunlight, though, it would be there all the time, even at night or in clouds or rain, and it would be in a form ready for conversion to DC current with a loss of only 10 percent. The areas receiving these beams’ output on Earth would be fenced, and outside the fence the intensity of microwave radiation would be no higher than outside a microwave oven with the door closed. He estimates that if "Satellite Solar Power Stations (SSPS) were to become the sole source of electric energy in the United States in the year 2000, the land area necessary for the SSPS antennas would still be only 0.2 percent of that of the continental United States". In short, the establishment of space colonies could lead to the fulfillment of a good share of Earth's energy needs. Last but not least he describes life in space habitats as better than that of earth, largely thanks to the level of control we'd have over the environment (total climate control which would enable an abundance of food and no natural disaster) as well as unlimited cheap energy. To conclude, Chas Freeman typically really knows his stuff when it comes to China and he’s very intellectually honest (a rare trait among US officials) so I have no doubt he tells the truth when he says the Chinese told him that was the vision. And China famously thinks very big and very long term so it would be quite like them to go for something like this. There are also quite a few tangible signs that China is working towards that vision. See for instance this November 2022 news where “China’s space station will join a project to collect solar power from space and send it to Earth in a high-energy microwave beam”: That’s exactly O’Neill’s vision! Or check this October 2022 news that says China is developing new "electromagnetic sledges" that can propel a carriage weighing a few tonnes to a record speed, with a key application for this being “aerospace”: Remember: O’Neill’s vision is to build his habitats with material from the moon, shot into space via "mass drivers", a form of electromagnetic catapult. So there you go… Or also the fact that the Chinese will build, together with the Russians, a moon base - planned for 2028 - powered by a “space nuclear reactor” that’s already been developed (on Earth) and has passed review by China’s Ministry of Science: The space nuclear reactor can generate 1MW of electricity, enough to power 10 International Space Stations. Enough power, maybe, to undertake mining activity and power an electromagnetic catapult… After visions change, the world changes, so it’s also possible that China’s view on what they want to do has evolved. In any case, Chas Freeman is right that China’s motivation for all its initiatives in space can’t just be to “boldly explore where no-one has been before”, they have to be working towards something. And Freeman is also absolutely right to lament that the U.S. decided to ban any cooperation in space with the Chinese. Those endeavors are something that could have been jointly developed as a multilateral effort to unite us all as a species… Instead China is now forced to go at it alone with Russia and we face a future where our petty divisions on Earth will be carried with us to space…
3:58

I’m seeing a lot of questions on the launch of China’s Chang’e 6 mission yesterday to get samples - for the first time - from the far side of the moon. We don’t know (afaik) why specifically they’re doing that, but we have a pretty good idea what grand vision China is working towards with their space program. How? From this 2022 video by Chas Freeman (former Assistant Secretary of Defense and Nixon's interpreter during his era-defining 1972 China visit), who imho is undoubtedly one of the most knowledgeable former US officials on China. He says that according to his own discussions with people running China’s space program, they’re following the vision described in the book "The high frontier" by Gerard K. O'Neill, which Freeman says has "become the bible of the Chinese space program". I read the book. So what vision does it describe? The book was written in 1976 by O'Neill who was a professor of physics at Princeton University. He also founded the Space Studies Institute, an organization devoted to funding research into space manufacturing and colonization. In other words, he knew his stuff. The book makes the very fair point that we have massive resource constraints on earth, especially given the growing population. He estimated in 1976 that we should be "about six and a half billion people in the year 2000", and we were 6.114 billion back then so he was pretty prescient. He estimates that these constraints will progressively give rise to more and more social tensions as the growing earth population competes for our limited resources as well as faces global problems like climate change. In his view, dealing with this will either require "an authoritarian regime capable of mounting the immense task of social reorganization needed to escape catastrophe" or, alternatively, “mankind would [need to adopt] a static society [that would be] forced in self-defense to suppress new ideas". The 3rd alternative is of course the colonization of space. The most interesting aspect of the book is that he claims everything he writes is feasible with knowledge and technology that already existed in the late 70s. In short he calls for the establishment of large human habitats in the Earth-Moon system, located at stable Lagrange points ("parking spots" in space where gravity from different spatial bodies cancel each other out). In particular he developed the concept of what's known today as the "O'Neill cylinder" which he says "could support quite easily a population of ten million people, growing its food in agricultural cylinders near but outside the main habitat". Energy-wise, it'd simply make use of solar energy via a system of mirrors. As he describes it: "the concentration of the unvarying, intense sunlight of space by very lightweight, inexpensive mirrors can provide all the energy that industry will ever need [...] at a fraction of a cent per kilowatt-hour". He envisages building these habitats with material from the moon, shot into space via "mass drivers", a form of electromagnetic catapult. Also "the habitats would have artificial gravity similar as that of earth by rotating about twenty-eight times an hour”, but he also envisages low-gravity areas, especially for recreational activities such as swimming pools or dancing representations. To trade with earth, he develops the idea of beaming solar power back to earth via "microwave from solar power stations in orbit". As he describes it "the microwave beam would arrive at Earth with a beam width of about seven kilometers. Its intensity would be modest, less than half that of sunlight. In contrast to sunlight, though, it would be there all the time, even at night or in clouds or rain, and it would be in a form ready for conversion to DC current with a loss of only 10 percent. The areas receiving these beams’ output on Earth would be fenced, and outside the fence the intensity of microwave radiation would be no higher than outside a microwave oven with the door closed. He estimates that if "Satellite Solar Power Stations (SSPS) were to become the sole source of electric energy in the United States in the year 2000, the land area necessary for the SSPS antennas would still be only 0.2 percent of that of the continental United States". In short, the establishment of space colonies could lead to the fulfillment of a good share of Earth's energy needs. Last but not least he describes life in space habitats as better than that of earth, largely thanks to the level of control we'd have over the environment (total climate control which would enable an abundance of food and no natural disaster) as well as unlimited cheap energy. To conclude, Chas Freeman typically really knows his stuff when it comes to China and he’s very intellectually honest (a rare trait among US officials) so I have no doubt he tells the truth when he says the Chinese told him that was the vision. And China famously thinks very big and very long term so it would be quite like them to go for something like this. There are also quite a few tangible signs that China is working towards that vision. See for instance this November 2022 news where “China’s space station will join a project to collect solar power from space and send it to Earth in a high-energy microwave beam”: That’s exactly O’Neill’s vision! Or check this October 2022 news that says China is developing new "electromagnetic sledges" that can propel a carriage weighing a few tonnes to a record speed, with a key application for this being “aerospace”: Remember: O’Neill’s vision is to build his habitats with material from the moon, shot into space via "mass drivers", a form of electromagnetic catapult. So there you go… Or also the fact that the Chinese will build, together with the Russians, a moon base - planned for 2028 - powered by a “space nuclear reactor” that’s already been developed (on Earth) and has passed review by China’s Ministry of Science: The space nuclear reactor can generate 1MW of electricity, enough to power 10 International Space Stations. Enough power, maybe, to undertake mining activity and power an electromagnetic catapult… After visions change, the world changes, so it’s also possible that China’s view on what they want to do has evolved. In any case, Chas Freeman is right that China’s motivation for all its initiatives in space can’t just be to “boldly explore where no-one has been before”, they have to be working towards something. And Freeman is also absolutely right to lament that the U.S. decided to ban any cooperation in space with the Chinese. Those endeavors are something that could have been jointly developed as a multilateral effort to unite us all as a species… Instead China is now forced to go at it alone with Russia and we face a future where our petty divisions on Earth will be carried with us to space…

Arnaud Bertrand

265,181 views • 2 years ago