Загрузка видео...

Не удалось загрузить видео

На главную

We present “3D magician”: TADA! Text to Animatable Digital Avatars. Given a textual description as input only, our method TADA generates expressive animatable 3D avatars with high-quality geometry and lifelike textures. (1/10)

HongweiYi2's profile picture

Hongwei Yi

2,991 subscribers

52,306 просмотров • 2 лет назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

(1/2) Happy to announce that Text2Tex has been accepted at #ICCV2023 🎉 Taking a mesh and a text prompt as input, Text2Tex generates high quality textures - it's fully automated and easy to scale to many models! Project: Video:
0:12

(1/2) Happy to announce that Text2Tex has been accepted at #ICCV2023 🎉 Taking a mesh and a text prompt as input, Text2Tex generates high quality textures - it's fully automated and easy to scale to many models! Project: Video:

Matthias Niessner

44,117 просмотров • 3 лет назад

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]
0:16

📢🚗✨ Excited to announce InfiniCube, our scalable generative model for dynamic 3D driving scene generation with high fidelity and controllability! InfiniCube generates very large-scale (300m×400m ~ 100,000m^2), dynamic 3D driving scenes given HD maps, 3D bounding boxes, and text prompts as controls. Website: Props to Yifan Lu for leading this project!!!! [1/N]

Xuanchi Ren

26,602 просмотров • 1 год назад

As we move more and more into 3D, texturing 3D assets will be essential ☝️ Repainting 3D Assets is a new AI method that can take any 3D asset and paint it with a given text prompt. The results, while low-resolution, are pretty impressive.
0:18

As we move more and more into 3D, texturing 3D assets will be essential ☝️ Repainting 3D Assets is a new AI method that can take any 3D asset and paint it with a given text prompt. The results, while low-resolution, are pretty impressive.

Dreaming Tulpa 🥓👑

16,291 просмотров • 2 лет назад

After 4 months in beta Avaturn avatars with ARKit blendshapes support released! We are live on Product Hunt! Support us at We also added TONS of new Features (see detailed description in the comment below) Shoutout to Chris Messina for hunting us, to Three.js for awesome dev tool, and a huge thank you to our phenomenal community of game developers & 3D creators - you make this journey worthwhile! #ProductHunt #3D #Avatars #Metaverse #GameDev
0:12

After 4 months in beta Avaturn avatars with ARKit blendshapes support released! We are live on Product Hunt! Support us at We also added TONS of new Features (see detailed description in the comment below) Shoutout to Chris Messina for hunting us, to Three.js for awesome dev tool, and a huge thank you to our phenomenal community of game developers & 3D creators - you make this journey worthwhile! #ProductHunt #3D #Avatars #Metaverse #GameDev

Avaturn

90,318 просмотров • 3 лет назад

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians paper page: Creating high-fidelity 3D head avatars has always been a research hotspot, but there remains a great challenge under lightweight sparse view setups. In this paper, we propose Gaussian Head Avatar represented by controllable 3D Gaussians for high-fidelity head avatar modeling. We optimize the neutral 3D Gaussians and a fully learned MLP-based deformation field to capture complex expressions. The two parts benefit each other, thereby our method can model fine-grained dynamic details while ensuring expression accuracy. Furthermore, we devise a well-designed geometry-guided initialization strategy based on implicit SDF and Deep Marching Tetrahedra for the stability and convergence of the training procedure. Experiments show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.
0:12

Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians paper page: Creating high-fidelity 3D head avatars has always been a research hotspot, but there remains a great challenge under lightweight sparse view setups. In this paper, we propose Gaussian Head Avatar represented by controllable 3D Gaussians for high-fidelity head avatar modeling. We optimize the neutral 3D Gaussians and a fully learned MLP-based deformation field to capture complex expressions. The two parts benefit each other, thereby our method can model fine-grained dynamic details while ensuring expression accuracy. Furthermore, we devise a well-designed geometry-guided initialization strategy based on implicit SDF and Deep Marching Tetrahedra for the stability and convergence of the training procedure. Experiments show our approach outperforms other state-of-the-art sparse-view methods, achieving ultra high-fidelity rendering quality at 2K resolution even under exaggerated expressions.

AK

65,847 просмотров • 2 лет назад

Say hello to $TADA! 👋🏽 📲 With the SwissBorg app, you can access #TADA (Ta-da) in 16 fiat currencies with just a few taps! 🤖 For AI to keep improving, we need high-quality, curated, and specialised datasets. 🧑🏿‍🤝‍🧑🏽🧑‍🤝‍🧑Ta-da offers human-generated datasets crowdsourced from 100k app users who are rewarded for their input. The more people contribute to their datasets, the more powerful Ta-da becomes!
0:11

Say hello to $TADA! 👋🏽 📲 With the SwissBorg app, you can access #TADA (Ta-da) in 16 fiat currencies with just a few taps! 🤖 For AI to keep improving, we need high-quality, curated, and specialised datasets. 🧑🏿‍🤝‍🧑🏽🧑‍🤝‍🧑Ta-da offers human-generated datasets crowdsourced from 100k app users who are rewarded for their input. The more people contribute to their datasets, the more powerful Ta-da becomes!

SwissBorg

24,579 просмотров • 1 год назад

Here is a preview of the new bang options for our 3D customizable vtuber model! ✨ Not everyone can afford a high end 1:1 3D vtuber model. Which is why my sister and I are working so hard to provide a customizable 3D vtuber avatar with the same level of quality and tech!🩷
0:11

Here is a preview of the new bang options for our 3D customizable vtuber model! ✨ Not everyone can afford a high end 1:1 3D vtuber model. Which is why my sister and I are working so hard to provide a customizable 3D vtuber avatar with the same level of quality and tech!🩷

Pastell & Palette🤍 3D Customizable Model!

22,189 просмотров • 1 месяц назад

📢GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans📢 We present a novel method to reconstruct hair strands from colorless 3D scans by extracting orientation cues directly from the mesh surface geometry by finding local characteristic lines and from shaded renderings using a neural 2D line detector. We enhance the reconstruction with a diffusion prior trained on synthetic hair data and adapted to each scan using a tailored text prompt, allowing us to recover both simple and complex hairstyles without relying on color input. To support further research, we also introduce Strands400, the largest publicly available dataset of 3D hair strand reconstructions from real-world scans of 400 different people, featuring complicated hairstyles, such as ponytails and buns. 🌍 📷 Great work by Rachmadio Noval L. Artem Sevastopolsky Egor Zakharov @ness_pris
0:15

📢GeomHair: Reconstruction of Hair Strands from Colorless 3D Scans📢 We present a novel method to reconstruct hair strands from colorless 3D scans by extracting orientation cues directly from the mesh surface geometry by finding local characteristic lines and from shaded renderings using a neural 2D line detector. We enhance the reconstruction with a diffusion prior trained on synthetic hair data and adapted to each scan using a tailored text prompt, allowing us to recover both simple and complex hairstyles without relying on color input. To support further research, we also introduce Strands400, the largest publicly available dataset of 3D hair strand reconstructions from real-world scans of 400 different people, featuring complicated hairstyles, such as ponytails and buns. 🌍 📷 Great work by Rachmadio Noval L. Artem Sevastopolsky Egor Zakharov @ness_pris

Matthias Niessner

12,466 просмотров • 1 год назад

(1/2) MonoNPHM will be presented as a #CVPR2024 Highlight! Our Neural Parametric Head Model parametrizes both geometry and appearance. With the learned model, we can then 3D reconstruct and track human heads from images or videos.
0:18

(1/2) MonoNPHM will be presented as a #CVPR2024 Highlight! Our Neural Parametric Head Model parametrizes both geometry and appearance. With the learned model, we can then 3D reconstruct and track human heads from images or videos.

Matthias Niessner

17,209 просмотров • 2 лет назад

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.
0:13

GSTAR: Gaussian Surface Tracking and Reconstruction Contributions: • A new framework for tracking and reconstructing dynamic scenes, combining 3D Gaussians and meshes to effectively manage changes in topology. • A method for Gaussian unbinding and surface re-meshing, allowing for the generation of new surfaces as topologies evolve. • A method for handling large or fast deformations of surfaces between frames using scene flow warping. Abstract (excerpt): However, tracking dynamic surfaces with 3D Gaussians remains challenging due to complex topology changes, such as surfaces appearing, disappearing, or splitting. To address these challenges, we propose GSTAR, a novel method that achieves photo-realistic rendering, accurate surface reconstruction, and reliable 3D tracking for general dynamic scenes with changing topology. Given multi-view captures as input, GSTAR binds Gaussians to mesh faces to represent dynamic objects. For surfaces with consistent topology, GSTAR maintains the mesh topology and tracks the meshes using Gaussians.

MrNeRF

22,698 просмотров • 1 год назад

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.
0:19

DroneSplat: 3D Gaussian Splatting for Robust 3D Reconstruction from In-the-Wild Drone Imagery Abstract: Drones have become essential tools for reconstructing wild scenes due to their outstanding maneuverability. Recent advances in radiance field methods have achieved remarkable rendering quality, providing a new avenue for 3D reconstruction from drone imagery. However, dynamic distractors in wild environments challenge the static scene assumption in radiance fields, while limited view constraints hinder the accurate capture of underlying scene geometry. To address these challenges, we introduce DroneSplat, a novel framework designed for robust 3D reconstruction from in-the-wild drone imagery. Our method adaptively adjusts masking thresholds by integrating local-global segmentation heuristics with statistical approaches, enabling precise identification and elimination of dynamic distractors in static scenes. We enhance 3D Gaussian Splatting with multi-view stereo predictions and a voxel-guided optimization strategy, supporting high-quality rendering under limited view constraints. For comprehensive evaluation, we provide a drone-captured 3D reconstruction dataset encompassing both dynamic and static scenes. Extensive experiments demonstrate that DroneSplat outperforms both 3DGS and NeRF baselines in handling in-the-wild drone imagery.

MrNeRF

21,346 просмотров • 1 год назад

How to generate 3D miniature city models and animate them using Kling AI? This visual effect can be created with Kling O1, and then rotated in 3D using Image to Video. The image prompt used for generation is as follows: Present a clear, 45° top-down isometric miniature 3D cartoon scene of New York featuring its most iconic landmarks and architectural elements. Use soft, refined textures with realistic PBR materials and gentle, lifelike lighting and shadows. Integrate the current weather conditions directly into the city environment to create an immersive atmospheric mood.Use a clean, minimalistic composition with a soft, solid-colored background. At the top-center, place the title "New York" in large bold text in white. You can create different effects by changing the city name according to your needs.
0:10

How to generate 3D miniature city models and animate them using Kling AI? This visual effect can be created with Kling O1, and then rotated in 3D using Image to Video. The image prompt used for generation is as follows: Present a clear, 45° top-down isometric miniature 3D cartoon scene of New York featuring its most iconic landmarks and architectural elements. Use soft, refined textures with realistic PBR materials and gentle, lifelike lighting and shadows. Integrate the current weather conditions directly into the city environment to create an immersive atmospheric mood.Use a clean, minimalistic composition with a soft, solid-colored background. At the top-center, place the title "New York" in large bold text in white. You can create different effects by changing the city name according to your needs.

Kling AI

34,303 просмотров • 7 месяцев назад

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:
0:12

Break-A-Scene: Extracting Multiple Concepts from a Single Image introduce the task of textual scene decomposition: given a single image of a scene that may contain several concepts, we aim to extract a distinct text token for each concept, enabling fine-grained control over the generated scenes. To this end, we propose augmenting the input image with masks that indicate the presence of target concepts. These masks can be provided by the user or generated automatically by a pre-trained segmentation model. We then present a novel two-phase customization process that optimizes a set of dedicated textual embeddings (handles), as well as the model weights, striking a delicate balance between accurately capturing the concepts and avoiding overfitting. We employ a masked diffusion loss to enable handles to generate their assigned concepts, complemented by a novel loss on cross-attention maps to prevent entanglement. We also introduce union-sampling, a training strategy aimed to improve the ability of combining multiple concepts in generated images. We use several automatic metrics to quantitatively compare our method against several baselines, and further affirm the results using a user study. Finally, we showcase several applications of our method paper page:

AK

154,511 просмотров • 3 лет назад

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.
0:18

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.

Sakana AI

403,145 просмотров • 1 год назад

Our focus is to bring immense value to our members. We’re going to be releasing many exciting products for OCMs. We believe that Bitcoin will be home to the highest value digital assets and we’re excited to be building there for all OCMs. Our newest collection coming to Bitcoin, OCM Dimensions, will be a first of its kind digital asset collection. Dimensions is high-end, 3D generative art. Our collectors will access the most high quality and groundbreaking digital assets, and only available through OCM. After Dimensions, we'll release another collection for our holders. This collection will be designed by renowned artist Alexis André, creator of Friendship Bracelets. Now is an amazing time to become a member of our Wealthy Digital Nation. Join our Discord for more details.
0:11

Our focus is to bring immense value to our members. We’re going to be releasing many exciting products for OCMs. We believe that Bitcoin will be home to the highest value digital assets and we’re excited to be building there for all OCMs. Our newest collection coming to Bitcoin, OCM Dimensions, will be a first of its kind digital asset collection. Dimensions is high-end, 3D generative art. Our collectors will access the most high quality and groundbreaking digital assets, and only available through OCM. After Dimensions, we'll release another collection for our holders. This collection will be designed by renowned artist Alexis André, creator of Friendship Bracelets. Now is an amazing time to become a member of our Wealthy Digital Nation. Join our Discord for more details.

OnChainMonkey®

37,312 просмотров • 3 лет назад

Visual Preset #01 Ink-Brush Cinematic 3D: A high-end cinematic 3D style where expressive ink-brush effects become the primary visual language for fast-paced anime action. Lately I've noticed that I've been experimenting with different visual presets across my videos and I'd like to explore that direction even further. Going forward, I'll be sharing some of these style experiments. The video below was generated using only a character sheet, a single-line scene description and the visual preset shown below. Created with Seedance 2.0 on Try ArtCraft Seedance 2.0 Prompt: A mesmerizing display of @[character]'s masterful swordsmanship. High-end cinematic 3D realism fused with expressive ink-brush action. High-sakuga anime choreography, sweeping sumi-e brush strokes, flowing ink splashes, dynamic calligraphic energy and graphic black ink trails define every movement. Extreme perspective, dramatic foreshortening, cinematic tracking shots, volumetric lighting, heavy atmospheric haze and explosive ink bursts replace conventional visual effects, while realistic materials and feature-film rendering preserve depth, weight and scale.
0:15

Visual Preset #01 Ink-Brush Cinematic 3D: A high-end cinematic 3D style where expressive ink-brush effects become the primary visual language for fast-paced anime action. Lately I've noticed that I've been experimenting with different visual presets across my videos and I'd like to explore that direction even further. Going forward, I'll be sharing some of these style experiments. The video below was generated using only a character sheet, a single-line scene description and the visual preset shown below. Created with Seedance 2.0 on Try ArtCraft Seedance 2.0 Prompt: A mesmerizing display of @[character]'s masterful swordsmanship. High-end cinematic 3D realism fused with expressive ink-brush action. High-sakuga anime choreography, sweeping sumi-e brush strokes, flowing ink splashes, dynamic calligraphic energy and graphic black ink trails define every movement. Extreme perspective, dramatic foreshortening, cinematic tracking shots, volumetric lighting, heavy atmospheric haze and explosive ink bursts replace conventional visual effects, while realistic materials and feature-film rendering preserve depth, weight and scale.

Kōda

38,527 просмотров • 16 дней назад

ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxies Contributions: 1) We propose ImmerseGen, a novel agent-guided 3D environment generation framework. It uses simplified geometric proxies with alpha-textured meshes to produce compact, photorealistic worlds ready for real-time mobile VR rendering. 2) We propose a novel RGBA texturing paradigm. It first synthesizes 8K terrain textures using a geometry-conditioned panorama generator via user-centric mapping, and then directly generates alpha-textured proxy assets, avoiding fidelity loss typically resulting from mesh decimation. 3) To automate scene creation from user prompts, we introduce VLM-based modeling agents equipped with a novel grid-based semantic analysis. This enables 3D spatial reasoning from 2D observations and ensures accurate asset placement. ImmerseGen further enhances immersion with dynamic effects and ambient audio for a multisensory experience. 4) Experiments on multiple scene-generation scenarios and live mobile VR applications show that ImmerseGen outperforms previous methods in visual quality, realism, spatial coherence, and rendering efficiency for immersive real-time VR experiences.
0:17

ImmerseGen: Agent-Guided Immersive World Generation with Alpha-Textured Proxies Contributions: 1) We propose ImmerseGen, a novel agent-guided 3D environment generation framework. It uses simplified geometric proxies with alpha-textured meshes to produce compact, photorealistic worlds ready for real-time mobile VR rendering. 2) We propose a novel RGBA texturing paradigm. It first synthesizes 8K terrain textures using a geometry-conditioned panorama generator via user-centric mapping, and then directly generates alpha-textured proxy assets, avoiding fidelity loss typically resulting from mesh decimation. 3) To automate scene creation from user prompts, we introduce VLM-based modeling agents equipped with a novel grid-based semantic analysis. This enables 3D spatial reasoning from 2D observations and ensures accurate asset placement. ImmerseGen further enhances immersion with dynamic effects and ambient audio for a multisensory experience. 4) Experiments on multiple scene-generation scenarios and live mobile VR applications show that ImmerseGen outperforms previous methods in visual quality, realism, spatial coherence, and rendering efficiency for immersive real-time VR experiences.

MrNeRF

14,225 просмотров • 1 год назад

home page hero ✨ Design notes: - "Forever" hero text dot pixel FX done in unicorn.studio. (I will do a whole tutorial on this later. Unicorn's WebGL engine is absolutely wild and very powerful / robust) - built in Framer - I wanted to recreate the iOS unlock effect where your home screen icons cascade into place in a beautifully timed choreography. This took a lot of careful timing using Framer's "Appear" effect on the hero text and surrounding avatars because it was super important that we didn't lose the legibility of our main message ("Build Your Forever Audience") with all the animations. - If you look closely, the choreography is setup to lead your eye through the hero text first starting with "Build Your" then "Forever" and finally "Audience." - With those text layers in place + the surrounding avatars, there is a slight 1 sec pause before the remaining elements slide in below and above (How it works, CTA buttons, announcement badge, and lastly the main nav). - All told the entire loading sequence is 6 seconds - Custom particle system powers the interactive star field (the stars slowly gravitate to your pointer position, and the star field perspective changes ever so subtly as you move your mouse around on the page) - I have 3 shooting stars made of small white line layers that start out off canvas rotated at different angles that shoot across to another point off canvas at random times on a loop effect. - Given this hero scene is in space, I wanted the surrounding avatar elements to "float" in low gravity mode. For this I used Framer's loop effect that slowly oscillates the layer's y position. I then offset the delay of each element randomly to stagger the floating loop so each avatar floats independently/randomly - The final major treatment for this hero scene was the scroll animations on the avatars. I wanted to create a bit of a warp speed effect when you scroll down, as if the avatars were being pulled or sucked into a worm hole as you scroll down below this hero fold. - To accomplish this, I applied Framer's scroll transform affect set to "section in view" on each of the floating avatars, and set the "scroll to" position of the upper avatars to be much, much further away on the y-axis than the "scroll to" position of the lower avatars. (eg. -1700px on upper most avatars vs. -600px on lowest positioned avatars). This effectively causes the upper avatars to slide up off the hero canvas with much greater velocity than the lower positioned avatars. - And when you scroll back up to the hero section, the inverse happens where the lower positioned avatars "arrive back in place" from up above the hero canvas before the upper avatars come back into the scene and settle in place. - Overall I wanted this hero section to feel alive. The floating avatars, particle system with very subtle star movements, and the Caustics effect on the "Forever" text all sort of move at the pace of slow breathing - which is a great pace to create a sense of life and comfort in your scene. Conversion Results (so far) - When this new Framer site launched along with Calaxy v1.9 release on Base a couple weeks ago, we saw a surge in traffic, around 20k page views in the first few days. - Of those 20k hits, 11k visited the app install page ( - which is our main CTA - We saw around 10K new users in the first week after v1.9 launch Overall I'm very happy with the new site and early performance metrics. Lots of tweaking to do but its a good start. If you are a designer building in Framer - hit me with any questions on the above hero notes. Happy to share more specifics! 👾
0:12

home page hero ✨ Design notes: - "Forever" hero text dot pixel FX done in unicorn.studio. (I will do a whole tutorial on this later. Unicorn's WebGL engine is absolutely wild and very powerful / robust) - built in Framer - I wanted to recreate the iOS unlock effect where your home screen icons cascade into place in a beautifully timed choreography. This took a lot of careful timing using Framer's "Appear" effect on the hero text and surrounding avatars because it was super important that we didn't lose the legibility of our main message ("Build Your Forever Audience") with all the animations. - If you look closely, the choreography is setup to lead your eye through the hero text first starting with "Build Your" then "Forever" and finally "Audience." - With those text layers in place + the surrounding avatars, there is a slight 1 sec pause before the remaining elements slide in below and above (How it works, CTA buttons, announcement badge, and lastly the main nav). - All told the entire loading sequence is 6 seconds - Custom particle system powers the interactive star field (the stars slowly gravitate to your pointer position, and the star field perspective changes ever so subtly as you move your mouse around on the page) - I have 3 shooting stars made of small white line layers that start out off canvas rotated at different angles that shoot across to another point off canvas at random times on a loop effect. - Given this hero scene is in space, I wanted the surrounding avatar elements to "float" in low gravity mode. For this I used Framer's loop effect that slowly oscillates the layer's y position. I then offset the delay of each element randomly to stagger the floating loop so each avatar floats independently/randomly - The final major treatment for this hero scene was the scroll animations on the avatars. I wanted to create a bit of a warp speed effect when you scroll down, as if the avatars were being pulled or sucked into a worm hole as you scroll down below this hero fold. - To accomplish this, I applied Framer's scroll transform affect set to "section in view" on each of the floating avatars, and set the "scroll to" position of the upper avatars to be much, much further away on the y-axis than the "scroll to" position of the lower avatars. (eg. -1700px on upper most avatars vs. -600px on lowest positioned avatars). This effectively causes the upper avatars to slide up off the hero canvas with much greater velocity than the lower positioned avatars. - And when you scroll back up to the hero section, the inverse happens where the lower positioned avatars "arrive back in place" from up above the hero canvas before the upper avatars come back into the scene and settle in place. - Overall I wanted this hero section to feel alive. The floating avatars, particle system with very subtle star movements, and the Caustics effect on the "Forever" text all sort of move at the pace of slow breathing - which is a great pace to create a sense of life and comfort in your scene. Conversion Results (so far) - When this new Framer site launched along with Calaxy v1.9 release on Base a couple weeks ago, we saw a surge in traffic, around 20k page views in the first few days. - Of those 20k hits, 11k visited the app install page ( - which is our main CTA - We saw around 10K new users in the first week after v1.9 launch Overall I'm very happy with the new site and early performance metrics. Lots of tweaking to do but its a good start. If you are a designer building in Framer - hit me with any questions on the above hero notes. Happy to share more specifics! 👾

Chadd Weston

15,697 просмотров • 9 месяцев назад

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.
0:12

Wonderland: Navigating 3D Scenes from a Single Image Contributions: • First, we introduce a representation for controllable 3D generation by leveraging the generative priors from camera-guided video diffusion models. Unlike image models, video diffusion models are trained on extensive video datasets. This enables them to capture comprehensive spatial relationships within scenes across multiple views and embed a form of "3D awareness" in their latent space, which allows us to maintain 3D consistency in novel view synthesis. • Second, to achieve controllable novel view generation, we empower video models with precise control over specified camera motions. We introduce a novel dual-branch conditioning mechanism that effectively incorporates desired diverse camera trajectories into the video diffusion model. This enables expansion of a single image into a multi-view consistent capture of a 3D scene with precise pose control. • Third, to achieve efficient 3D reconstruction, we directly transform video latents into 3DGS. We propose a novel latent-based large reconstruction model (LaLRM) that lifts video latents to 3D in a feed-forward manner. With this design, during inference, our model directly predicts 3DGS from a single input image, effectively aligning the generation and reconstruction tasks—and bridging image space and 3D space—through the video latent space. Compared with reconstructing scenes from images, the video latent space offers a 256× spatial-temporal reduction while retaining essential and consistent 3D structural details. Such a high degree of compression is crucial, as it allows the LaLRM to handle a wider range of 3D scenes within the reconstruction framework, with the same memory constraints.

MrNeRF

52,801 просмотров • 1 год назад