Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Splines instead of Gaussians 😉 Introducing Neural Spline Fields, which can see through occlusions! We learn to represent a stack of misaligned captures as a multi-layer image sandwich. Then you can extract your favorite layer to remove occlusions, reflections, or even your own shadows from the scene! Paper and... show more

Felix Heide

1,988 subscribers

137,323 просмотров • 2 лет назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля Yun-Ta Tsai

Yun-Ta Tsai2 лет назад

Great work!

Фото профиля ELLIE X

ELLIE X2 лет назад

i feel like this feature would do numbers in dingboard @yacineMTB

Фото профиля Mark Kretschmann

Mark Kretschmann2 лет назад

this is nuts :)

Фото профиля Alex Nichol

Alex Nichol2 лет назад

I've always wanted to do something like this to see through grates! Super neat

Фото профиля Jonathan Craig

Jonathan Craig2 лет назад

Is this purely for 2-dimensional images, or is there a technique like this for higher dimensional spaces where some of the geometric information is obstructed from being "witnessable" directly by lower dimensional obstructions?

Фото профиля Ilya Sutskever's hairline

Ilya Sutskever's hairline2 лет назад

Technically speaking, everything is obstruction

Фото профиля PHBD

PHBD2 лет назад

The reflection reconstruction from burst is so cool, even more so than high quality occlusion removal imo.

Фото профиля Jb007

Jb0072 лет назад

Does this work with clothing? lol 😂

Фото профиля eren

eren2 лет назад

never seen something like this before. this is amazing

Фото профиля Ian Timotheos Pilon

Ian Timotheos Pilon2 лет назад

Cc @Scobleizer

Похожие видео

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!

Code for Neural Spline Fields is out! We use burst image fusion to see through occlusions and remove reflections. We just released the code here: Comes with a fun set of notebooks and tutorials!

Felix Heide

19,117 просмотров • 2 лет назад

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene reconstruction to produce novel wide field-of-view projections of the environment. Code, data, and info: Amazing work with Ilya Chugunov, Amogh Joshi, Kiran Murthy, François Bleibel

Implicit Neural Light Spheres lets you turn panoramic captures into dynamic wide FOV renders (with real-time rendering!). Instead of generating panoramas with image stitching, we use neural light spheres to jointly estimate the camera path and a high-resolution scene reconstruction to produce novel wide field-of-view projections of the environment. Code, data, and info: Amazing work with Ilya Chugunov, Amogh Joshi, Kiran Murthy, François Bleibel

Felix Heide

10,301 просмотров • 1 год назад

The Gondry Exercise: start with a single image and see how far and wide you can take it creatively. The starting image can be anything you want, as boring or complex as you wish. Then use Gen-4 to prompt for any combination of actions or changes to the scene. See how far you can go, and you will get a good measure of how much you need to exercise your creative mind.

The Gondry Exercise: start with a single image and see how far and wide you can take it creatively. The starting image can be anything you want, as boring or complex as you wish. Then use Gen-4 to prompt for any combination of actions or changes to the scene. See how far you can go, and you will get a good measure of how much you need to exercise your creative mind.

Cristóbal Valenzuela

15,622 просмотров • 1 год назад

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Don't talk directly to a model. You can make your application 10x more flexible by simply adding one intermediate layer between your code and the model. This is the simplest trick you can use to improve the architecture of whatever you are building.

Santiago

88,884 просмотров • 4 месяцев назад

New short course: Collaborative Writing and Coding with OpenAI Canvas! Explore new ways to write and code with OpenAI Canvas, a user-friendly interface that allows you to brainstorm, draft, and refine text and code in collaboration with ChatGPT. In the short course, created with OpenAI, and taught by , a research lead at OpenAI, you’ll learn to use Canvas to enhance your workflows. Canvas lets you go beyond simple chat interactions. It provides a side-by-side workspace where you and ChatGPT can edit and refine text or code collaboratively. This makes brainstorming, drafting, and iterating as you write feel more natural and effective. As the first major update to ChatGPT’s visual interface since its launch in 2022, Canvas gives a new, innovative approach to collaboration with AI. For instance, after writing the first version of your code, Canvas can review it and give suggestions for improvement. It can also help with debugging by adding logging, identifying problems to fix, and writing comments. In addition, you'll also learn what it takes to train the model for an interface like Canvas. In this video-only short course, you’ll: - Learn how to ask for in-line feedback and control the iteration of your work by directly editing selected areas of your text or code from the model’s output. - Learn how to access quick automation tools in a shortcut menu that allows you to modify your writing tone and length, enhance your code, and restore previous versions of your work. - Learn how to use Canvas as a research assistant tool with an example of asking the model to reason through the screenshot of a plot to write a research report, in which you can ask questions within the created report. - Ask the model to write Python code to replicate the graph seen on a screenshot image. - Go behind the scenes of how you can create a video game, such as Space Battleship, from scratch, edit it, and display it in one self-contained HTML file. - Get a real-world application example of creating a SQL database from the image of its architecture. - Understand the model training and design processes that power Canvas! Please sign up here:

New short course: Collaborative Writing and Coding with OpenAI Canvas! Explore new ways to write and code with OpenAI Canvas, a user-friendly interface that allows you to brainstorm, draft, and refine text and code in collaboration with ChatGPT. In the short course, created with OpenAI, and taught by , a research lead at OpenAI, you’ll learn to use Canvas to enhance your workflows. Canvas lets you go beyond simple chat interactions. It provides a side-by-side workspace where you and ChatGPT can edit and refine text or code collaboratively. This makes brainstorming, drafting, and iterating as you write feel more natural and effective. As the first major update to ChatGPT’s visual interface since its launch in 2022, Canvas gives a new, innovative approach to collaboration with AI. For instance, after writing the first version of your code, Canvas can review it and give suggestions for improvement. It can also help with debugging by adding logging, identifying problems to fix, and writing comments. In addition, you'll also learn what it takes to train the model for an interface like Canvas. In this video-only short course, you’ll: - Learn how to ask for in-line feedback and control the iteration of your work by directly editing selected areas of your text or code from the model’s output. - Learn how to access quick automation tools in a shortcut menu that allows you to modify your writing tone and length, enhance your code, and restore previous versions of your work. - Learn how to use Canvas as a research assistant tool with an example of asking the model to reason through the screenshot of a plot to write a research report, in which you can ask questions within the created report. - Ask the model to write Python code to replicate the graph seen on a screenshot image. - Go behind the scenes of how you can create a video game, such as Space Battleship, from scratch, edit it, and display it in one self-contained HTML file. - Get a real-world application example of creating a SQL database from the image of its architecture. - Understand the model training and design processes that power Canvas! Please sign up here:

Andrew Ng

128,180 просмотров • 1 год назад

🟢Substance Painter Tip With "Anchor Points" you can link layers together (e.g. use one layer as a mask for another), which makes workflows much faster. Example: You have a shadow layer and want to add an outline around it. Instead of repainting the outline every time you change the shadows, use an Anchor Point: 1. Add an Anchor Point to the shadow layer 2. Add a Fill Filter for the shadow outline layer 3. For the Fill grayscale, select the Anchor Point 4. Add a "Mask Outline" filter 5. If you paint new shadows now, the outline will adjust automatically!

🟢Substance Painter Tip With "Anchor Points" you can link layers together (e.g. use one layer as a mask for another), which makes workflows much faster. Example: You have a shadow layer and want to add an outline around it. Instead of repainting the outline every time you change the shadows, use an Anchor Point: 1. Add an Anchor Point to the shadow layer 2. Add a Fill Filter for the shadow outline layer 3. For the Fill grayscale, select the Anchor Point 4. Add a "Mask Outline" filter 5. If you paint new shadows now, the outline will adjust automatically!

Keeki | 3D Artist

38,950 просмотров • 2 месяцев назад

"There's even more exciting technologies and coins out there. There's another one called Zano that allows anybody to create their own privacy tokens where nobody can see what's going on. Then there's a project called Confidential Layer ( Confidential Layer ), which is creating a bridge from $BTC, $BCH, and $ETH onto Zano, and when it's on Zano, you have very, very strong privacy with that as well." ~Roger Ver Learn more about Zano:

"There's even more exciting technologies and coins out there. There's another one called Zano that allows anybody to create their own privacy tokens where nobody can see what's going on. Then there's a project called Confidential Layer ( Confidential Layer ), which is creating a bridge from $BTC, $BCH, and $ETH onto Zano, and when it's on Zano, you have very, very strong privacy with that as well." ~Roger Ver Learn more about Zano:

Zano

405,971 просмотров • 1 год назад

“Why do you look at the speck of sawdust in your brother’s eye and pay no attention to the plank in your own eye? How can you say to your brother, ‘Let me take the speck out of your eye,’ when all the time there is a plank in your own eye? You hypocrite, first take the plank out of your own eye, and then you will see clearly to remove the speck from your brother’s eye.”

“Why do you look at the speck of sawdust in your brother’s eye and pay no attention to the plank in your own eye? How can you say to your brother, ‘Let me take the speck out of your eye,’ when all the time there is a plank in your own eye? You hypocrite, first take the plank out of your own eye, and then you will see clearly to remove the speck from your brother’s eye.”

Michael Heise

104,726 просмотров • 6 месяцев назад

Mentioned Dead As Disco was a banger. Did I mention you can upload tracks of your own choosing, set the BPM, and then vibe out to your favorite anime bangers?

Mentioned Dead As Disco was a banger. Did I mention you can upload tracks of your own choosing, set the BPM, and then vibe out to your favorite anime bangers?

FightinCowboy

27,080 просмотров • 3 месяцев назад

Millions of people are trying out new workflows pairing Canva and AI assistants. The latest unlock? Whether you're coming from Gemini, Claude, ChatGPT, or Microsoft Copilot, you can use Canva's Magic Layers to unlock every visual element as a separate, editable layer to remix, refine, and make your own.

Millions of people are trying out new workflows pairing Canva and AI assistants. The latest unlock? Whether you're coming from Gemini, Claude, ChatGPT, or Microsoft Copilot, you can use Canva's Magic Layers to unlock every visual element as a separate, editable layer to remix, refine, and make your own.

Canva

38,369 просмотров • 1 месяц назад

Image editors, video resizers, custom shaders… if you can think of it, you can now build it. Introducing Tools in Google Flow. Starting today, you can describe the tools you wish you had, and Google Flow will help build them for your workflows. Explore our Tools gallery for inspiration, including some from fellow creatives, share your own, or even remix existing Tools to fit your needs. Learn how to get started in the thread below #GoogleIO

Image editors, video resizers, custom shaders… if you can think of it, you can now build it. Introducing Tools in Google Flow. Starting today, you can describe the tools you wish you had, and Google Flow will help build them for your workflows. Explore our Tools gallery for inspiration, including some from fellow creatives, share your own, or even remix existing Tools to fit your needs. Learn how to get started in the thread below #GoogleIO

Google Flow

255,305 просмотров • 2 месяцев назад

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna 👩🏽‍💻✨, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

AI can write more code than any team can review by hand, and a pull request can look fine while hiding a security issue or missing a requirement. In our new short course, AI Code Review, built in collaboration with Qodo and taught by Nnenna 👩🏽‍💻✨, you'll learn the practices that make AI code review effective: review before you open a pull request, give the reviewer full context about your codebase, and triage findings by risk. Then you'll build your own review agent, from a context engine that finds the right code to a team of specialized reviewers. Enroll for free:

DeepLearning.AI

53,674 просмотров • 13 дней назад

After the last promotion, we knew we needed to do something exceptional to prove our love for Succinct. Introducing Prooforia a Succinct NFT marketplace made by @ChizaramCh (yeah, he did 80% of the work) and me. Here, you can list your art and digital assets, trade them, own them, and have a lot of fun. Try it We made this for you 👉 Succinct crashout zkDan

After the last promotion, we knew we needed to do something exceptional to prove our love for Succinct. Introducing Prooforia a Succinct NFT marketplace made by @ChizaramCh (yeah, he did 80% of the work) and me. Here, you can list your art and digital assets, trade them, own them, and have a lot of fun. Try it We made this for you 👉 Succinct crashout zkDan

Jason 🟡

17,187 просмотров • 1 год назад

Seeing the World through Your Eyes paper page: The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections. This task is challenging due to 1) the difficulty of accurately estimating eye poses and 2) the entangled appearance of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

Seeing the World through Your Eyes paper page: The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like. By imaging the eyes of a moving person, we can collect multiple views of a scene outside the camera's direct line of sight through the reflections in the eyes. In this paper, we reconstruct a 3D scene beyond the camera's line of sight using portrait images containing eye reflections. This task is challenging due to 1) the difficulty of accurately estimating eye poses and 2) the entangled appearance of the eye iris and the scene reflections. Our method jointly refines the cornea poses, the radiance field depicting the scene, and the observer's eye iris texture. We further propose a simple regularization prior on the iris texture pattern to improve reconstruction quality. Through various experiments on synthetic and real-world captures featuring people with varied eye colors, we demonstrate the feasibility of our approach to recover 3D scenes using eye reflections.

AK

470,318 просмотров • 3 лет назад

We built LobsterX 🦞, an OpenClaw🦞 specialized for document work on your computer. It uses high-accuracy document parsing, extraction, classification through LlamaCloud, meaning it can comb through complicated PDFs (with scans, tables, diagrams) and extract out 100% accurate context! It can run as a Telegram bot and is built on top of agentfs (Turso) as a file system. Big shoutout to Clelia Bertelli (🦙/acc). This is a fun project inspired by OpenClaw🦞’s success, and besides being a fun tool to use, it can be a great reference for building your own generalized coding agents! Readme: LlamaCloud:

We built LobsterX 🦞, an OpenClaw🦞 specialized for document work on your computer. It uses high-accuracy document parsing, extraction, classification through LlamaCloud, meaning it can comb through complicated PDFs (with scans, tables, diagrams) and extract out 100% accurate context! It can run as a Telegram bot and is built on top of agentfs (Turso) as a file system. Big shoutout to Clelia Bertelli (🦙/acc). This is a fun project inspired by OpenClaw🦞’s success, and besides being a fun tool to use, it can be a great reference for building your own generalized coding agents! Readme: LlamaCloud:

Jerry Liu

27,071 просмотров • 6 месяцев назад

Introducing powered by Over the Reality. FreeGaussian is a new way to transform your existing photos, videos, and 360 captures for free into high-quality 3D Gaussian splats, directly in your browser. Whether you already have: ↳ a 360 recording captured with any 360 camera ↳ a set of photos ↳ or even a simple video from a place you visited you can now turn it into a photorealistic 3D Gaussian splat. Just go to upload your content, and let the reconstruction happen. Once it’s ready, you can download your 3D Gaussian splat and point cloud, and use them across your own projects, pipelines, or applications.

Introducing powered by Over the Reality. FreeGaussian is a new way to transform your existing photos, videos, and 360 captures for free into high-quality 3D Gaussian splats, directly in your browser. Whether you already have: ↳ a 360 recording captured with any 360 camera ↳ a set of photos ↳ or even a simple video from a place you visited you can now turn it into a photorealistic 3D Gaussian splat. Just go to upload your content, and let the reconstruction happen. Once it’s ready, you can download your 3D Gaussian splat and point cloud, and use them across your own projects, pipelines, or applications.

Over the Reality 🌐

86,261 просмотров • 3 месяцев назад

You can travel to Italy, but you’ll never be an Italian. You can travel to France but you’ll never be a Frenchman. You can live in Germany but you’ll never be a German. You can pack your bags and live the rest of your life in China or Japan, but you’ll never be Chinese or Japanese. Yet you can come from any one of those countries to the United States of America, and you can still be an American – so long as you work hard, you play by the rules, you make your contributions, wait your turn, pledge allegiance to the flag, and obtain your citizenship.

You can travel to Italy, but you’ll never be an Italian. You can travel to France but you’ll never be a Frenchman. You can live in Germany but you’ll never be a German. You can pack your bags and live the rest of your life in China or Japan, but you’ll never be Chinese or Japanese. Yet you can come from any one of those countries to the United States of America, and you can still be an American – so long as you work hard, you play by the rules, you make your contributions, wait your turn, pledge allegiance to the flag, and obtain your citizenship.

Vivek Ramaswamy

6,114,488 просмотров • 3 месяцев назад

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Agentic Document Extraction now supports field extraction! Many doc extraction use cases extract specific fields from forms and other structured documents. You can now input a picture or PDF of an invoice, request the vendor name, item list, and prices, and get back the extracted fields. Or input a medical form and specify a schema to extract patient name, patient ID, insurance number, etc. One cool feature: If you don't feel like writing a schema (json specification of what fields to extract) yourself, upload one sample document and write a natural language prompt saying what you want, and we automatically generate a schema for you. See the video for details!

Andrew Ng

193,053 просмотров • 1 год назад

SqueezeMe: Efficient Gaussian Avatars for VR TL;DR: Three of these Gaussian Splatting avatars can be run at 72 frames per second. It runs locally on a Meta Quest 3 VR headset. Abstract (excerpt): While previous methods require a desktop GPU for real-time inference of a single avatar, we aim to squeeze multiple Gaussian avatars onto a portable virtual reality headset with real-time drivable inference. We begin by training a previous work, Animatable Gaussians, on a high-quality dataset captured with 512 cameras. The Gaussians are animated by controlling a base set of Gaussians with linear blend skinning (LBS) motion, and then further adjusting them with a neural network decoder to correct their appearance. When deploying the model on a Meta Quest 3 VR headset, we find two major computational bottlenecks: the decoder and the rendering. To accelerate the decoder, we train the Gaussians in UV-space instead of pixel-space and distill the decoder to a single neural network layer. Further, we discover that neighborhoods of Gaussians can share a single corrective from the decoder, providing an additional speedup. To accelerate the rendering, we develop a custom pipeline in Vulkan that runs on the mobile GPU. Putting it all together, we run 3 Gaussian avatars concurrently at 72 FPS on a VR headset.

MrNeRF

27,104 просмотров • 1 год назад