Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Depth-aware light injection in TypeGPU I got a 448x448 monocular depth model down to ~8 ms on my M4 Pro across ~250 dispatches, which is fast enough to use in realtime :D Since the inference is written directly in TypeGPU, I can just feed the depth buffer straight into... show more

Konrad Reczko

7,633 subscribers

10,104,231 Aufrufe • vor 1 Monat •via X (Twitter)

Wissenschaft & Technologie Bildung

Anya Rossi• Live Now

Private livecam show

50 Kommentare

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Here are some fun intermediate textures, the depth and constructed normals

Profilbild von Fernando

Fernandovor 1 Monat

Really nice. But you shouldn't cast a shadow when the light source is between you and the wall

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

True, but it looks cool :D Doubles as fake godrays

Profilbild von Marc Krenn

Marc Krennvor 1 Monat

dope! what depth model are you using?

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Repacked DepthArt

Profilbild von Marc Krenn

Marc Krennvor 1 Monat

thanks, man! :)

Profilbild von dax

daxvor 1 Monat

cool

Profilbild von Rehan Ali Shah

Rehan Ali Shahvor 1 Monat

Can you share the link of the code you are using for this video. I want to try it.

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Sure! I’m building it as a TypeGPU example so it will be fully open and runnable just like my selfie segmentation inference

Profilbild von vogel

vogelvor 1 Monat

woah this is insane wtf holyshit

Profilbild von グスタボ 🇧🇷

グスタボ 🇧🇷vor 1 Monat

can this add dynamic backgrounds in webcam?

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Yeah, you get raw depth and pixel color you can do whatever you want with it

Profilbild von Mustafa.

Mustafa.vor 1 Monat

this is sickk

Profilbild von Rubén

Rubénvor 1 Monat

You invented Navi! :)

Profilbild von Marc

Marcvor 1 Monat

Does this use the TrueDepth camera? VisionCamera?

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

This on web but I will port it to Vision Camera on RN too

Profilbild von Marc

Marcvor 1 Monat

Nice! let me know how it goes, VisionCamera gives you depth frames via the CameraDepthOutput:

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

I actually wanted to run my custom network and just use the external texture interface via react-native-webgpu. Worked nicely for me last time :D

Profilbild von Marc

Marcvor 1 Monat

wdym exactly? no depth streaming? and which external texture interface do you mean, NativeBuffer?

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Yup, exactly, I just take the raw camera info and run the inference via TypeGPU compute shaders

Profilbild von Marc

Marcvor 1 Monat

ah yes that is the perfect approach for stuff like this. if you build this with VisionCamera can you show me the code? :)

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Of course, I love the lib btw, I can run code on the camera frames in a way that never reaches JS outside of native handles, brilliant

Profilbild von Sam Sheffer

Sam Sheffervor 1 Monat

whoa

Profilbild von Indra55

Indra55vor 1 Monat

Dayummm!! Where's the code

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Soon, I have to clean it up and handle licensing the weights

Profilbild von Indra55

Indra55vor 1 Monat

Waiting!!

Profilbild von ahmet

ahmetvor 1 Monat

wtf have you built?!?!?!

Profilbild von Gyan Jara Hatke

Gyan Jara Hatkevor 1 Monat

😳😍👌

Profilbild von Kacper Kapuściak

Kacper Kapuściakvor 1 Monat

what kind of sorcery is this

Profilbild von Ackerman

Ackermanvor 1 Monat

yoo this is so cool, is it opensource?

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Soon (it will be joining the family of TypeGPU examples I will post an update when it’s ready

Profilbild von John Kim

John Kimvor 1 Monat

very cool. I can see it being very useful for live streaming.

Profilbild von Google @ Home

Google @ Homevor 1 Monat

@grok_owner if you could plz implement this in your livestream to have a fairy fly around you the chat would appreciate it

Profilbild von Michał Wierzbiński

Michał Wierzbińskivor 1 Monat

Brought some stable fluids to the party. Can confirm: TypeGPU is, ineed, fire 🔥🔥

Profilbild von gfodor.id

gfodor.idvor 1 Monat

Wow, definitely gonna check this model out

Profilbild von Atom the UiUx Designer

Atom the UiUx Designervor 1 Monat

Sorcery

Profilbild von Artur Chakhvadze

Artur Chakhvadzevor 1 Monat

This is so sick!!!

Profilbild von Joseph Azar

Joseph Azarvor 1 Monat

that's smooth 🤘

Profilbild von taf 🝮

taf 🝮vor 1 Monat

@sivori Amazing!

Profilbild von am.will

am.willvor 1 Monat

the lighting physics are the most impressive part. very cool.

Profilbild von Dave Katague

Dave Kataguevor 1 Monat

dude you are fucking brilliant

Profilbild von adyingdeath

adyingdeathvor 1 Monat

This project is super cool. Like a AR game haha.

Profilbild von Aman Gupta

Aman Guptavor 1 Monat

Can we use lidar sensor on iphone for accurate depth ??

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

That would be amazing, on web I doubt it but on RN maybe

Profilbild von Duncan

Duncanvor 1 Monat

wizardry

Profilbild von PDF

PDFvor 1 Monat

The shadow effect annoys me, when the hand is between the camera and the light source the light would be directly hitting the wall there should not be a shadow casted from the hand

Profilbild von Konrad Reczko

Konrad Reczkovor 1 Monat

Yeah I get that a lot, good news is the shadow is togglable and I already took the feedback and tweaked the shadows so they are less offensive while still getting the “godray” effect which I like

Profilbild von PDF

PDFvor 1 Monat

Niceee, all the best!

Profilbild von Fred Wanders

Fred Wandersvor 1 Monat

I ve literally been working on something like this for web design. Your implementation is brilliant

Profilbild von Shweta Bharadwaj

Shweta Bharadwajvor 1 Monat

brilliant stuff, konrad!

Ähnliche Videos

You can do some pretty fun realtime VFX once the depth output is just another (Type)GPU buffer :D I basically yoinked my old 3D fire sim (that is actually a fluid sim but do not tell anyone) and plugged it into the depth pipeline together with hand tracking Everything here is built from scratch in TypeGPU, so inference, simulation and rendering can share GPU resources and interleave passes directly. That’s 2 NN inferences, a full 3D fluid sim, relighting and rendering in one command encoder, still under 16 ms with no CPU sync in between

You can do some pretty fun realtime VFX once the depth output is just another (Type)GPU buffer :D I basically yoinked my old 3D fire sim (that is actually a fluid sim but do not tell anyone) and plugged it into the depth pipeline together with hand tracking Everything here is built from scratch in TypeGPU, so inference, simulation and rendering can share GPU resources and interleave passes directly. That’s 2 NN inferences, a full 3D fluid sim, relighting and rendering in one command encoder, still under 16 ms with no CPU sync in between

Konrad Reczko

27,182 Aufrufe • vor 1 Monat

TypeGPU + ruNNtime + Jev TypeSafe AI is a very fun combo :D ruNNtime gives me efficient local inference, TypeGPU lets inference and rendering share GPU resources directly with zero copy. That’s 3 separate NN inferences plus rendering, all happening in realtime Since we control the pipeline, Jev can just sit in the middle and add the semantic bit. camera + mic → Moonshine + YOLO26 + DepthART → Jev → lights, shadows and bloom

TypeGPU + ruNNtime + Jev TypeSafe AI is a very fun combo :D ruNNtime gives me efficient local inference, TypeGPU lets inference and rendering share GPU resources directly with zero copy. That’s 3 separate NN inferences plus rendering, all happening in realtime Since we control the pipeline, Jev can just sit in the middle and add the semantic bit. camera + mic → Moonshine + YOLO26 + DepthART → Jev → lights, shadows and bloom

Konrad Reczko

90,251 Aufrufe • vor 18 Tagen

Anybody remember this game? Just got the game and going to break it down & go in-depth like you’ve never seen before on my YouTube Page The game that got us to #1 in the country! 2007 MU vs ku game at Arrowhead.

Anybody remember this game? Just got the game and going to break it down & go in-depth like you’ve never seen before on my YouTube Page The game that got us to #1 in the country! 2007 MU vs ku game at Arrowhead.

Chase Daniel

228,539 Aufrufe • vor 2 Jahren

Thank you all for the kind words under the demo I posted :D The demo is now officially public and open source for you to play with, modify and build on top of. Feel free to tag me if you make something cool with it :) Also thank you Iwo Plaza | TypeGPU for participating in my silly video haha You can run it like any other TypeGPU example here: The source code lives in the TypeGPU monorepo: The repacked weights used by the example live here:

Thank you all for the kind words under the demo I posted :D The demo is now officially public and open source for you to play with, modify and build on top of. Feel free to tag me if you make something cool with it :) Also thank you Iwo Plaza | TypeGPU for participating in my silly video haha You can run it like any other TypeGPU example here: The source code lives in the TypeGPU monorepo: The repacked weights used by the example live here:

Konrad Reczko

336,571 Aufrufe • vor 1 Monat

My biggest annoyances with VS Code (and in turn Cursor) is I've never been able to get Find + Multi-Cursor with arrow keys working like I did in Sublime Text I use it a lot to edit lists of text fast I go Cmd+Find "SGD" for example, then Find All, then I literally have 10 cursors on every SGD, I can arrow key through the letters around it, or use Option/Alt-Left or Right to skip through words, or Command-Left or Right to to the beginning or the end of the line Then I can use Shift to select I'm super fast at editing like that but none of the shortcuts work in VS Code or Cursor similarly The internet says use Cmd+Shift+L to do it, but all that happens in Cursor is I end up in the AI chat and I don't wanna go there

My biggest annoyances with VS Code (and in turn Cursor) is I've never been able to get Find + Multi-Cursor with arrow keys working like I did in Sublime Text I use it a lot to edit lists of text fast I go Cmd+Find "SGD" for example, then Find All, then I literally have 10 cursors on every SGD, I can arrow key through the letters around it, or use Option/Alt-Left or Right to skip through words, or Command-Left or Right to to the beginning or the end of the line Then I can use Shift to select I'm super fast at editing like that but none of the shortcuts work in VS Code or Cursor similarly The internet says use Cmd+Shift+L to do it, but all that happens in Cursor is I end up in the AI chat and I don't wanna go there

@levelsio

177,684 Aufrufe • vor 2 Jahren

"This is absolutely stacked... I think this is a good enough squad to go deep in lots of competitions." Phil Kitromilides breaks down the incredible squad depth at Barcelona 🔁

"This is absolutely stacked... I think this is a good enough squad to go deep in lots of competitions." Phil Kitromilides breaks down the incredible squad depth at Barcelona 🔁

CBS Sports Golazo ⚽️

17,577 Aufrufe • vor 18 Tagen

After playing empulse i think its a little disappointing how accessible the movement is. It lacks depth and skill. Any 1 can be moving fast without much effort. Games like titanfall or quake have incredible depth in its movement to the point where you can spend hours (1/3)

After playing empulse i think its a little disappointing how accessible the movement is. It lacks depth and skill. Any 1 can be moving fast without much effort. Games like titanfall or quake have incredible depth in its movement to the point where you can spend hours (1/3)

GetQuakedOn

60,827 Aufrufe • vor 3 Monaten

I just got this underwater scooter. It is designed for scuba diving, rated to 130 ft depth and 1 hour of continuous use. Also fun to use in the pool. Next step is to get my son scuba certified. Then after that spear fishing.

I just got this underwater scooter. It is designed for scuba diving, rated to 130 ft depth and 1 hour of continuous use. Also fun to use in the pool. Next step is to get my son scuba certified. Then after that spear fishing.

Wall Street Mav

174,759 Aufrufe • vor 1 Monat

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

This is insane!🔥 This is a 720p (2D) video I shot of my kids 15 years ago! I used AI to upscale it, depth-map it, & used the depth map to convert it to 3D SBS, & then converted that to Spatial Video for Apple Vision Pro (MV-HEVC). My son is currently a senior in high school, and my daughter is a sophomore in college. This video has complete spatial depth when viewed in the Apple Vision Pro. Obviously, you can't appreciate that via this post. But it's like I'm there & I can see into this room. Talk about giving you the feels.🥹 This can be done with ANY video. 🤯 I'm working on refining the workflow, as it's a bit cumbersome. I plan to share it, but holy cow! I had to post this.

Blaine Brown

1,468,930 Aufrufe • vor 2 Jahren

Arteta speaking after the draw to Wolves: “I think we need to go through the pain and you need to go through that looking in the mirror and understanding what the game requires now and the next action is on Sunday. We need to keep all that in our tummies to show it on Sunday.”

Arteta speaking after the draw to Wolves: “I think we need to go through the pain and you need to go through that looking in the mirror and understanding what the game requires now and the next action is on Sunday. We need to keep all that in our tummies to show it on Sunday.”

Connor Humm

60,050 Aufrufe • vor 7 Monaten

Was farming first encounter of ghosts to get the helmet I was missing (yeah I know it sucked) and I noticed that there are actually A LOT of details and stuff I just never bothered to notice in these play spaces. For example. Did you you know you can actually SEE Ahsa in this area before dropping down into the methane ocean for the first time? It honestly wants me to go back through all the dungeons in the game and take proper time to look through all the stuff in the world. Is that something you guys would be interested in?

Was farming first encounter of ghosts to get the helmet I was missing (yeah I know it sucked) and I noticed that there are actually A LOT of details and stuff I just never bothered to notice in these play spaces. For example. Did you you know you can actually SEE Ahsa in this area before dropping down into the methane ocean for the first time? It honestly wants me to go back through all the dungeons in the game and take proper time to look through all the stuff in the world. Is that something you guys would be interested in?

NAPainter_ GCX 2026 ✈️

154,528 Aufrufe • vor 4 Monaten

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

PowerInfer - a high-speed inference engine for deploying LLMs locally. Just came across this super interesting project on speeding up inference. It's not MoE but it's a simple approach that exploits the high locality in LLM inference to design a GPU-CPU hybrid inference engine. Hot-activated neurons are preloaded onto the GPU for fast access, while cold-activated neurons (the majority) are computed on the CPU. This approach significantly reduces GPU memory demands and CPU-GPU data transfer. It achieves an average token generation rate of 13.20 tokens/s, with a peak of 29.08 tokens/s, across various LLMs on a single NVIDIA RTX 4090 GPU. It's on only 18% lower than that achieved by a top-tier server-grade A100 GPU. It also significantly outperforms llama.cpp by up to 11.69x while retaining model accuracy. There is a lot more innovation around inference that's coming fast. Really encouraged by the study on sparse computation to enhance the computational efficiency of LLMs. It's now possible to use PowerInfer with Llama 2 and Faclon 40B. Mistral-7B support is coming soon!

elvis

261,756 Aufrufe • vor 2 Jahren

$I was wrong. I used to tell people that I would never go to a medical doctor unless it was for a broken bone, fracture or some form of structural damage. After having an experience like that I decided I really don't wanna go to a hospital or medical doctor because all they're gonna do is put my bone back in place and then put it in a cast for the next couple months and I'm gonna wonder later on why it never feels the same after locking my bone into the same position. It doesn't make sense to me. Animals in the wild break bones all of the time and fully recover, as long as they don't get eaten by a predator. If I ever have any structural damage again in the future or broken bones I'm gonna use the animal method: just fast, lay down in the sun and let nature heal me.$

I was wrong. I used to tell people that I would never go to a medical doctor unless it was for a broken bone, fracture or some form of structural damage. After having an experience like that I decided I really don't wanna go to a hospital or medical doctor because all they're gonna do is put my bone back in place and then put it in a cast for the next couple months and I'm gonna wonder later on why it never feels the same after locking my bone into the same position. It doesn't make sense to me. Animals in the wild break bones all of the time and fully recover, as long as they don't get eaten by a predator. If I ever have any structural damage again in the future or broken bones I'm gonna use the animal method: just fast, lay down in the sun and let nature heal me.

Riley Check

23,107 Aufrufe • vor 5 Monaten

.Compute Network just connected with Pod the Squire and turns staked $ZERO into useable OR sellable inference 49.39M $ZERO is currently staked (ATH) The dual mechanic: - Stake $ZERO, earn daily inference rights - Sell unused inference through - Every staked token has a yield path $15K in $ZERO has also been bought back or burned + $15K has been paid out to stakers

.Compute Network just connected with Pod the Squire and turns staked $ZERO into useable OR sellable inference 49.39M $ZERO is currently staked (ATH) The dual mechanic: - Stake $ZERO, earn daily inference rights - Sell unused inference through - Every staked token has a yield path $15K in $ZERO has also been bought back or burned + $15K has been paid out to stakers

MCG

18,129 Aufrufe • vor 3 Monaten

I’m bullish on $META over the next 6 to 12 months. Long term target: 820. Macro structure is still bullish and we’re sitting in a discount range as we head into earnings. In this video, I’ll break down the setup in depth and walk through why I finally took a position.

I’m bullish on $META over the next 6 to 12 months. Long term target: 820. Macro structure is still bullish and we’re sitting in a discount range as we head into earnings. In this video, I’ll break down the setup in depth and walk through why I finally took a position.

Peter DiCarlo

144,110 Aufrufe • vor 2 Monaten

🚨Oreshnik Nightmare is COMING 🚨Russia will take action, sooner rather than later. 🚨We are in a very dangerous situation. 🚨🚨🚨🚨🚨🚨🚨🚨🚨 Scott Ritter: If nuclear-armed states provide strike capacity to a non-nuclear state and use that to strike the strategic depth of Russia, Russia will have to use nuclear weapons preemptively against that threat. The Russian government put the Ukrainians on notice that if drones struck Moscow, Kiev would disappear. NATO missiles on Ukrainian soil striking into the Russian depth. The Flamingo, the FP5, is a British-made missile using German guidance equipment, brought to their target by American and NATO intelligence. Without this, Ukraine would not have this weapon. Ukraine's other weapons are using corridors that go through Poland into the Baltics so they can strike, ... in the vicinity of Saint Petersburg or through Kazakhstan so they can strike into the strategic depth.

🚨Oreshnik Nightmare is COMING 🚨Russia will take action, sooner rather than later. 🚨We are in a very dangerous situation. 🚨🚨🚨🚨🚨🚨🚨🚨🚨 Scott Ritter: If nuclear-armed states provide strike capacity to a non-nuclear state and use that to strike the strategic depth of Russia, Russia will have to use nuclear weapons preemptively against that threat. The Russian government put the Ukrainians on notice that if drones struck Moscow, Kiev would disappear. NATO missiles on Ukrainian soil striking into the Russian depth. The Flamingo, the FP5, is a British-made missile using German guidance equipment, brought to their target by American and NATO intelligence. Without this, Ukraine would not have this weapon. Ukraine's other weapons are using corridors that go through Poland into the Baltics so they can strike, ... in the vicinity of Saint Petersburg or through Kazakhstan so they can strike into the strategic depth.

Ignorance, the root and stem of all evil

37,607 Aufrufe • vor 4 Monaten

Jane Fonda: “This is the first time in the history of the US that an authoritarian regime has broken through. Worldwide historians who study these things all agree it has never happened so fast in any industrialized democracy. He is moving to consolidate very very fast which means right now we have to use the tools we have to stop it”

Jane Fonda: “This is the first time in the history of the US that an authoritarian regime has broken through. Worldwide historians who study these things all agree it has never happened so fast in any industrialized democracy. He is moving to consolidate very very fast which means right now we have to use the tools we have to stop it”

Marco Foster

831,592 Aufrufe • vor 6 Monaten

Ripgrep is slow. Nowgrep is fast, because I bypass the Windows slop and go straight to NTFS. Here's Ripgrep vs. Nowgrep searching through 300k files on a drive with 2M+. Nowgrep is written from scratch in C99. No borrow checker.

Ripgrep is slow. Nowgrep is fast, because I bypass the Windows slop and go straight to NTFS. Here's Ripgrep vs. Nowgrep searching through 300k files on a drive with 2M+. Nowgrep is written from scratch in C99. No borrow checker.

Charlie Malmqvist

305,757 Aufrufe • vor 1 Jahr