Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX...

47,291 görüntüleme • 1 yıl önce •via X (Twitter)

9 Yorum

Bilawal Sidhu profil fotoğrafı
Bilawal Sidhu1 yıl önce

Project/paper here: If y'all enjoyed this post, you might also enjoy my writing on all things spatial ai and creative tech:

Min Choi profil fotoğrafı
Min Choi1 yıl önce

This is genius. I'm playing with SAM 2 and this was the exact limitation that I was running into for my use

Bilawal Sidhu profil fotoğrafı
Bilawal Sidhu1 yıl önce

Same! @BlacePlugins ya'll need to implement this paper for your after effects SAM 2 plugin

Kishen Patel profil fotoğrafı
Kishen Patel1 yıl önce

very cool! there’s definitely something satisfying when intuitive ways of solving ML problems work. like kalman filter to track noisy movement just makes sense - basically a signal processing problem. another might be SSMs for modeling a differential system.

Bilawal Sidhu profil fotoğrafı
Bilawal Sidhu1 yıl önce

Totally — sometimes the best solutions come from recognizing when an ML problem is actually a classical problem in disguise!

subnet Sam profil fotoğrafı
subnet Sam1 yıl önce

wow this is incredibly cool! i woudl use this for non nefarious purposes

Bilawal Sidhu profil fotoğrafı
Bilawal Sidhu1 yıl önce

Lol. VFX and surveillance about to get a nice boost 😂

Maskai profil fotoğrafı
Maskai1 yıl önce

Very interesting 🧐

Wit Of Gold profil fotoğrafı
Wit Of Gold1 yıl önce

Impressive

Benzer Videolar

Chamath: Two terms you need to pay attention to in AI are Prefill and Decode “There's two terms that I think you're going to hear a ton about over these next few years.” “The first term is prefill, and the next is decode.” “What prefill and decode are, are two very distinct ways of how models think, and how a model goes through the process of answering a question that you ask it.” “And so when you send a prompt to AI, what happens is that the model processes it. This is called the reading phase or prefill.” “It reads your entire prompt all at once. And then it does a bunch of math, calculates all these relationships between all the words, and it stores them in temporary memory.” “The problem is that this is really compute bound. So it requires massive brute force. And Nvidia GPUs crush here.” “And their architecture is designed for massive parallel processing, which makes them really amazing at digesting these long prompts.” “So the problem just gets bigger and bigger, Nvidia just completely dominates.” “But the next phase though, this critical phase, the decode phase, is the writing phase, right?” “So the model starts to generate a response, you ask it a question and its response, one token at a time.” “And then to pick the next token to pick the next word, it has to look back at everything it has said already so that it doesn't hallucinate.” “The problem is that this is incredibly memory bandwidth constrained.” “And in our architecture, a long time ago, we made these design decisions from day one.” “And so what we did was we took a very different architectural approach, we took a very conservative process technology. We weren't pushing the boundaries of physics.” “And we used a lot of what's called SRAM. So memory on the chip so that we could do this decode thing as well or better than everybody else.” “And so now when you put these two things together, I just think it's going to create a huge acceleration in the ability for this entire infrastructure layer to get much cheaper and much more valuable, which I suspect then it'll have a lot more developer pull, you'll get a lot more applications being built, billions and billions of more people using it.”

The All-In Podcast

567,660 görüntüleme • 9 ay önce

From Eric Vishria on how the top AI founders are building products completely opposite of the SaaS era: "One of the things that is really different in the AI world versus the SaaS world, is that in the SaaS world, over and over again, you had people who really understood the customer. And the problem. And then they understood a domain. They understood what the technology was more or less capable of. But it wasn't a real question of if you could build something or not. For example, take Salesforce, Workday, and ServiceNow. CRM existed before Salesforce. HR management existed before Workday. Same thing with ServiceNow. So in every case, Salesforce followed Siebel. Workday followed Peoplesoft. ServiceNow followed Peregrine and Remedy, and others. So they were just kind of, cloud SaaS versions of the prior generation product. They just understood the customers. They understood the problem. And they were just like, here's a better version. And that evolved a little bit over time in SaaS land. But that's what it is. And so product development in that way was done by people who really understood the customer and the problems. And then just took advantage of the next wave. And this is almost diametrically opposite of product development in the AI era. When I look at the teams that are having the most success today, they have intimate knowledge of the models. They are right on the frontier of understanding which models are better at what, and why, and when. And what they're going to be good at and what they're not going to be good at. And what they're spending their time on, is figuring out how do I apply this capability of this model to this domain or to this user. So they're actually working inside out or technology out, versus customer problem in. And of course, they understand the customer problem. And a lot of times they have firsthand knowledge of it. But they're really close to the metal and capability, and they're applying it. And I think this is a really different way to develop products than in SaaS. I started my career as a product manager a long time ago, and it's almost the complete opposite of everything you learned. "Listen to the customer, understand it, then bring it back to the engineering and product teams." If you did that right now, ask a bunch of customers what they want out of AI, and you brought it back, for the most part, it may not be possible today with today's technology. Whereas the teams that are winning right now really understand the technology and are applying it out. And so I think this reversal matters. I think it's a big difference in terms of how companies are getting built. And maybe even the types of entrepreneurs that will be successful. I'm not sure. You're seeing some real change there. Look at the Bret Taylor's at Sierra. That's a super, super technical founder who really gets it. Brett and Clay really get it. You look at Michael and his co-founders at Cursor. They're super technical founders and they get it. They all really understand what these things can and can't do. And that's a pretty different dynamic relative to the way the best SaaS companies got built." Link in bio for the full conversation going deep on the current class of startups going from zero to $100m+ in ARR within 12 months.

The Peel

209,752 görüntüleme • 1 yıl önce