正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

New 🤗 transformers release includes a very powerful Multimodel Large Language Model (MLLM) by Microsoft called KOSMOS-2! 🤩 The highlight of KOSMOS-2 is grounding, the model is incredibly accurate! 🌎 Play with the demo here 👉 But how does this model work? Let's take a look! 👀🧶

merve

86,581 subscribers

143,816 次观看 • 2 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

11 条评论

merve 的头像

merve2 年前

Grounding helps machine learning models relate to real-world examples. Including grounding makes models more performant by means of accuracy and robustness during inference. It also helps reduce the so-called "hallucinations" in language models.

merve 的头像

merve2 年前

In Kosmos-2, model is grounded to perform following tasks and is evaluated on 👇 - multimodal grounding & phrase grounding, e.g. localizing the object through natural language query - multimodal referring, e.g. describing object characteristics & location - perception-language tasks - language understanding and generation

merve 的头像

merve2 年前

The dataset used for grounding, called GRiT is also available on Hugging Face Hub 👉 Thanks to transformers integration, you can use KOSMOS-2 with few lines of code 🤩 See below! 👇

merve 的头像

merve2 年前

also big kudos to @ydshieh for implementing this in transformers ✨

Rainmaker 的头像

Rainmaker2 年前

Can Machine Learning beat the market? Check out this post on my free Substack where I share code and commentary for an XGBoost model and a Random Forest model that both deliver powerful performances.

merve 的头像

merve2 年前

multimodal* 🥲

Luis C 的头像

Luis C2 年前

@Microsoft Wow, its very fast on an A40

iamrobotbear (bk) 的头像

iamrobotbear (bk)2 年前

@Microsoft License?

Vlad 的头像

Vlad2 年前

@ClementDelangue @Microsoft 🤔 I could use this to improve I'm using the Blip model and is not bad but this looks like it could give more accurate results.

SkalskiP 的头像

SkalskiP2 年前

@Microsoft Yup KOSMOS-2 is awesome!

Risichad 🦾 的头像

Risichad 🦾2 年前

@Microsoft AI can understand a video at 3fps then !!!

相关视频

I'm stoked to be showing my challenge students how to use AI with ROS 2! In this demo the robot detects the coke can using a Large Language Model called CLIPSeg, which uses a text prompt "A can of soda" to segment the image .

I'm stoked to be showing my challenge students how to use AI with ROS 2! In this demo the robot detects the coke can using a Large Language Model called CLIPSeg, which uses a text prompt "A can of soda" to segment the image .

Dr. John Vial

12,274 次观看 • 1 年前

Introducing WildDet3D, a grounding model for monocular 3D object detection in the wild. A question I keep coming back to is: what is the right backbone for robotics foundation models? Should it be a video model, a language model, or perhaps a grounding model? WildDet3D is our first step in exploring that direction.

Introducing WildDet3D, a grounding model for monocular 3D object detection in the wild. A question I keep coming back to is: what is the right backbone for robotics foundation models? Should it be a video model, a language model, or perhaps a grounding model? WildDet3D is our first step in exploring that direction.

Jiafei Duan

12,072 次观看 • 2 个月前

WhisperKit x OpenAI Whisper Large v3 Turbo Released a few hours ago, this model is 8x faster than the original Large v3 model while retaining its high-quality multilingual capabilities. Here is a demo using our test app. Test app: GitHub (MIT): OpenAI release:

WhisperKit x OpenAI Whisper Large v3 Turbo Released a few hours ago, this model is 8x faster than the original Large v3 model while retaining its high-quality multilingual capabilities. Here is a demo using our test app. Test app: GitHub (MIT): OpenAI release:

argmax

47,779 次观看 • 1 年前

$When dividing by a fraction, a lot of people think about the measurement model, but we can think about the partitive model too! Here's why I think the partitive model is powerful.$

When dividing by a fraction, a lot of people think about the measurement model, but we can think about the partitive model too! Here's why I think the partitive model is powerful.

Howie Hua

10,789 次观看 • 8 个月前

Two videos of Russian interceptor drones with AI-targeting The first is the 'Bolt' UAV, a new model with ~15km range The second is the Yolka, a very common model with 2-3km range. It has no explosives, only kinetic forces

Two videos of Russian interceptor drones with AI-targeting The first is the 'Bolt' UAV, a new model with ~15km range The second is the Yolka, a very common model with 2-3km range. It has no explosives, only kinetic forces

EventsInUkraine

10,509 次观看 • 21 天前

A glimpse at the new EGLD economics model. So how does the model work? How does it capture value? How is it better than what we had before? One word. Growth. 🔥🛠️

A glimpse at the new EGLD economics model. So how does the model work? How does it capture value? How is it better than what we had before? One word. Growth. 🔥🛠️

Beniamin Mincu |🇺🇸/acc

21,154 次观看 • 8 个月前

I've seen so many people criticize the stock sonic model and say the model needs changing. But no, the model is fine. the problem is with how it's always posed. here for example with the unleashed model you can simply deform the brows and the muzzle to curve. This is pretty much the sonic unleashed game model but with a good rig that makes anything possible with the face. (Rig by DANCADA³ᴰ)

I've seen so many people criticize the stock sonic model and say the model needs changing. But no, the model is fine. the problem is with how it's always posed. here for example with the unleashed model you can simply deform the brows and the muzzle to curve. This is pretty much the sonic unleashed game model but with a good rig that makes anything possible with the face. (Rig by DANCADA³ᴰ)

TBSF

192,128 次观看 • 1 年前

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

Haider.

552,316 次观看 • 2 年前

See how Heavy Pulp made a trailer worthy of the big screen with this powerful new model:

See how Heavy Pulp made a trailer worthy of the big screen with this powerful new model:

xAI

8,188,261 次观看 • 7 天前

Language is the future for how we interact with robots. Today Wayve is sharing a first look at LINGO-1, a new vision-language-action AI model. To give you a glimpse of its capabilities, here is a video of me playing with LINGO-1 yesterday morning.

Language is the future for how we interact with robots. Today Wayve is sharing a first look at LINGO-1, a new vision-language-action AI model. To give you a glimpse of its capabilities, here is a video of me playing with LINGO-1 yesterday morning.

Alex Kendall

154,734 次观看 • 2 年前

Karp has been saying it for years: The Large Language Model is a commodity

Karp has been saying it for years: The Large Language Model is a commodity

Jack Prescott

37,483 次观看 • 13 天前

Over the past few years, we have taken a gigantic leap forward in our decades-long quest to build intelligent machines: the advent of the large language model or LLM. This is how they work

Over the past few years, we have taken a gigantic leap forward in our decades-long quest to build intelligent machines: the advent of the large language model or LLM. This is how they work

Financial Times

40,840 次观看 • 2 年前

tried making myself with the 2wintails customizable model 👀 how does the face tracking look?

tried making myself with the 2wintails customizable model 👀 how does the face tracking look?

❌2𝔴𝔦𝔫𝔱𝔞𝔦𝔩𝔰❌ Customisable Model OUT NOW

36,720 次观看 • 5 个月前

Apple built a large foundation model and fine-tuned it on multiple tasks. But they are doing something very clever: They load a single model in memory and use different adapters to specialize the model on the fly. I recorded a video to show you how to write the code to do the same thing Apple is doing. I explain everything step by step. Here is what I'll show you in the video: 1. We'll load two datasets 2. Then load a large model 3. Then, we'll fine-tune the model on both datasets I'll use LoRA to fine-tune the model. This process creates two small adapters, each specializing in solving one of the datasets. The base model's original parameters will remain unchanged. From here: 4. We'll generate a list of tasks 5. We'll load the correct adapter to solve each task The large model I'm using needs 346 MB of memory, but I only need to load it once. Each adapter is only 2.7 MB. I only need to load the base model once and pair it with any of the fine-tuned adapters. Minimum memory footprint and I can solve multiple tasks. Hope this helps!

Apple built a large foundation model and fine-tuned it on multiple tasks. But they are doing something very clever: They load a single model in memory and use different adapters to specialize the model on the fly. I recorded a video to show you how to write the code to do the same thing Apple is doing. I explain everything step by step. Here is what I'll show you in the video: 1. We'll load two datasets 2. Then load a large model 3. Then, we'll fine-tune the model on both datasets I'll use LoRA to fine-tune the model. This process creates two small adapters, each specializing in solving one of the datasets. The base model's original parameters will remain unchanged. From here: 4. We'll generate a list of tasks 5. We'll load the correct adapter to solve each task The large model I'm using needs 346 MB of memory, but I only need to load it once. Each adapter is only 2.7 MB. I only need to load the base model once and pair it with any of the fine-tuned adapters. Minimum memory footprint and I can solve multiple tasks. Hope this helps!

Santiago

84,747 次观看 • 1 年前

Nick Fuentes says SNEAKO is literally a Large Language Model 😂 "He functions like a prediction model, he doesn't have any real beliefs; he detects which way the wind is blowing, and he DESPERATELY tries to say the 'hot take' of the day."

Nick Fuentes says SNEAKO is literally a Large Language Model 😂 "He functions like a prediction model, he doesn't have any real beliefs; he detects which way the wind is blowing, and he DESPERATELY tries to say the 'hot take' of the day."

Fuentes Updates

24,432 次观看 • 27 天前

I've had early access to Kling AI brand new 2.0 video model and done extensive testing and trust me when I say that this model upgrade is AMAZING!!! 🔥 The amount of incredible dynamic action that you can generate now is next level. If you want to make things go fast, the new model completely excels at this, movement looks very natural and fluid. This is a major release for Kling! Here is a collection of some clips I've generated with the 2.0 model.

Travis Davids

87,789 次观看 • 1 年前

$TSLA Take a look at the Welcome Signal of the new Model Y!

$TSLA Take a look at the Welcome Signal of the new Model Y!

Tsla Chan

88,758 次观看 • 1 年前

I went with the Wildcat next. I never get tired of opening a new model! Very excited for this one - I do love a Wildcat. It’s amazing how therapeutic and relaxing model making is.

I went with the Wildcat next. I never get tired of opening a new model! Very excited for this one - I do love a Wildcat. It’s amazing how therapeutic and relaxing model making is.

Dr Sarah-Louise Miller 🇺🇦

20,804 次观看 • 1 年前

Since the first edition of the HBL PSL, the tournament has followed a Player Draft Model. As the league transitions to a Player Auction Model, let's look at how the Player Auction will work 👇 #HBLPSL | #NewEra

Since the first edition of the HBL PSL, the tournament has followed a Player Draft Model. As the league transitions to a Player Auction Model, let's look at how the Player Auction will work 👇 #HBLPSL | #NewEra

PakistanSuperLeague

52,731 次观看 • 4 个月前