正在加载视频...

视频加载失败

返回首页

Zhilin at GTC: Introducing Attention Residuals Learning selective memory, rather than mechanically accumulating everything, is the beauty of attention. Many of you have probably read Attention Is All You Need, the 2017 Transformer paper that brought “human-like” attention into the model’s field of view. From that point on, models...

Kimi_Moonshot's profile picture

Kimi.ai

182,122 subscribers

115,162 次观看 • 3 个月前 •via X (Twitter)

科学技术
LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Steve Nash engaged in self-regulated learning early! His father, canny as a fox, pointed out what he did well and encouraged him to bottle the details. Remember this, remember that! Remember this, remember that! Memory - but it was never about remembering the performance outcomes, it was always performance process - the actions and decisions that helped the young Nash to score, not the scoring itself. All detail! All detail! This has been called attentional density (by neuroscientist David Schwartz). When you pay attention to something, your brain fires in patterns akin to that something. The more you think about it, the more the brain fires that way. The more the brain fires that way, the more chance you have of remembering what you’re paying attention to. Pay attention to the details! Pay attention to the details!
0:45

Steve Nash engaged in self-regulated learning early! His father, canny as a fox, pointed out what he did well and encouraged him to bottle the details. Remember this, remember that! Remember this, remember that! Memory - but it was never about remembering the performance outcomes, it was always performance process - the actions and decisions that helped the young Nash to score, not the scoring itself. All detail! All detail! This has been called attentional density (by neuroscientist David Schwartz). When you pay attention to something, your brain fires in patterns akin to that something. The more you think about it, the more the brain fires that way. The more the brain fires that way, the more chance you have of remembering what you’re paying attention to. Pay attention to the details! Pay attention to the details!

Daniel Abrahams

50,933 次观看 • 5 个月前

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:
2:41

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,220 次观看 • 1 年前

6⃣ Main Different Modes of Attention + explanations Can you notice when you are spontaneously in each of these types of attention throughout the week? Most think attention mode 2. is the only way to be concentrated = sustaining engagement with desired phenomena for long periods of time. But actually you can learn to maintain each mode of attention and concentrate in different ways - knowing this radically opens up one’s potential in meditation practice and access to many more states of mind and insights. 🤯 ¡This animation visually illustrates different ways in which attention can behave, but note that these different modes are relevant across the senses (not just with seeing)!
0:38

6⃣ Main Different Modes of Attention + explanations Can you notice when you are spontaneously in each of these types of attention throughout the week? Most think attention mode 2. is the only way to be concentrated = sustaining engagement with desired phenomena for long periods of time. But actually you can learn to maintain each mode of attention and concentrate in different ways - knowing this radically opens up one’s potential in meditation practice and access to many more states of mind and insights. 🤯 ¡This animation visually illustrates different ways in which attention can behave, but note that these different modes are relevant across the senses (not just with seeing)!

Roger This

12,742 次观看 • 1 年前

POSOBIEC: People won’t read a Supreme Court ruling, but they’ll watch a Netflix miniseries on it If you're not paying attention to who is in control of this, then you're not paying attention to who's in control of the minds of the masses
1:13

POSOBIEC: People won’t read a Supreme Court ruling, but they’ll watch a Netflix miniseries on it If you're not paying attention to who is in control of this, then you're not paying attention to who's in control of the minds of the masses

Jack Posobiec

310,123 次观看 • 7 个月前

Alex Pereira shares what has frustrated him most following his loss to Ciryl Gane: "Herb Dean came in, and I said, 'Listen, I understand that sometimes in a fight, eye pokes can happen and low blows can happen... but this guy has a history of shots to the back of the head, so please pay attention to those during the fight.' After we spoke with him, he brought Mike Beltran over and said, 'Just so you understand, the back of the head is here.' So it seemed like he absorbed the information. Then, the next day, seeing everything that happened and knowing that we brought it to his attention, he did not do his job."
3:24

Alex Pereira shares what has frustrated him most following his loss to Ciryl Gane: "Herb Dean came in, and I said, 'Listen, I understand that sometimes in a fight, eye pokes can happen and low blows can happen... but this guy has a history of shots to the back of the head, so please pay attention to those during the fight.' After we spoke with him, he brought Mike Beltran over and said, 'Just so you understand, the back of the head is here.' So it seemed like he absorbed the information. Then, the next day, seeing everything that happened and knowing that we brought it to his attention, he did not do his job."

Ariel Helwani

758,212 次观看 • 27 天前

I would pay close attention to this clip about USOs (Unidentified, Submerged Objects) from our recent episode of WEAPONIZED. You are going to be hearing a lot more about this soon. The information is right in front of you - if you read between the lines - and soon enough, you won’t need to read between the lines 😉🪼 WATCH : LISTEN : LEARN :
1:57

I would pay close attention to this clip about USOs (Unidentified, Submerged Objects) from our recent episode of WEAPONIZED. You are going to be hearing a lot more about this soon. The information is right in front of you - if you read between the lines - and soon enough, you won’t need to read between the lines 😉🪼 WATCH : LISTEN : LEARN :

Jeremy Kenyon Lockyer Corbell

132,000 次观看 • 1 年前

Raskin: This seems to me to be one more pathetic effort to distract and divert attention from the fact that the president's name appeared more than 1,000 times already in the small fraction of material released from Epstein and from the failure of the majority to move to disclose this information.
0:43

Raskin: This seems to me to be one more pathetic effort to distract and divert attention from the fact that the president's name appeared more than 1,000 times already in the small fraction of material released from Epstein and from the failure of the majority to move to disclose this information.

Acyn

26,735 次观看 • 7 个月前

🇺🇸 Chamath just revealed the truth on Rogan… attention is what runs it all. Google ranks the web by attention. Facebook and Instagram are built to capture attention. The most important AI paper ever written is literally called "Attention Is All You Need." Every system that now runs the world reduces to the same single currency. And at some point, that stops sounding like branding and starts sounding like a warning. Joe Rogan, Chamath Palihapitiya
1:56

🇺🇸 Chamath just revealed the truth on Rogan… attention is what runs it all. Google ranks the web by attention. Facebook and Instagram are built to capture attention. The most important AI paper ever written is literally called "Attention Is All You Need." Every system that now runs the world reduces to the same single currency. And at some point, that stops sounding like branding and starts sounding like a warning. Joe Rogan, Chamath Palihapitiya

Mario Nawfal

314,714 次观看 • 2 个月前

my favorite bit of attention to detail in this scene is that the Pilots and operator speak English, rather than Japanese Extra bit of realism that makes me appreciate it all the more
0:56

my favorite bit of attention to detail in this scene is that the Pilots and operator speak English, rather than Japanese Extra bit of realism that makes me appreciate it all the more

MA5K CEO

188,571 次观看 • 11 个月前

“I Started Losing My Sense of Self. I’ve actually never even talked about this. I’ve never told anybody this because it sounds probably stupid, but I really started paying attention to depth perception and trying to view the world in more of a three-dimensional experience. It’s also like I lose a sense of self - not completely, but maybe 75% of the way there.” Sadhguru
16:41

“I Started Losing My Sense of Self. I’ve actually never even talked about this. I’ve never told anybody this because it sounds probably stupid, but I really started paying attention to depth perception and trying to view the world in more of a three-dimensional experience. It’s also like I lose a sense of self - not completely, but maybe 75% of the way there.” Sadhguru

Shawn Ryan

289,166 次观看 • 22 天前

I have investigated the situation. EUW is doomed and we are only getting worse, give soloq 5% more of your attention and euw is saved, players arent even able to process basic information anymore
7:29

I have investigated the situation. EUW is doomed and we are only getting worse, give soloq 5% more of your attention and euw is saved, players arent even able to process basic information anymore

Veigarv2

199,338 次观看 • 2 年前

#Colts HC Shane Steichen on QB Anthony Richardson: “I’ve said ‘the process, the process’ for a long time, and what I mean by that is the attention to detail in everything he does. From the classroom to the walkthroughs to practice …” “He’s gonna start the rest of the season.”
1:28

#Colts HC Shane Steichen on QB Anthony Richardson: “I’ve said ‘the process, the process’ for a long time, and what I mean by that is the attention to detail in everything he does. From the classroom to the walkthroughs to practice …” “He’s gonna start the rest of the season.”

James Boyd

140,105 次观看 • 1 年前

Why is it so challenging for Vice President Kamala Harris to construct a straightforward sentence? READ: "I feel very strongly about the importance as a general matter of engaging in U.S. policy as it relates to foreign affairs in a way that we pay attention, of course, to the immediate concerns and threats if they exist, but that we also pay attention to 10, 20, 30 years down the line and what we are developing now that will be to the benefit of our country then."
0:31

Why is it so challenging for Vice President Kamala Harris to construct a straightforward sentence? READ: "I feel very strongly about the importance as a general matter of engaging in U.S. policy as it relates to foreign affairs in a way that we pay attention, of course, to the immediate concerns and threats if they exist, but that we also pay attention to 10, 20, 30 years down the line and what we are developing now that will be to the benefit of our country then."

Simon Ateba

1,757,741 次观看 • 2 年前

I hear so often from the Dommes I work with that they struggle with people online fetichizing them and simply seeing them for how sexy and beautiful they are. They project their fantasies and their desires onto you. That stops immediately once you move the attention from you to them. From 'look at me' to 'I see you'. What does that look like? When you create content, think of them and what this scene or that narrative is evoking. What will they learn from you? What they want is not to passively watch how sexy you are, but for you to train them, to give them instructions, to teach them, to guide them, to be in charge, to command them. This is not being an object but the main subject. The Authority figure. How is your content already doing that. The sexy photos can still be there, they are important to already capture des attention. But what you do with that attention once you have it, is where the power dynamic is established. Positioning yourself as more than a stunning Goddess, but actually a woman who has a voice, opinions, perspective, a philosophy, a way to doing things, teaching them what you like, how you like it, why you like it, already makes them want to be that for you. You hold the attention, you hold the power, so you direct it. And for that, you want them to know you get them and you know what lives within them... that creates the desire for you to be the one exposing it. You instantly build trust. Not because you demanded it, but because you earned it: you showed them you know what you are doing. You have experience, you understand them. They are not told to come see you, they are seduced into it. They desire it. And they will work for it. This will attract better clients (real subs) and instead of you trying to get their attention, they will work to earn yours. If you want to learn more about power dynamics, building a brand as a Pro or the psychology behind BDSM, you can now access all my trainings and classes in one place for a fraction of the cost of The Dominatrix Academy. And you can reinvest the total amount towards the Program. Message me [SECRET] for the details. This offer is not available on my website.
0:51

I hear so often from the Dommes I work with that they struggle with people online fetichizing them and simply seeing them for how sexy and beautiful they are. They project their fantasies and their desires onto you. That stops immediately once you move the attention from you to them. From 'look at me' to 'I see you'. What does that look like? When you create content, think of them and what this scene or that narrative is evoking. What will they learn from you? What they want is not to passively watch how sexy you are, but for you to train them, to give them instructions, to teach them, to guide them, to be in charge, to command them. This is not being an object but the main subject. The Authority figure. How is your content already doing that. The sexy photos can still be there, they are important to already capture des attention. But what you do with that attention once you have it, is where the power dynamic is established. Positioning yourself as more than a stunning Goddess, but actually a woman who has a voice, opinions, perspective, a philosophy, a way to doing things, teaching them what you like, how you like it, why you like it, already makes them want to be that for you. You hold the attention, you hold the power, so you direct it. And for that, you want them to know you get them and you know what lives within them... that creates the desire for you to be the one exposing it. You instantly build trust. Not because you demanded it, but because you earned it: you showed them you know what you are doing. You have experience, you understand them. They are not told to come see you, they are seduced into it. They desire it. And they will work for it. This will attract better clients (real subs) and instead of you trying to get their attention, they will work to earn yours. If you want to learn more about power dynamics, building a brand as a Pro or the psychology behind BDSM, you can now access all my trainings and classes in one place for a fraction of the cost of The Dominatrix Academy. And you can reinvest the total amount towards the Program. Message me [SECRET] for the details. This offer is not available on my website.

Ms. Malissia

15,105 次观看 • 2 个月前

Ansem on tokenized attention "What people miss is the tokenized attention piece. a lot of people say it's valueless and I think that's really untrue" "Last cycle Solana was trying to sell their first round of phones and weren't doing a great job. Then BONK airdropped tokens to people holding those phones" "When the tokens started going up, the airdrop was worth more than the phone cost. So people just started buying the phone and it sold out immediately" "Attention is very intangible, but tokens are one of the only ways you can ascribe value to attention in markets"
0:57

Ansem on tokenized attention "What people miss is the tokenized attention piece. a lot of people say it's valueless and I think that's really untrue" "Last cycle Solana was trying to sell their first round of phones and weren't doing a great job. Then BONK airdropped tokens to people holding those phones" "When the tokens started going up, the airdrop was worth more than the phone cost. So people just started buying the phone and it sold out immediately" "Attention is very intangible, but tokens are one of the only ways you can ascribe value to attention in markets"

Market Bubble

46,831 次观看 • 11 天前

This isn’t empowerment, it is attention monetized… the more you show, the more you get rewarded… and the more you rely on it, the less you build anything real. Y’all are being taught to use your body more and less of brain… brainwashed into believing that’s the power. Tragic!
2:59

This isn’t empowerment, it is attention monetized… the more you show, the more you get rewarded… and the more you rely on it, the less you build anything real. Y’all are being taught to use your body more and less of brain… brainwashed into believing that’s the power. Tragic!

Dr. God Abeg ooo

10,236 次观看 • 2 个月前

Elon Musk on learning physics: What matters are not the formulas, but the thinking process that led to the discovery of those formulas. “I just recommend studying physics. What matters is not remembering a bunch of formulas. What matters is the thinking process that led to the discovery of those formulas. It's sort of critical thinking, first principles analysis, trying to understand what is true at a most fundamental level and then reasoning up from there and testing your conclusions against the most fundamental truths in any given arena. This is how you can figure out whether something is more likely to be true or not. It's good to think in terms of probabilities. If you receive information about a subject, that should change the probability of your conclusion, but not the certainty of your conclusion. Physics teaches you that you should not be 100% certain about any given prediction. Now, there's some things that are highly likely, but physics teaches you you got to assign a probability to something being true. And then as you learn more information, your original conclusion may be wrong, then you should then change your mind based on the new information.” Town Hall in Lancaster, Pennsylvania, October 24, 2024
1:01

Elon Musk on learning physics: What matters are not the formulas, but the thinking process that led to the discovery of those formulas. “I just recommend studying physics. What matters is not remembering a bunch of formulas. What matters is the thinking process that led to the discovery of those formulas. It's sort of critical thinking, first principles analysis, trying to understand what is true at a most fundamental level and then reasoning up from there and testing your conclusions against the most fundamental truths in any given arena. This is how you can figure out whether something is more likely to be true or not. It's good to think in terms of probabilities. If you receive information about a subject, that should change the probability of your conclusion, but not the certainty of your conclusion. Physics teaches you that you should not be 100% certain about any given prediction. Now, there's some things that are highly likely, but physics teaches you you got to assign a probability to something being true. And then as you learn more information, your original conclusion may be wrong, then you should then change your mind based on the new information.” Town Hall in Lancaster, Pennsylvania, October 24, 2024

ELON CLIPS

67,501 次观看 • 5 个月前

Elon Musk exposes how the deception is going on with the media: “The biggest deception is the choice of narrative. The media can write a story about this or that, but only a few stories can go on the front page So you’re deciding what people should pay attention to. Instead of it being what people actually care about, it’s what a handful of editors care about. They’re telling the people what to pay attention to” This is exactly why Elon built 𝕏 the way he did…..so the truth and important stories bubble up organically from the people, not dictated by a small group of influential editors
0:39

Elon Musk exposes how the deception is going on with the media: “The biggest deception is the choice of narrative. The media can write a story about this or that, but only a few stories can go on the front page So you’re deciding what people should pay attention to. Instead of it being what people actually care about, it’s what a handful of editors care about. They’re telling the people what to pay attention to” This is exactly why Elon built 𝕏 the way he did…..so the truth and important stories bubble up organically from the people, not dictated by a small group of influential editors

X Freeze

13,721 次观看 • 4 个月前