正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Mamba by hand ✍️ ~ I made this exercise as I studied the math in the Mamba paper. It is a small yet complete Mamba implementation. Thanks Albert Gu for answering my questions and checking. 👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

65,751 subscribers

55,476 次观看 • 2 年前 •via X (Twitter)

教育科学技术

Anya Rossi• Live Now

Private livecam show

6 条评论

Tom Yeh 的头像

Tom Yeh2 年前

Download xlsx from Github:

Nothing is something 的头像

Nothing is something2 年前

@_albertgu This is some good stuff... Go for Jamba implementation...

Tom Yeh 的头像

Tom Yeh2 年前

@_albertgu I heard about it but haven't had time to read the paper yet.

Yigal Weinberger 的头像

Yigal Weinberger2 年前

@_albertgu Looks great! Always wanted to see the process in action

Tom Yeh 的头像

Tom Yeh2 年前

@_albertgu I am not familiar with this. How important is it?

chus 的头像

chus2 年前

@_albertgu This is very cool, thank you!

相关视频

Self-Attention by hand ✍️ Excel ~ I designed this exercise for students to practice the QKV math. I also created a medium and a large version to show how the attention matrix grows quadratically as the sequence gets longer. 👇Join the 'AI Math' community. Download xlsx.

Self-Attention by hand ✍️ Excel ~ I designed this exercise for students to practice the QKV math. I also created a medium and a large version to show how the attention matrix grows quadratically as the sequence gets longer. 👇Join the 'AI Math' community. Download xlsx.

Tom Yeh

125,723 次观看 • 2 年前

Transformer by hand ✍️ in Excel ~ I just released my first-ever "Full-Stack" implementation of the Transformer model. 👇Download xlsx to give it a try!

Transformer by hand ✍️ in Excel ~ I just released my first-ever "Full-Stack" implementation of the Transformer model. 👇Download xlsx to give it a try!

Tom Yeh

2,998,207 次观看 • 1 年前

I flipped through 300 pages of deep learning math puzzles by hand ✍️ This is the proof copy of the "Deep Learning Math Workbook" I made to help you learn and practice 12 foundational math concepts below👇

I flipped through 300 pages of deep learning math puzzles by hand ✍️ This is the proof copy of the "Deep Learning Math Workbook" I made to help you learn and practice 12 foundational math concepts below👇

Tom Yeh

2,378,415 次观看 • 1 年前

As someone who's built with modern LLM architectures, this Laplace transform lecture is the closest thing to a "why Mamba works" explainer I've ever seen released for free. Everyone thinks transformers are the only path to modern LLMs. State space models like Mamba and S4 challenge that using math Laplace built in the 1780s. This video shows why. 25 minutes. Bookmark & watch today.

As someone who's built with modern LLM architectures, this Laplace transform lecture is the closest thing to a "why Mamba works" explainer I've ever seen released for free. Everyone thinks transformers are the only path to modern LLMs. State space models like Mamba and S4 challenge that using math Laplace built in the 1780s. This video shows why. 25 minutes. Bookmark & watch today.

fintex

110,467 次观看 • 2 个月前

Can you follow the math of a Transformer model by hand ✍️? Download PDF 👉 I hope this hand-drawn animation makes it easier. p.s., This has one error. Can you can spot it and tell me in the comment?

Can you follow the math of a Transformer model by hand ✍️? Download PDF 👉 I hope this hand-drawn animation makes it easier. p.s., This has one error. Can you can spot it and tell me in the comment?

Tom Yeh

14,578 次观看 • 11 个月前

As someone who ships LLM systems in production, this Laplace transform video is the closest thing to a "why Mamba beats transformers" explainer I've ever seen released for free. Everyone thinks attention is what makes LLMs powerful. Laplace proved in 1785 that any signal breaks into exponential pieces - locate the poles, expose the frequencies. Mamba and Claude run on this exact trick. 34 minutes. Bookmark & watch today. The math is older than the United States.

As someone who ships LLM systems in production, this Laplace transform video is the closest thing to a "why Mamba beats transformers" explainer I've ever seen released for free. Everyone thinks attention is what makes LLMs powerful. Laplace proved in 1785 that any signal breaks into exponential pieces - locate the poles, expose the frequencies. Mamba and Claude run on this exact trick. 34 minutes. Bookmark & watch today. The math is older than the United States.

Eron

67,007 次观看 • 1 个月前

The effects of the drug “spice” in Birmingham- this is known as a “mamba attack” and leaves the user often incapacitated.

The effects of the drug “spice” in Birmingham- this is known as a “mamba attack” and leaves the user often incapacitated.

Scarcity News

175,334 次观看 • 2 年前

[Transformer] by Hand✍️📺 5-minute Video Tutorial Anna Rahn made this short video to explain the Transformer exercise for my Computer Vision course last spring. In 5 minutes, she demonstrates the key calculations of the Transformer by hand with pen and paper! Anna is a fantastic student. I am lucky to have her in my lab!

[Transformer] by Hand✍️📺 5-minute Video Tutorial Anna Rahn made this short video to explain the Transformer exercise for my Computer Vision course last spring. In 5 minutes, she demonstrates the key calculations of the Transformer by hand with pen and paper! Anna is a fantastic student. I am lucky to have her in my lab!

Tom Yeh

133,460 次观看 • 2 年前

As someone who ships LLM systems in production, this 22-minute Laplace transform video is the closest thing to a "why Mamba solves what transformers can't" explainer I've ever seen released for free. Everyone thinks scaling transformers is the only path forward. State space models beat them on long context using math Laplace built in 1785. This video shows the trick. Bookmark & watch today. The math is older than the United States.

As someone who ships LLM systems in production, this 22-minute Laplace transform video is the closest thing to a "why Mamba solves what transformers can't" explainer I've ever seen released for free. Everyone thinks scaling transformers is the only path forward. State space models beat them on long context using math Laplace built in 1785. This video shows the trick. Bookmark & watch today. The math is older than the United States.

fintex

26,102 次观看 • 2 个月前

Before she was the Gold Mamba, Jewell B Loyd was a kid in love with basketball’s artistry and grit. Our first episode, ‘Mamba Made’ dives deep into Jewell’s roots, showcasing her family support and relentless work ethic that shaped her into a Seattle Storm superstar. This is – MADE Presented by CarMax Season 2 Watch the full episode on YouTube and the WNBA App

Before she was the Gold Mamba, Jewell B Loyd was a kid in love with basketball’s artistry and grit. Our first episode, ‘Mamba Made’ dives deep into Jewell’s roots, showcasing her family support and relentless work ethic that shaped her into a Seattle Storm superstar. This is – MADE Presented by CarMax Season 2 Watch the full episode on YouTube and the WNBA App

WNBA

37,351 次观看 • 2 年前

The Red Flags of the Cagayan Governor Mamba Few days ago Cagayan Governor Mamba said that he sees no threat from the influx of the Chinese students in Cagayan, but what do we expect from a pro-Beijing governor? It makes sense, right? He knows! If what the sources are saying is correct, thousands of Chinese students are hard to miss especially in a provincial set-up. In fact if thousands of Chinese students enroll even in a university in Metro Manila, I highly doubt people will not notice it and wonder. I clearly remember how Cagayan Governor Mamba opposed the EDCA sites in his province and cannot stop praising China at the same time. But then there’s more… reading a lot about this last night and I guess even with the publicly available information, there’s really a basis to be concerned of this “friendship” between Gov. Mamba and China. A country with a direct conflict of interest with us, our national security and our national interests! :/ Mamba wants China to invest in Cagayan airport and seaport Chinese Ambassador to Manila Huang Xilian visited Cagayan November last year, he even went as far as saying that China is not just their steadfast ally and friend but “more so of a big a big brother” lols During the said visit Gov. Mamba urged China to invest in a seaport and an airport with a three-kilometer runway. Hayyy this Gov. Mamba is something else :/ Read more here: Mamba wants stronger ties with Beijing I do not understand how despite the repeated harassment of China in the West Philippine Sea against our own men and fisherfolks, within our EEZ… Governor still hopes for stronger ties with China. “I have always believed that neighbors cannot be enemies, otherwise you will never feel safe in your own home,” -Governor Mamba, Hangzhou China May 24 Read more here: Video (c): Senate of the Philippines Youtube Page Link:

The Red Flags of the Cagayan Governor Mamba Few days ago Cagayan Governor Mamba said that he sees no threat from the influx of the Chinese students in Cagayan, but what do we expect from a pro-Beijing governor? It makes sense, right? He knows! If what the sources are saying is correct, thousands of Chinese students are hard to miss especially in a provincial set-up. In fact if thousands of Chinese students enroll even in a university in Metro Manila, I highly doubt people will not notice it and wonder. I clearly remember how Cagayan Governor Mamba opposed the EDCA sites in his province and cannot stop praising China at the same time. But then there’s more… reading a lot about this last night and I guess even with the publicly available information, there’s really a basis to be concerned of this “friendship” between Gov. Mamba and China. A country with a direct conflict of interest with us, our national security and our national interests! :/ Mamba wants China to invest in Cagayan airport and seaport Chinese Ambassador to Manila Huang Xilian visited Cagayan November last year, he even went as far as saying that China is not just their steadfast ally and friend but “more so of a big a big brother” lols During the said visit Gov. Mamba urged China to invest in a seaport and an airport with a three-kilometer runway. Hayyy this Gov. Mamba is something else :/ Read more here: Mamba wants stronger ties with Beijing I do not understand how despite the repeated harassment of China in the West Philippine Sea against our own men and fisherfolks, within our EEZ… Governor still hopes for stronger ties with China. “I have always believed that neighbors cannot be enemies, otherwise you will never feel safe in your own home,” -Governor Mamba, Hangzhou China May 24 Read more here: Video (c): Senate of the Philippines Youtube Page Link:

Just

68,404 次观看 • 2 年前

New chapter! Context problems. Who can solve these by hand ✍️? Download PDF: The first five: 1. Mark the context: the window as cells you can count 2. What fills the window: the system prompt, the user, the model's reply 3. Turn by turn: every turn is the same block, and most of it is the reply 4. Will it fit? Number a new message into what's left, and see it run out 5. Turns until full: how many rounds before there is no room Previously, I shared Chapter 1: Token Problems 1 ~ 20 Why am I designing these math problems? AI is making people (including me) think less. It's just too easy to ask AI a few questions about a new concept and start to believe I get it. We used to practice on a coding problem, but now AI writes the code, and solving it technically is not the same as knowing anything. So I recast each concept as a simple math problem you have to work with pen and paper. Using our hands is one way to motivate ourselves to start thinking again. ✍️ If you solve them by hand, leave a comment to let me know. I will give you a shout out in my next post. Download, print, solve, and learn. ✍️ ~ Prof. Tom Yeh

New chapter! Context problems. Who can solve these by hand ✍️? Download PDF: The first five: 1. Mark the context: the window as cells you can count 2. What fills the window: the system prompt, the user, the model's reply 3. Turn by turn: every turn is the same block, and most of it is the reply 4. Will it fit? Number a new message into what's left, and see it run out 5. Turns until full: how many rounds before there is no room Previously, I shared Chapter 1: Token Problems 1 ~ 20 Why am I designing these math problems? AI is making people (including me) think less. It's just too easy to ask AI a few questions about a new concept and start to believe I get it. We used to practice on a coding problem, but now AI writes the code, and solving it technically is not the same as knowing anything. So I recast each concept as a simple math problem you have to work with pen and paper. Using our hands is one way to motivate ourselves to start thinking again. ✍️ If you solve them by hand, leave a comment to let me know. I will give you a shout out in my next post. Download, print, solve, and learn. ✍️ ~ Prof. Tom Yeh

Tom Yeh

11,651 次观看 • 1 个月前

Passing Medicare for All would lift a huge weight off the shoulders of small business owners like Mamba at Baobab Fare in Detroit. And you MUST stop by for the Samaki and espresso in passion fruit juice…

Passing Medicare for All would lift a huge weight off the shoulders of small business owners like Mamba at Baobab Fare in Detroit. And you MUST stop by for the Samaki and espresso in passion fruit juice…

Dr. Abdul El-Sayed

10,735 次观看 • 8 个月前

The full vid is on the site now. I changed up one of the scenes a bit. Alexandra (AI Actress) makes her full debut in “Codename: Big Mamba”. If you can’t support the site, please consider Liking and Sharing my work!

The full vid is on the site now. I changed up one of the scenes a bit. Alexandra (AI Actress) makes her full debut in “Codename: Big Mamba”. If you can’t support the site, please consider Liking and Sharing my work!

Size Cinema

18,772 次观看 • 3 个月前

Nemotron 3 Full Breakdown With the help of Joey Conway from NVIDIA AI getting into the specifics around why Nemotron 3 is kind of a big deal Biggest headline with Nemotron is: Hybrid Mamba Transformer, Latent MoE, and MTP Hybrid Mamba Transformer essentially attacks right at the Attention mechanism to make the overhead sub-quadratic, but unlike quantizing KV Cache or swapping out attention head, NVIDIA chose Mamba-2 Latent MoE helps further optimize on sparsity by down projecting the dimensions so you're doing less math and less memory movement between HBM and SRAM, you're saving a ton, and NVIDIA made a conscious choice to add more experts given the surplus Finally, MTP or multi token prediction where the model can see future tokens to be more expressive in training and also option to use for speculative decoding during inference Oh, also the model adopts the new OpenMDW 1.1 License

Nemotron 3 Full Breakdown With the help of Joey Conway from NVIDIA AI getting into the specifics around why Nemotron 3 is kind of a big deal Biggest headline with Nemotron is: Hybrid Mamba Transformer, Latent MoE, and MTP Hybrid Mamba Transformer essentially attacks right at the Attention mechanism to make the overhead sub-quadratic, but unlike quantizing KV Cache or swapping out attention head, NVIDIA chose Mamba-2 Latent MoE helps further optimize on sparsity by down projecting the dimensions so you're doing less math and less memory movement between HBM and SRAM, you're saving a ton, and NVIDIA made a conscious choice to add more experts given the surplus Finally, MTP or multi token prediction where the model can see future tokens to be more expressive in training and also option to use for speculative decoding during inference Oh, also the model adopts the new OpenMDW 1.1 License

Caleb Eom

50,460 次观看 • 3 个月前

Advanced RAG techniques by Hand ✍️ ~ Watch my lecture: Last year, I delivered a keynote lecture titled “Beginner’s Guide to RAG” sponsored by Sarah Torres. But don’t be fooled by the name. I walked through several advanced techniques the AI by Hand ✍️ way, breaking them down conceptually the the AI by Hand way. Today, many of those techniques are already powering real production systems. Had I filled my lecture with API calls and library-specific syntax, the content would have already become obsolete months ago. Fortunately, I focused on the fundamentals and concepts. My lecture is still as fresh and relevant as ever. Watch: ---- 100% original, made by hand ✍️ Join 40k readers of my newsletter:

Advanced RAG techniques by Hand ✍️ ~ Watch my lecture: Last year, I delivered a keynote lecture titled “Beginner’s Guide to RAG” sponsored by Sarah Torres. But don’t be fooled by the name. I walked through several advanced techniques the AI by Hand ✍️ way, breaking them down conceptually the the AI by Hand way. Today, many of those techniques are already powering real production systems. Had I filled my lecture with API calls and library-specific syntax, the content would have already become obsolete months ago. Fortunately, I focused on the fundamentals and concepts. My lecture is still as fresh and relevant as ever. Watch: ---- 100% original, made by hand ✍️ Join 40k readers of my newsletter:

Tom Yeh

20,685 次观看 • 11 个月前

Creating Bitcoin keys securely entirely off-line. I made this (extremely ugly but) functional prototype Codex32 wooden computer that can calculate Bitcoin keys using nothing more than dice, some paper and a pen. You do all the math by hand, including calculating the checksum.

Creating Bitcoin keys securely entirely off-line. I made this (extremely ugly but) functional prototype Codex32 wooden computer that can calculate Bitcoin keys using nothing more than dice, some paper and a pen. You do all the math by hand, including calculating the checksum.

FractalEncrypt ∞/21M

187,320 次观看 • 2 年前

llm.c by Hand✍️ C programming + matrix multiplication by hand This combination is perhaps as low as we can get to explain how the Transformer works. Special thanks to Andrej Karpathy for encouraging early feedback and tetsuo //: 👾 for helping me understand the pragma magic. I hope this exercise can help people peak further into the LLM black box.

llm.c by Hand✍️ C programming + matrix multiplication by hand This combination is perhaps as low as we can get to explain how the Transformer works. Special thanks to Andrej Karpathy for encouraging early feedback and tetsuo //: 👾 for helping me understand the pragma magic. I hope this exercise can help people peak further into the LLM black box.

Tom Yeh

311,609 次观看 • 2 年前