Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

Project #2: LLM Visualization So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)

Brendan Bycroft

4,913 subscribers

1,201,038 просмотров • 2 лет назад •via X (Twitter)

Образование Наука и технологии Новости и политика

Anya Rossi• Live Now

Private livecam show

Комментарии: 12

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

It also contains a walkthrough/guide of the steps, as well as a few interactive elements to play with. Why, you ask? For what purpose did I put all the time & effort into this project?

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

There's a real advantage to unpacking a set of abstractions, flattening them out. Abstractions can be useful for terseness and management, but they can be a real blocker to seeing the big picture.

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

With this, you can see the whole thing at once. You can see where the computation takes place, its complexity, and relative sizes of the tensors & weights.

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

The model with all the animations is tiiny, to make it tractable. For comparison, I threw in a few of the larger models (GPT-2, GPT-3), render-only. And when you see what it takes to just produce a single value in a mat-mul, the sheer scale of these things becomes apparent.

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

(Here's what goes into calculating a _single_ output value of a matrix-multiply)

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

What about understanding what each layer does? Uhh, sorry, won't be much help. The project just came out of "Let's build a 3D viz!", so the scope is a bit limited. It's more: here's a way to learn & digest the algorithm, and perhaps think about how to optimize the process.

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

As for what I got out of creating this: before I made it, I mostly knew how image convolution nets worked, but language-based models seemed kinda magical in comparison. Well, now I know them in a fair amount of detail!

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

I also learnt a good amount of GL (dF/dx, fwidth, ubos, instancing), and animation approaches. So, uhh, even if no-one sees this, the project definitely has some value to me.

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

Oh yeah, the link is here: Works best on desktop (sorry mobile). Left-click drag, right-click rotate, scroll to zoom. And hover over the tensor cells. Blue cells are weights/parameters, green cells are intermediate values. Each cell is a single number!

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

Well, I hope you find it interesting. Let me know your thoughts! And if someone makes it through the walkthrough and finds it a little ~incomplete towards the end I might even getting around to fix it (my attention has largely turned to other projects oops)

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

Short follow-up thread:

Фото профиля Brendan Bycroft

Brendan Bycroft2 лет назад

A technical guide to how I built it:

Похожие видео

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

elvis

39,431 просмотров • 1 год назад

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

Charles 🎉 Frye

21,976 просмотров • 1 год назад

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

@levelsio

182,634 просмотров • 10 месяцев назад

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

Reynante Martinez

21,985 просмотров • 10 месяцев назад

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,282 просмотров • 2 лет назад

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

Pietro Schirano

120,768 просмотров • 1 год назад

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

Jing Yu Koh

124,857 просмотров • 2 лет назад

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Broooooklyn

45,536 просмотров • 18 дней назад

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

TBPN

11,445 просмотров • 1 год назад

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

sphinx

49,162 просмотров • 2 лет назад

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

Wuraola Oyewusi

31,918 просмотров • 3 лет назад

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Cohere

736,226 просмотров • 27 дней назад

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

Kalyan KS

16,643 просмотров • 1 год назад

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 просмотров • 20 дней назад

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Rohan Paul

105,662 просмотров • 1 год назад

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

the government man

23,773 просмотров • 1 год назад

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

cocktail peanut

14,745 просмотров • 1 год назад

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Mickapouel

40,561 просмотров • 1 месяц назад

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

Jake Fitzgerald

22,113 просмотров • 1 месяц назад

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

Jay Alammar

184,375 просмотров • 7 месяцев назад