Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Project #2: LLM Visualization So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)

Brendan Bycroft

4,917 subscribers

1,201,152 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie Nachrichten & Politik

Anya Rossi• Live Now

Private livecam show

12 Kommentare

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

It also contains a walkthrough/guide of the steps, as well as a few interactive elements to play with. Why, you ask? For what purpose did I put all the time & effort into this project?

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

There's a real advantage to unpacking a set of abstractions, flattening them out. Abstractions can be useful for terseness and management, but they can be a real blocker to seeing the big picture.

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

With this, you can see the whole thing at once. You can see where the computation takes place, its complexity, and relative sizes of the tensors & weights.

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

The model with all the animations is tiiny, to make it tractable. For comparison, I threw in a few of the larger models (GPT-2, GPT-3), render-only. And when you see what it takes to just produce a single value in a mat-mul, the sheer scale of these things becomes apparent.

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

(Here's what goes into calculating a _single_ output value of a matrix-multiply)

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

What about understanding what each layer does? Uhh, sorry, won't be much help. The project just came out of "Let's build a 3D viz!", so the scope is a bit limited. It's more: here's a way to learn & digest the algorithm, and perhaps think about how to optimize the process.

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

As for what I got out of creating this: before I made it, I mostly knew how image convolution nets worked, but language-based models seemed kinda magical in comparison. Well, now I know them in a fair amount of detail!

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

I also learnt a good amount of GL (dF/dx, fwidth, ubos, instancing), and animation approaches. So, uhh, even if no-one sees this, the project definitely has some value to me.

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

Oh yeah, the link is here: Works best on desktop (sorry mobile). Left-click drag, right-click rotate, scroll to zoom. And hover over the tensor cells. Blue cells are weights/parameters, green cells are intermediate values. Each cell is a single number!

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

Well, I hope you find it interesting. Let me know your thoughts! And if someone makes it through the walkthrough and finds it a little ~incomplete towards the end I might even getting around to fix it (my attention has largely turned to other projects oops)

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

Short follow-up thread:

Profilbild von Brendan Bycroft

Brendan Bycroftvor 2 Jahren

A technical guide to how I built it:

Ähnliche Videos

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

elvis

39,431 Aufrufe • vor 1 Jahr

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

Charles 🎉 Frye

21,976 Aufrufe • vor 1 Jahr

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

@levelsio

182,634 Aufrufe • vor 10 Monaten

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

Reynante Martinez

21,985 Aufrufe • vor 10 Monaten

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

Pietro Schirano

120,814 Aufrufe • vor 1 Jahr

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,282 Aufrufe • vor 2 Jahren

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

Jing Yu Koh

124,878 Aufrufe • vor 2 Jahren

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Broooooklyn

45,536 Aufrufe • vor 29 Tagen

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

TBPN

11,445 Aufrufe • vor 1 Jahr

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

sphinx

49,162 Aufrufe • vor 2 Jahren

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

Wuraola Oyewusi

31,918 Aufrufe • vor 3 Jahren

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Cohere

738,286 Aufrufe • vor 1 Monat

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 Aufrufe • vor 1 Monat

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

Kalyan KS

16,643 Aufrufe • vor 1 Jahr

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Rohan Paul

105,834 Aufrufe • vor 1 Jahr

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

the government man

23,773 Aufrufe • vor 1 Jahr

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

cocktail peanut

14,745 Aufrufe • vor 1 Jahr

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Mickapouel

41,366 Aufrufe • vor 2 Monaten

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

Jake Fitzgerald

22,113 Aufrufe • vor 2 Monaten

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

Jay Alammar

184,406 Aufrufe • vor 7 Monaten