Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Project #2: LLM Visualization So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)

Brendan Bycroft

4,925 subscribers

1,201,234 görüntüleme • 2 yıl önce •via X (Twitter)

Eğitim Bilim & Teknoloji Haberler & Politika

Anya Rossi• Live Now

Private livecam show

12 Yorum

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

It also contains a walkthrough/guide of the steps, as well as a few interactive elements to play with. Why, you ask? For what purpose did I put all the time & effort into this project?

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

There's a real advantage to unpacking a set of abstractions, flattening them out. Abstractions can be useful for terseness and management, but they can be a real blocker to seeing the big picture.

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

With this, you can see the whole thing at once. You can see where the computation takes place, its complexity, and relative sizes of the tensors & weights.

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

The model with all the animations is tiiny, to make it tractable. For comparison, I threw in a few of the larger models (GPT-2, GPT-3), render-only. And when you see what it takes to just produce a single value in a mat-mul, the sheer scale of these things becomes apparent.

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

(Here's what goes into calculating a _single_ output value of a matrix-multiply)

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

What about understanding what each layer does? Uhh, sorry, won't be much help. The project just came out of "Let's build a 3D viz!", so the scope is a bit limited. It's more: here's a way to learn & digest the algorithm, and perhaps think about how to optimize the process.

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

As for what I got out of creating this: before I made it, I mostly knew how image convolution nets worked, but language-based models seemed kinda magical in comparison. Well, now I know them in a fair amount of detail!

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

I also learnt a good amount of GL (dF/dx, fwidth, ubos, instancing), and animation approaches. So, uhh, even if no-one sees this, the project definitely has some value to me.

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

Oh yeah, the link is here: Works best on desktop (sorry mobile). Left-click drag, right-click rotate, scroll to zoom. And hover over the tensor cells. Blue cells are weights/parameters, green cells are intermediate values. Each cell is a single number!

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

Well, I hope you find it interesting. Let me know your thoughts! And if someone makes it through the walkthrough and finds it a little ~incomplete towards the end I might even getting around to fix it (my attention has largely turned to other projects oops)

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

Short follow-up thread:

Brendan Bycroft profil fotoğrafı

Brendan Bycroft2 yıl önce

A technical guide to how I built it:

Benzer Videolar

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

LLM Visualization This is actually pretty amazing! It helps to visualize the core components of LLMs like nano-gpt and GPT-3.

elvis

39,431 görüntüleme • 1 yıl önce

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

We've run thousands of LLM inference serving benchmarks at Modal. We're releasing the results so you don't have to. We're releasing the code so that you can. Introducing: The LLM Engineer's Almanac. Just in time for the AI Engineer World's Fair.

Charles 🎉 Frye

21,976 görüntüleme • 1 yıl önce

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

Nope, but it has full ChatGPT access via IRC and runs a real but small LLM in DOS

@levelsio

182,634 görüntüleme • 10 ay önce

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

A little behind-the-scenes look on the latest visualization project I did. Lit and directed the traditional way. Created with Blender and rendered with Cycles. #b3d #blender

Reynante Martinez

21,985 görüntüleme • 10 ay önce

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,282 görüntüleme • 2 yıl önce

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

I needed a quick tool to convert any sort of GitHub links into a format an LLM could read. So I built it. GitHub → LLM It's completely free and works with projects, folders, or files 🪄 Deployed on Replit ⠕

Pietro Schirano

120,814 görüntüleme • 1 yıl önce

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

LLM agents have demonstrated promise in their ability to automate computer tasks, but face challenges with multi-step reasoning and planning. Towards addressing this, we propose an inference-time tree search algorithm for LLM agents to explicitly perform exploration and multi-step planning in interactive web environments. It is the first tree search algorithm for LLM agents that shows effectiveness on realistic and complex web environments: on the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%.

Jing Yu Koh

124,878 görüntüleme • 2 yıl önce

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Finally found a meaningful scenario for running LLM in the browser, peering into the internals of LLM to see what's really happening inside

Broooooklyn

45,882 görüntüleme • 1 ay önce

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

"Orgs and cos building MCP servers are taking an LLM-first approach to what the API needs to expose to the agent(s)." - Nikunj Handa from OpenAI "For example, Stripe has a bunch of APIs that can be used to create a subscription/customer/product/price. For an LLM, it can just combine that into a single function." "Instead of returning this massive JSON object, they can return something very specific to the task being solved, so that the LLM can more easily understand what's happening." "It's an opportunity to rewrite your APIs to be very LLM-first. Why do 2 hours of work, when you can do it in 4 lines of code under a minute?"

TBPN

11,445 görüntüleme • 1 yıl önce

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

If you have a mac, it’s super easy to run any LLM on it now. Basically your own private jail broken chatgpt. Runs offline. Literally a 2 step process. 1) go to and download the app 2) open terminal -> run the model 3) there's no step 3

sphinx

49,162 görüntüleme • 2 yıl önce

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

ChatGPT, explained in Yoruba.(1/2) 🎥: Bankole Oluwafemi #ChatGPT #yoruba #LLM

Wuraola Oyewusi

31,918 görüntüleme • 3 yıl önce

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Introducing: Cohere Command A+ We’ve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.

Cohere

738,393 görüntüleme • 1 ay önce

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

The bottleneck in LLM inference isn't compute. It's how fast you can move the weights. Our CTO Mathias Lechner, Mathias Lechner, joins Piotr Mazurek, Piotr Mazurek (in SF 🌉), from our inference team, to discuss what actually limits token throughput and how we're optimizing for it.

Liquid AI

21,393 görüntüleme • 1 ay önce

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

👨🏻‍💻 LLM Engineer Toolkit - Collection of 120+ LLM Libraries Category Wise LLM Engineer Toolkit repository contains a curated list of 120+ LLM libraries category wise. 🚀 LLM Training 🧱 LLM Application Development 🩸LLM RAG 🟩 LLM Inference 🚧 LLM Serving 📤 LLM Data Extraction 🌠 LLM Data Generation 💎 LLM Agents ⚖️ LLM Evaluation 🔍 LLM Monitoring 📅 LLM Prompts 📝 LLM Structured Outputs 🛑 LLM Safety and Security 💠 LLM Embedding Models ❇️ Others Repo -

Kalyan KS

16,643 görüntüleme • 1 yıl önce

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section. (link in comment)

Rohan Paul

105,834 görüntüleme • 1 yıl önce

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

I made an infinite website that makes itself. It has every possible page so now nobody ever needs to make a website again Every time you visit a page that's never been visited before, the path and filename alone is passed to an LLM that generates the entire content

the government man

23,773 görüntüleme • 1 yıl önce

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

Hallucinator I wrote a gradio app to play with Standard Intelligence's Hertz model. Hertz inference works like LLM but purely audio based--the inference happens voice to voice. Hallucinator uses Hertz to autocomplete any speech clip, often resulting in some hilarious hallucinations.

cocktail peanut

14,745 görüntüleme • 1 yıl önce

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Such a fun project. Glad I got to play a small part in it! The 🐐 of streaming

Mickapouel

41,366 görüntüleme • 2 ay önce

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

To generate 3D models in Codex with text-to-cad: Start a new project, write your prompt for a 3D model, and include a link to clone the text-to-cad repo. Codex handles the rest. The repo is open source and all scripts / ui run locally. Link below.

Jake Fitzgerald

22,113 görüntüleme • 2 ay önce

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

The Illustrated NeurIPS 2025: A Visual Map of the AI Frontier New blog post! NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters, summaries, and Cohere LLM-generated explanations that make the field easier to grasp. Link in thread!

Jay Alammar

184,406 görüntüleme • 7 ay önce