Загрузка видео...

Не удалось загрузить видео

На главную

ComfyUI Tutorial for Beginners: Text-to-Image & Image-to-Image Workflows One graph, two workflows and Module 2 is where you build it node by node instead of loading someone else's. We start from a blank canvas and wire every connection, then change a single value and turn the same graph into...

ComfyUI's profile picture

ComfyUI

63,526 subscribers

61,166 просмотров • 1 месяц назад •via X (Twitter)

Наука и технологииОбразование
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 8

Фото профиля ComfyUI
ComfyUI1 месяц назад

Run the workflow:

Фото профиля ZenithAi
ZenithAi1 месяц назад

Finally a ComfyUI tutorial that explains why the nodes connect

Фото профиля Evia AI
Evia AI1 месяц назад

Great tutorial for making ComfyUI workflows truly understandable

Фото профиля Elise Tech Ai
Elise Tech Ai1 месяц назад

Excellent beginner-friendly breakdown—learning the logic behind each node makes ComfyUI much easier to master.

Фото профиля LRD
LRD1 месяц назад

Top

Фото профиля Sani Ai Tech
Sani Ai Tech1 месяц назад

Great beginner-friendly breakdown that makes ComfyUI far less intimidating

Фото профиля Eva Morgan
Eva Morgan1 месяц назад

ComfyUI’s node-based workflow gives creators more control and understanding behind the generation process, not just the final output.

Фото профиля Alice The Ai Expert
Alice The Ai Expert1 месяц назад

Perfect beginner breakdown building node by node makes ComfyUI finally click!

Похожие видео

ComfyUI Tutorial for Beginners: Full Node Graph Basics It's a production-grade generation backend with an adaptable front end and Module 1 is where the node graph stops being intimidating. We boot up Comfy Cloud, tour the full UI, and build your first workflow from the ground up. What you'll learn: → Comfy Cloud vs. Desktop - which to use and when → Navigating the graph: pan, zoom, frame, select → The 5 core nodes behind every text-to-image workflow → Checkpoints, LoRAs, VAEs, CLIP - matching model versions → Get/Set nodes + reroutes to kill the spaghetti → Subgraphs: complex workflows collapsed into one node → App Mode: a front end for non-technical collaborators → Templates vs. Blueprints vs. Workflows → The 7 node categories that make the whole library click No installs. No missing models. Open it and generate.
1:12:15

ComfyUI Tutorial for Beginners: Full Node Graph Basics It's a production-grade generation backend with an adaptable front end and Module 1 is where the node graph stops being intimidating. We boot up Comfy Cloud, tour the full UI, and build your first workflow from the ground up. What you'll learn: → Comfy Cloud vs. Desktop - which to use and when → Navigating the graph: pan, zoom, frame, select → The 5 core nodes behind every text-to-image workflow → Checkpoints, LoRAs, VAEs, CLIP - matching model versions → Get/Set nodes + reroutes to kill the spaghetti → Subgraphs: complex workflows collapsed into one node → App Mode: a front end for non-technical collaborators → Templates vs. Blueprints vs. Workflows → The 7 node categories that make the whole library click No installs. No missing models. Open it and generate.

ComfyUI

15,034 просмотров • 2 месяцев назад

ComfyUI for Animation: Keyframe Exploration Character reference → camera angle → finished keyframe. One graph, any shot. Step 1: load your character or character sheet into the input node. Step 2: drag the direction, elevation, and distance controls on the Multi-Angle Camera node. It writes the camera description automatically and passes it to GPT Image 2 as the prompt. Step 3 (optional): load a background to place your character in a scene, and describe a pose, expression, or action in the Additional Action node. Turn either off if you don't need it. Step 4: run it. GPT Image 2 combines all three inputs into the keyframe. Click the link below to run the workflow ⬇️
0:56

ComfyUI for Animation: Keyframe Exploration Character reference → camera angle → finished keyframe. One graph, any shot. Step 1: load your character or character sheet into the input node. Step 2: drag the direction, elevation, and distance controls on the Multi-Angle Camera node. It writes the camera description automatically and passes it to GPT Image 2 as the prompt. Step 3 (optional): load a background to place your character in a scene, and describe a pose, expression, or action in the Additional Action node. Turn either off if you don't need it. Step 4: run it. GPT Image 2 combines all three inputs into the keyframe. Click the link below to run the workflow ⬇️

ComfyUI

19,950 просмотров • 24 дней назад

7 steps and 2 clues created a woman who was bought a drink by a stranger. She doesn't exist. A stranger sat at a club bar last week. She picked out a man, started a conversation, bought him a round. He smiled back. The whole scene lived in a 6-second clip that pulled comments asking who she was. She was never there. No club, no drinks, no face that belongs to anyone. Here is the full build, 7 steps end to end. Open Pinterest, search “girl selfie,” save one image. Open Nano Banana Pro and swap the face with a single prompt. That photo now belongs to no one. Move the image into a node editor. Create 3 nodes: a reference image node, a text node, a video generator node. Drop the photo into the image node. Write 1 video prompt into the text node, the line that turns a still frame into motion. Connect text to text-in, image to image-in. Select Kling 2.6, set duration, switch sound on. Hit generate. 2 prompts. One face that exists nowhere. A 6-second clip people argued was real. The next person who buys you a drink at a bar might run on a text prompt.
0:45

7 steps and 2 clues created a woman who was bought a drink by a stranger. She doesn't exist. A stranger sat at a club bar last week. She picked out a man, started a conversation, bought him a round. He smiled back. The whole scene lived in a 6-second clip that pulled comments asking who she was. She was never there. No club, no drinks, no face that belongs to anyone. Here is the full build, 7 steps end to end. Open Pinterest, search “girl selfie,” save one image. Open Nano Banana Pro and swap the face with a single prompt. That photo now belongs to no one. Move the image into a node editor. Create 3 nodes: a reference image node, a text node, a video generator node. Drop the photo into the image node. Write 1 video prompt into the text node, the line that turns a still frame into motion. Connect text to text-in, image to image-in. Select Kling 2.6, set duration, switch sound on. Hit generate. 2 prompts. One face that exists nowhere. A 6-second clip people argued was real. The next person who buys you a drink at a bar might run on a text prompt.

RGK

16,209 просмотров • 4 месяцев назад

How to Convert Your ComfyUI Workflow into an APP Mode Workflow App Mode is a new way to view and run any ComfyUI workflow. When you enter App Mode, the node graph disappears and is replaced by a clean, purpose-built interface: just the inputs your user needs.
4:04

How to Convert Your ComfyUI Workflow into an APP Mode Workflow App Mode is a new way to view and run any ComfyUI workflow. When you enter App Mode, the node graph disappears and is replaced by a clean, purpose-built interface: just the inputs your user needs.

ComfyUI

79,362 просмотров • 5 месяцев назад

Seedream 5.0 Pro is now in ComfyUI via Partner Nodes. ByteDance's latest image model brings: → Character & product consistency across edits → Precision region editing → Infographics, flowcharts & structured layouts → In-image text across 14 languages One model. Text-to-image, editing, and multi-image inputs. Click the link below for the workflow 👇
0:32

Seedream 5.0 Pro is now in ComfyUI via Partner Nodes. ByteDance's latest image model brings: → Character & product consistency across edits → Precision region editing → Infographics, flowcharts & structured layouts → In-image text across 14 languages One model. Text-to-image, editing, and multi-image inputs. Click the link below for the workflow 👇

ComfyUI

14,756 просмотров • 3 месяцев назад

Ideogram 4.5 is now in ComfyUI. It's an edit model built to fix drift. 4.5 keeps the rest of your image intact, edit after edit. → Multi-turn edits that stay clean → Edit 24MP+ images without downscaling, with crop edges preserved so the edit stitches back in seamlessly → Translate text in signage, packaging, and banners → Style reference, sketch-to-image, depth-to-image, and photo restoration Click the link below to run the workflows ⬇️
0:30

Ideogram 4.5 is now in ComfyUI. It's an edit model built to fix drift. 4.5 keeps the rest of your image intact, edit after edit. → Multi-turn edits that stay clean → Edit 24MP+ images without downscaling, with crop edges preserved so the edit stitches back in seamlessly → Translate text in signage, packaging, and banners → Style reference, sketch-to-image, depth-to-image, and photo restoration Click the link below to run the workflows ⬇️

ComfyUI

27,357 просмотров • 18 дней назад

ComfyUI for Advertising: Storyboards & Style Frames from a Shot List Shot list → storyboard → style frames. Two passes, one graph. Pass 1: type your shots into the shot list node. GPT Image returns all 9 panels as a single sketch sheet, plus each panel saved out on its own. Product image optional as reference. Pass 2: hand that sheet + a mood board to Nano Banana Pro. Every panel comes back as commercial photography, graded to your board. Works with hand-drawn boards too.
1:14

ComfyUI for Advertising: Storyboards & Style Frames from a Shot List Shot list → storyboard → style frames. Two passes, one graph. Pass 1: type your shots into the shot list node. GPT Image returns all 9 panels as a single sketch sheet, plus each panel saved out on its own. Product image optional as reference. Pass 2: hand that sheet + a mood board to Nano Banana Pro. Every panel comes back as commercial photography, graded to your board. Works with hand-drawn boards too.

ComfyUI

13,329 просмотров • 1 месяц назад

Go from concept to campaign film in a matter of hours. All with just a single starting image. And all without the need to shut down Times Square. Start by sketching out your idea and generating your hero image. Then use Nano Banana Pro to create your storyboard and Gen-4.5 Image to Video to bring it all to life. Reply MOVE for the full workflow BTS.
0:46

Go from concept to campaign film in a matter of hours. All with just a single starting image. And all without the need to shut down Times Square. Start by sketching out your idea and generating your hero image. Then use Nano Banana Pro to create your storyboard and Gen-4.5 Image to Video to bring it all to life. Reply MOVE for the full workflow BTS.

Runway

77,258 просмотров • 8 месяцев назад

FLUX 3 Video is live in ComfyUI via Partner Nodes Black Forest Labs FLUX 3 Model highlights: → 20s in a single generation → native audio, not dubbed on after → text-to-video, image-to-video, keyframes, continuation → multi-scene and multi-angle in one prompt → multilingual dialogue with lipsync that holds → typography that sits in the scene → chain clips agentically for longer stories One model across video, audio, image, and action. Click the link below to download the workflows and learn more with blog
0:31

FLUX 3 Video is live in ComfyUI via Partner Nodes Black Forest Labs FLUX 3 Model highlights: → 20s in a single generation → native audio, not dubbed on after → text-to-video, image-to-video, keyframes, continuation → multi-scene and multi-angle in one prompt → multilingual dialogue with lipsync that holds → typography that sits in the scene → chain clips agentically for longer stories One model across video, audio, image, and action. Click the link below to download the workflows and learn more with blog

ComfyUI

25,715 просмотров • 2 месяцев назад

ComfyUI for Animation: Concept Art to Any Angle Concept art → 3D proxy → any angle you want. One graph, two methods. Step 1: load your concept art. Tripo generates a 3D model and saves it as a GLB. Drag it into Load 3D and set material mode to Normal. Step 2 (stills): rotate the viewport to your angle. Load 3D outputs the view, and GPT Image 2 renders it back as a polished on-style illustration. Step 3 (camera moves): record a path in the viewport. That clip becomes the motion reference for Seedance 2.0, which generates the move through the stylized scene. Click the link below to run the workflow ⬇️
1:30

ComfyUI for Animation: Concept Art to Any Angle Concept art → 3D proxy → any angle you want. One graph, two methods. Step 1: load your concept art. Tripo generates a 3D model and saves it as a GLB. Drag it into Load 3D and set material mode to Normal. Step 2 (stills): rotate the viewport to your angle. Load 3D outputs the view, and GPT Image 2 renders it back as a polished on-style illustration. Step 3 (camera moves): record a path in the viewport. That clip becomes the motion reference for Seedance 2.0, which generates the move through the stylized scene. Click the link below to run the workflow ⬇️

ComfyUI

42,630 просмотров • 1 месяц назад

CLIP by hand ✍️ ~ 13 steps walkthrough below CLIP, Contrastive Language-Image Pre-training, is OpenAI's answer to a question that sounds impossible: how do you put a sentence and a picture in the same space? CLIP shipped when OpenAI was still open, and those embeddings were shared far and wide. Almost every multimodal model you use today descends from them. How does it work? Goal: learn one shared embedding space for text and images. = 1. Given = A mini batch of three text-image pairs. OpenAI trained the original on 400 million. = 2. Text to vectors = Let us look up each word with word2vec. = 3. Image to vectors = We cut each image into two patches and flatten them. Now text and pixels are both just numbers. = 4. The other pairs = Repeat steps 2 and 3 for the rest of the batch. = 5. Encode = Let us push both sides through their encoders, a linear layer and a ReLU. In practice these are transformers, but the shape of the operation is the same. = 6. Mean pooling = We average across the columns, so each image and each sentence collapses to a single vector. = 7. Projection = The text vectors are 3D and the image vectors are 4D, so they cannot be compared at all. A linear layer projects both to 2D. That 2D space is the shared embedding space, and getting here is the whole point of the model. = 8. Prepare for matmul = Let us copy the text vectors down and the transposed image vectors across. = 9. MatMul = We multiply, which takes the dot product of every text vector with every image vector. Each cell is one estimate of how well a sentence matches a picture. = 10. Softmax, e to the power = Raise e to each cell. To keep it hand sized we approximate e with 3. = 11. Softmax, sum = Sum each row for image to text, each column for text to image. = 12. Softmax, normalize = Divide, and out come two similarity matrices, one per direction. = 13. Loss gradients = The targets are identity matrices: a pair that belongs together should score 1, every other cell 0. Subtract the target from the similarity and you have the gradients, in both directions. The takeaway: pairing a picture with a sentence comes down to a single dot product. Everything before step 9 is the work of getting them into one shared space, so that the dot product finally means something. 💾 Save this post!
0:36

CLIP by hand ✍️ ~ 13 steps walkthrough below CLIP, Contrastive Language-Image Pre-training, is OpenAI's answer to a question that sounds impossible: how do you put a sentence and a picture in the same space? CLIP shipped when OpenAI was still open, and those embeddings were shared far and wide. Almost every multimodal model you use today descends from them. How does it work? Goal: learn one shared embedding space for text and images. = 1. Given = A mini batch of three text-image pairs. OpenAI trained the original on 400 million. = 2. Text to vectors = Let us look up each word with word2vec. = 3. Image to vectors = We cut each image into two patches and flatten them. Now text and pixels are both just numbers. = 4. The other pairs = Repeat steps 2 and 3 for the rest of the batch. = 5. Encode = Let us push both sides through their encoders, a linear layer and a ReLU. In practice these are transformers, but the shape of the operation is the same. = 6. Mean pooling = We average across the columns, so each image and each sentence collapses to a single vector. = 7. Projection = The text vectors are 3D and the image vectors are 4D, so they cannot be compared at all. A linear layer projects both to 2D. That 2D space is the shared embedding space, and getting here is the whole point of the model. = 8. Prepare for matmul = Let us copy the text vectors down and the transposed image vectors across. = 9. MatMul = We multiply, which takes the dot product of every text vector with every image vector. Each cell is one estimate of how well a sentence matches a picture. = 10. Softmax, e to the power = Raise e to each cell. To keep it hand sized we approximate e with 3. = 11. Softmax, sum = Sum each row for image to text, each column for text to image. = 12. Softmax, normalize = Divide, and out come two similarity matrices, one per direction. = 13. Loss gradients = The targets are identity matrices: a pair that belongs together should score 1, every other cell 0. Subtract the target from the similarity and you have the gradients, in both directions. The takeaway: pairing a picture with a sentence comes down to a single dot product. Everything before step 9 is the work of getting them into one shared space, so that the dot product finally means something. 💾 Save this post!

Tom Yeh

20,929 просмотров • 2 месяцев назад

I gave GPT Image 2.5 an iPhone Duo commercial concept instead of asking for one standalone image. The interesting result was the storyboard. GPT Image 2.5 is coming to CapCut, where it will be available through Design Studio and AI Image. That means the visual planning stage can connect directly with the rest of the creative workflow. Start with the product idea → build the storyboard → move into video production → refine with AI Edit. If one scene feels off, you can keep working from the same creative direction instead of rebuilding everything. That feels much closer to commercial production than one-shot generation. #GPTImage25 #CapCutPC #AIVideo #Seedance25
0:27

I gave GPT Image 2.5 an iPhone Duo commercial concept instead of asking for one standalone image. The interesting result was the storyboard. GPT Image 2.5 is coming to CapCut, where it will be available through Design Studio and AI Image. That means the visual planning stage can connect directly with the rest of the creative workflow. Start with the product idea → build the storyboard → move into video production → refine with AI Edit. If one scene feels off, you can keep working from the same creative direction instead of rebuilding everything. That feels much closer to commercial production than one-shot generation. #GPTImage25 #CapCutPC #AIVideo #Seedance25

Kuria | AI

11,854 просмотров • 1 месяц назад

In May, we integrated 11 new models spanning image, 3D, audio, video, and multimodal. The highlights: → Krea 2 — style-first image generation, live as a Partner Node on day one. Competes on how the frame looks, not just what's in it → VOID (Netflix) — removes objects and everything they caused: shadows, reflections, physical interactions. Apache 2.0, natively supported → Tripo 3.1 + TripoSplat — one image to a full 3D Gaussian asset, end to end Also live in May: Gemma 4, HidDream-O1-Image, Stable Audio 3, BiRefNet, MoGe, Claude, OpenRouter, Luma UNI-1 Learn more with our blog 👇
0:47

In May, we integrated 11 new models spanning image, 3D, audio, video, and multimodal. The highlights: → Krea 2 — style-first image generation, live as a Partner Node on day one. Competes on how the frame looks, not just what's in it → VOID (Netflix) — removes objects and everything they caused: shadows, reflections, physical interactions. Apache 2.0, natively supported → Tripo 3.1 + TripoSplat — one image to a full 3D Gaussian asset, end to end Also live in May: Gemma 4, HidDream-O1-Image, Stable Audio 3, BiRefNet, MoGe, Claude, OpenRouter, Luma UNI-1 Learn more with our blog 👇

ComfyUI

20,784 просмотров • 4 месяцев назад

Comfy MCP is now local and open-source! The #1 ask after Comfy Cloud MCP shipped in June. Your agent sees your actual install. So every node, custom nodes included, and every model on disk. It fetches files, starts your instance, gets a workflow to the point where it runs. It is now the easiest way to help with your local Minimax H3 workflows! Cloud MCP still does everything it did. Tell your agent where a job goes, or let it decide.
0:20

Comfy MCP is now local and open-source! The #1 ask after Comfy Cloud MCP shipped in June. Your agent sees your actual install. So every node, custom nodes included, and every model on disk. It fetches files, starts your instance, gets a workflow to the point where it runs. It is now the easiest way to help with your local Minimax H3 workflows! Cloud MCP still does everything it did. Tell your agent where a job goes, or let it decide.

ComfyUI

185,508 просмотров • 2 месяцев назад

Workflows vs. Graphs, clearly explained! workflows are great, and almost everyone has one. here is the ceiling: a workflow decides every step before it runs. you drew eight boxes in March. six months later the same three fire, every single time, and the other five have never once been reached. then a case arrives that nobody drew, and it goes to the closest wrong box. quietly, with a green status, because from the inside that looks exactly like success. Graph engineering fixes this by moving the decision: not what the steps do, but when the steps get chosen. you need both, and here is the sentence that resolves the whole confusion: a workflow decides the steps before it runs. a graph decides them while it runs. ↳ drawn in advance: the boxes, the branches, the order, the error path ↳ decided at runtime: how many units exist, what each one is allowed to see, which ones get created at all Prompts → Context → Harness → Loops → Graphs branches do not make it a graph. the branches were drawn in advance too, which means every one of them is a case you already thought of. the trick is knowing which part is allowed to be fixed. the node kinds are fixed. a splitter is a splitter, a gate is a gate, a merge is code. what is not fixed is how many of them exist this run, and that is decided after something has been read. one thing to know before you scale it. a workflow fails in a way that never pages anyone. ↳ the wrong branch ran, every check inside it passed, and the output is well formed ↳ nothing errored, because routing to the wrong box is not an error, it is a route that last one catches careful people. you cannot test your way out of it either, because the test suite was written from the same diagram that has the gap in it. and the one that eats whole nights: a workflow that has never surprised you is not stable, it is narrow. if it has run four hundred times and produced the same three shapes, it is not handling your work. it is handling the part of your work that fits it, and you have quietly stopped sending it the rest. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓
0:51

Workflows vs. Graphs, clearly explained! workflows are great, and almost everyone has one. here is the ceiling: a workflow decides every step before it runs. you drew eight boxes in March. six months later the same three fire, every single time, and the other five have never once been reached. then a case arrives that nobody drew, and it goes to the closest wrong box. quietly, with a green status, because from the inside that looks exactly like success. Graph engineering fixes this by moving the decision: not what the steps do, but when the steps get chosen. you need both, and here is the sentence that resolves the whole confusion: a workflow decides the steps before it runs. a graph decides them while it runs. ↳ drawn in advance: the boxes, the branches, the order, the error path ↳ decided at runtime: how many units exist, what each one is allowed to see, which ones get created at all Prompts → Context → Harness → Loops → Graphs branches do not make it a graph. the branches were drawn in advance too, which means every one of them is a case you already thought of. the trick is knowing which part is allowed to be fixed. the node kinds are fixed. a splitter is a splitter, a gate is a gate, a merge is code. what is not fixed is how many of them exist this run, and that is decided after something has been read. one thing to know before you scale it. a workflow fails in a way that never pages anyone. ↳ the wrong branch ran, every check inside it passed, and the output is well formed ↳ nothing errored, because routing to the wrong box is not an error, it is a route that last one catches careful people. you cannot test your way out of it either, because the test suite was written from the same diagram that has the gap in it. and the one that eats whole nights: a workflow that has never surprised you is not stable, it is narrow. if it has run four hundred times and produced the same three shapes, it is not handling your work. it is handling the part of your work that fits it, and you have quietly stopped sending it the rest. below i have quoted my full guide on graph engineering. it covers the three topologies, the verifier patterns, and where the gate should actually open. save this and read it below ↓

Hanako

16,810 просмотров • 1 месяц назад

context engineering vs graph engineering. every few months the list gets a new word and everyone treats it as a replacement for the last one. these two are not on the same list. one decides what the model sees this turn, the other decides what exists at all. the cleanest way to tell them apart is to ask what a single unit of work looks like. > context engineering is the window the window opens empty, every single time. you assemble what goes in it. the prompt, the docs, the history, the tool results. the assembling is the work. the window only grows. it never shrinks on its own, so eventually something gets dropped. usually from the middle. usually without telling you. then the turn ends and the window is thrown away. not archived, thrown away. the next turn opens empty again and you re-explain what you already explained. good context engineering is knowing what to leave out, not what to pack in. the unit of work is one window. > graph engineering is the structure the same material arrives from the same sources. instead of packing it into a window, you pull entities out of it, resolve the duplicates into one node, and write typed edges between them. nothing here is stored as text you hope to find again. it is stored as a thing with a name and its connections to other things. when the turn ends, the graph is still there. the next turn does not start from zero. it starts by querying what already exists, and the query walks edges instead of guessing at similarity. good graph engineering is deciding what counts as the same thing twice. the unit of work is one relationship. > they are not alternatives the graph is what refills the window. context engineering decides what fits. graph engineering decides what there is to choose from. remove the graph and every session starts blind. remove the context work and the best structure in the world arrives as an unreadable dump. that also tells you which one broke. the answer drifted from what you actually said, or forgot something from this same session. that is the window. the answer is coherent but invents a connection that does not exist, or cannot join two facts it has clearly seen. that is the structure. people debug the prompt because the prompt is the easiest thing to edit. it keeps taking the blame for failures that live a layer down. save this - then read the full breakdown below
0:40

context engineering vs graph engineering. every few months the list gets a new word and everyone treats it as a replacement for the last one. these two are not on the same list. one decides what the model sees this turn, the other decides what exists at all. the cleanest way to tell them apart is to ask what a single unit of work looks like. > context engineering is the window the window opens empty, every single time. you assemble what goes in it. the prompt, the docs, the history, the tool results. the assembling is the work. the window only grows. it never shrinks on its own, so eventually something gets dropped. usually from the middle. usually without telling you. then the turn ends and the window is thrown away. not archived, thrown away. the next turn opens empty again and you re-explain what you already explained. good context engineering is knowing what to leave out, not what to pack in. the unit of work is one window. > graph engineering is the structure the same material arrives from the same sources. instead of packing it into a window, you pull entities out of it, resolve the duplicates into one node, and write typed edges between them. nothing here is stored as text you hope to find again. it is stored as a thing with a name and its connections to other things. when the turn ends, the graph is still there. the next turn does not start from zero. it starts by querying what already exists, and the query walks edges instead of guessing at similarity. good graph engineering is deciding what counts as the same thing twice. the unit of work is one relationship. > they are not alternatives the graph is what refills the window. context engineering decides what fits. graph engineering decides what there is to choose from. remove the graph and every session starts blind. remove the context work and the best structure in the world arrives as an unreadable dump. that also tells you which one broke. the answer drifted from what you actually said, or forgot something from this same session. that is the window. the answer is coherent but invents a connection that does not exist, or cannot join two facts it has clearly seen. that is the structure. people debug the prompt because the prompt is the easiest thing to edit. it keeps taking the blame for failures that live a layer down. save this - then read the full breakdown below

Hanako

19,160 просмотров • 2 месяцев назад

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).
1:20

[CLIP] by Hand ✍️ The CLIP (Contrastive Language–Image Pre-training) model, a groundbreaking work by OpenAI, redefines the intersection of computer vision and natural language processing. It is the basis of all the multi-modal foundation models we see today. How does CLIP work? Goal: 🟨 Learn a shared embedding space for text and image [1] Given ↳ A mini batch of 3 text-image pairs ↳ OpenAI used 400 million text-image pairs to train its original CLIP model. Process 1st pair: "big table" [2] 🟪 Text → 2 Vectors (3D) ↳ Look up word embedding vectors using word2vec. [3] 🟩 Image → 2 Vectors (4D) ↳ Divide the image into two patches. ↳ Flatten each patch [4] Process other pairs ↳ Repeat [2]-[3] [5] 🟪 Text Encoder & 🟩 Image Encoder ↳ Encode input vectors into feature vectors ↳ Here, both encoders are simple one layer perceptron (linear + ReLU) ↳ In practice, the encoders are usually transformer models. [6] 🟪 🟩 Mean Pooling: 2 → 1 vector ↳ Average 2 feature vectors into a single vector by averaging across the columns ↳ The goal is to have one vector to represent each image or text [7] 🟪 🟩 -> 🟨 Projection ↳ Note that the text and image feature vectors from the encoders have different dimensions (3D vs. 4D). ↳ Use a linear layer to project image and text vectors to a 2D shared embedding space. 🏋️ Contrastive Pre-training 🏋️ [8] Prepare for MatMul ↳ Copy text vectors (T1,T2,T3) ↳ Copy the transpose of image vectors (I1,I2,I3) ↳ They are all in the 2D shared embedding space. [9] 🟦 MatMul ↳ Multiply T and I matrices. ↳ This is equivalent to taking dot product between every pair of image and text vectors. ↳ The purpose is to use dot product to estimate the similarity between a pair of image-text. [10] 🟦 Softmax: e^x ↳ Raise e to the power of the number in each cell ↳ To simplify hand calculation, we approximate e^□ with 3^□. [11] 🟦 Softmax: ∑ ↳ Sum each row for 🟩 image→🟪 text ↳ Sum each column for 🟪 text→ 🟩 image [12] 🟦 Softmax: 1 / sum ↳ Divide each element by the column sum to obtain a similarity matrix for 🟪 text→🟩 image ↳ Divide each element by the row sum to obtain a similarity matrix for 🟩 image→🟪 text [13] 🟥 Loss Gradients ↳ The "Targets" for the similarity matrices are Identity Matrices. ↳ Why? If I and T come from the same pair (i=j), we want the highest value, which is 1, and 0 otherwise. ↳ Apply the simple equation of [Similarity - Target] to compute gradients of for both directions. ↳ Why so simple? Because when Softmax and Cross-Entropy Loss are used together, the math magically works out that way. ↳ These gradients kick off the backpropagation process to update weights and biases of the encoders and projection layers (red borders).

Tom Yeh

67,924 просмотров • 2 лет назад

ComfyUI Tutorial for Beginners: LoRAs, Style Transfer & ControlNets Module 3 is where you stop pulling the lever and start directing. Prompt iteration gets you close. These are the tools that get you exactly where you wanted to land. Style, structure, and pose, set before the sampler ever runs. What you'll learn: → How a LoRA attaches to a base model to change its output without retraining it → Why a LoRA has to match its base model family, and where to find them on Comfy Hub and Civitai → Model strength vs. CLIP strength — two dials, two different jobs → Stacking multiple LoRAs at different weights until the style is yours → Training your own: dataset, trigger word captions, step count, TrainLoRA locally → Reading the plot loss graph instead of guessing whether it worked → Appending to a LoRA you've already trained → Style transfer two ways: denoise-driven img2img vs. IP-Adapter reading style straight off a reference → ControlNets with DWPose, Depth Anything V2, and canny — composition locked before the prompt is read → Side by side: what each control map actually preserves → Fixing weak LoRAs, overcooked stacks, mismatched preprocessors, and noisy transfers Click the links below to run the workflow ⬇️
48:09

ComfyUI Tutorial for Beginners: LoRAs, Style Transfer & ControlNets Module 3 is where you stop pulling the lever and start directing. Prompt iteration gets you close. These are the tools that get you exactly where you wanted to land. Style, structure, and pose, set before the sampler ever runs. What you'll learn: → How a LoRA attaches to a base model to change its output without retraining it → Why a LoRA has to match its base model family, and where to find them on Comfy Hub and Civitai → Model strength vs. CLIP strength — two dials, two different jobs → Stacking multiple LoRAs at different weights until the style is yours → Training your own: dataset, trigger word captions, step count, TrainLoRA locally → Reading the plot loss graph instead of guessing whether it worked → Appending to a LoRA you've already trained → Style transfer two ways: denoise-driven img2img vs. IP-Adapter reading style straight off a reference → ControlNets with DWPose, Depth Anything V2, and canny — composition locked before the prompt is read → Side by side: what each control map actually preserves → Fixing weak LoRAs, overcooked stacks, mismatched preprocessors, and noisy transfers Click the links below to run the workflow ⬇️

ComfyUI

12,761 просмотров • 1 месяц назад

the most powerful image agent has arrived. flowith image agent doesn’t just make better images. It builds perfect visual workflows. - zoom-worthy details - text that actually works - search-aware creative intelligence - images to videos with seedance 2.0 from one prompt to a complete campaign workflow. like, comment, and retweet for 20 random membership drops!
0:25

the most powerful image agent has arrived. flowith image agent doesn’t just make better images. It builds perfect visual workflows. - zoom-worthy details - text that actually works - search-aware creative intelligence - images to videos with seedance 2.0 from one prompt to a complete campaign workflow. like, comment, and retweet for 20 random membership drops!

Flowith

24,817 просмотров • 5 месяцев назад