Загрузка видео...

Не удалось загрузить видео

На главную

Rewrote tiny-diffusion to be 3x smaller! Went from 951 lines to just 364, all contained in one file. As simple as possible, but not simpler. I also added a tiny GPT implementation as a comparison (312 lines, inspired by Andrej Karpathy). The two implementations are ~80% identical. The model...

nathanrs's profile picture

Nathan Barry

2,822 subscribers

161,578 просмотров • 7 месяцев назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”
1:02

Google isn’t betting on a single AI architecture. Sundar Pichai, CEO of Google: “We’re going to push the diffusion paradigm as hard as possible.” “All of today’s mainline Gemini models are autoregressive. Diffusion is a different paradigm.” “For the same capability, diffusion can be much faster.” “It’s behind the mainline models today, but there will be areas where it’s the right tool.” “We’re pushing multiple directions in parallel, and bringing them together where it makes sense.”

Forward Future

152,933 просмотров • 7 месяцев назад

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.
5:11

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 просмотров • 3 лет назад

How to create your Based Agent with 0 lines of code Recorded a quick demo video to show how easy it is and walk you through the entire process step-by-step All the links are in the comment below
5:07

How to create your Based Agent with 0 lines of code Recorded a quick demo video to show how easy it is and walk you through the entire process step-by-step All the links are in the comment below

lincoln.base.eth

148,939 просмотров • 1 год назад

tiny-diffusion, but Japanese! I wonder how logographic languages (Japanese, Chinese, etc) compare to phonetic/alphabetic languages in generation quality and speed with character-level tokenizers. The main difference is the semantic-value-per-token. Fewer tokens are needed to express an idea, which leads to fewer AR and diffusion steps. My main question is how would it affect the entropy in the output distributions. Lower entropy benefits parallel decoding. I could see arguments on both sides on how it affects it. One main benefit is that you have fewer opportunities to mangle words, leading to less obvious mistakes.
0:21

tiny-diffusion, but Japanese! I wonder how logographic languages (Japanese, Chinese, etc) compare to phonetic/alphabetic languages in generation quality and speed with character-level tokenizers. The main difference is the semantic-value-per-token. Fewer tokens are needed to express an idea, which leads to fewer AR and diffusion steps. My main question is how would it affect the entropy in the output distributions. Lower entropy benefits parallel decoding. I could see arguments on both sides on how it affects it. One main benefit is that you have fewer opportunities to mangle words, leading to less obvious mistakes.

nathan (in sf)

54,185 просмотров • 7 месяцев назад

What if writing code was as simple as talking to your device? Watch my GPT-4 voice assistant: - take in a complex coding task - write the code - create a PR on my GitHub repo All I had to do was tell it what to do. This is the future of software.
3:09

What if writing code was as simple as talking to your device? Watch my GPT-4 voice assistant: - take in a complex coding task - write the code - create a PR on my GitHub repo All I had to do was tell it what to do. This is the future of software.

Mckay Wrigley

1,347,845 просмотров • 3 лет назад

Andrej Karpathy: "To get the most out of the tools that have become available now, you have to remove yourself as the bottleneck. You cannot be there to prompt the next thing. You need to take yourself outside the loop. You have to arrange things such that they are completely autonomous. The more you can maximize your token throughput and not be in the loop, the better. This is the goal. So, I kind of mentioned that the name of the game now is to increase your leverage. I put in very few tokens just once in a while, and a huge amount of stuff happens on my behalf." --- From No Priors YT channel (link in comment)
0:48

Andrej Karpathy: "To get the most out of the tools that have become available now, you have to remove yourself as the bottleneck. You cannot be there to prompt the next thing. You need to take yourself outside the loop. You have to arrange things such that they are completely autonomous. The more you can maximize your token throughput and not be in the loop, the better. This is the goal. So, I kind of mentioned that the name of the game now is to increase your leverage. I put in very few tokens just once in a while, and a huge amount of stuff happens on my behalf." --- From No Priors YT channel (link in comment)

Rohan Paul

172,407 просмотров • 3 месяцев назад

Absolutely massive student, farmer and citizen protest against the corrupt regime today in Belgrade. The Serbian regime went to such lengths as to cancel train and bus lines to Belgrade from various towns in Serbia, but, as you can see in the video, this demonstration just might be one of the largest since the democratic revolution of 2000. Video by CRTA.
0:23

Absolutely massive student, farmer and citizen protest against the corrupt regime today in Belgrade. The Serbian regime went to such lengths as to cancel train and bus lines to Belgrade from various towns in Serbia, but, as you can see in the video, this demonstration just might be one of the largest since the democratic revolution of 2000. Video by CRTA.

Aleksandar Djokic (Александар Джокич)

159,467 просмотров • 1 год назад

You'd think that all the geoglyphs of the Nazca region have been documented by now, but that's not the case. There are dozens of geometric geoglyphs that don't resemble the anthropomorphic or zoomorphic figurative designs of the most famous figures, nor the trapezoidal shapes that are found throughout the region's desert. Here is an example of these geometric motifs, whose lines are often too fine to be seen from an airplane or on satellite images. This photogrammetric model and the overlay of lines was created by the talented Marián Marčiš from drone videos. I'll be going back to Nazca next month to film as many new ones as I can. Who do you think made them and why?
1:01

You'd think that all the geoglyphs of the Nazca region have been documented by now, but that's not the case. There are dozens of geometric geoglyphs that don't resemble the anthropomorphic or zoomorphic figurative designs of the most famous figures, nor the trapezoidal shapes that are found throughout the region's desert. Here is an example of these geometric motifs, whose lines are often too fine to be seen from an airplane or on satellite images. This photogrammetric model and the overlay of lines was created by the talented Marián Marčiš from drone videos. I'll be going back to Nazca next month to film as many new ones as I can. Who do you think made them and why?

Weird Old World

31,385 просмотров • 11 месяцев назад

This is MicroCAD. The programming language for your 3D projects. You can design your own Lego block in a few lines of code, ready to be exported as an STL file for 3D printing.
0:57

This is MicroCAD. The programming language for your 3D projects. You can design your own Lego block in a few lines of code, ready to be exported as an STL file for 3D printing.

Marco Franzon

113,663 просмотров • 9 месяцев назад

🇯🇵 While a lot of tourists love going to chain ramen shops and standing in long lines, smaller shops such as Ramen Chonmage in Osaka are quite amazing. I went here once before and I found the staff to be so friendly.
3:35

🇯🇵 While a lot of tourists love going to chain ramen shops and standing in long lines, smaller shops such as Ramen Chonmage in Osaka are quite amazing. I went here once before and I found the staff to be so friendly.

haruka no yume【はるかのゆめ】

64,376 просмотров • 2 месяцев назад

In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies, tools, and light to create images and scenes. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗠𝗲𝗱𝗶𝗮 + 𝗧𝗼𝗼𝗹𝘀 I experimented with clay, manipulating different types and colors along with a selection of prompts. I used a magnifying glass, tracked in real-time, to focus the diffusion process on specific areas. Combining these tools created a dynamic and inspiring experience. Using magic clay to layer shapes and colors as a base for revealing landscapes and hidden worlds, and the magnifying glass to focus and reveal these details, was particularly effective. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗟𝗶𝗴𝗵𝘁 I used light as my method of interaction with the img2img diffusion. This approach felt special right away. There was something magical about holding a physical light source and seeing it influence the generated visuals. I iterated on this technique with themes like Rococo architecture, flowers, Brutalist architecture, hidden worlds, and origami landscapes. 𝗜𝗻𝗸 + 𝗛𝘆𝗯𝗿𝗶𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘀 I also used ink in milk as a means of physical interaction with the diffusion pipeline. As I drop ink into milk, shapes come alive instantly. By learning to manipulate the combination of physical and digital elements, I steered the generated output toward my areas of interest. These iterations extended beyond ink in milk to include the format in which these elements are contained: a circular plate or a triptych of small stainless steel trays. These formats provide a structured yet flexible framework to explore themes and narratives across multiple visual spaces. It's magical. Some of that last iteration has been captured in this insightful article by Fast Company: #stablediffusion #realtime #ai
1:11

In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies, tools, and light to create images and scenes. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗠𝗲𝗱𝗶𝗮 + 𝗧𝗼𝗼𝗹𝘀 I experimented with clay, manipulating different types and colors along with a selection of prompts. I used a magnifying glass, tracked in real-time, to focus the diffusion process on specific areas. Combining these tools created a dynamic and inspiring experience. Using magic clay to layer shapes and colors as a base for revealing landscapes and hidden worlds, and the magnifying glass to focus and reveal these details, was particularly effective. 𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗟𝗶𝗴𝗵𝘁 I used light as my method of interaction with the img2img diffusion. This approach felt special right away. There was something magical about holding a physical light source and seeing it influence the generated visuals. I iterated on this technique with themes like Rococo architecture, flowers, Brutalist architecture, hidden worlds, and origami landscapes. 𝗜𝗻𝗸 + 𝗛𝘆𝗯𝗿𝗶𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘀 I also used ink in milk as a means of physical interaction with the diffusion pipeline. As I drop ink into milk, shapes come alive instantly. By learning to manipulate the combination of physical and digital elements, I steered the generated output toward my areas of interest. These iterations extended beyond ink in milk to include the format in which these elements are contained: a circular plate or a triptych of small stainless steel trays. These formats provide a structured yet flexible framework to explore themes and narratives across multiple visual spaces. It's magical. Some of that last iteration has been captured in this insightful article by Fast Company: #stablediffusion #realtime #ai

Hugues Bruyère

86,381 просмотров • 2 лет назад

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when you train the model to predict the next token you’re not just teaching it to guess the next word but how to best encode the human knowledge it sees. better compression means better abstraction means better reasoning at some point, compression stops looking like storage or a database (as some like to call it on X) and looks like an approximation of understanding.
0:58

3Blue1Brown’s new video explains why every LLM is actually a compression machine. everyone describes pre-training as “next token prediction” but that’s just the surface-level objective. in reality it is a means to making the most efficient text compressor. prediction and compression are two sides of the same coin. when you train the model to predict the next token you’re not just teaching it to guess the next word but how to best encode the human knowledge it sees. better compression means better abstraction means better reasoning at some point, compression stops looking like storage or a database (as some like to call it on X) and looks like an approximation of understanding.

ℏεsam

120,081 просмотров • 2 месяцев назад

The Chainlink CCIP SDK is now live and ready to turbocharge cross-chain development: ⬡ Integrate CCIP in just a few lines of code ⬡ Use developer tools you already know ⬡ Leverage Chainlink’s defense-in-depth security The software development kit streamlines cross-chain development by allowing developers to use JavaScript to integrate Chainlink CCIP and create a token transfer frontend dApp with just a few lines of code. Developers are empowered to build with tools they already know and use daily, as the CCIP SDK UI components are built on top of the leading Web3 development framework Wagmi. This provides easy access to the standard for cross-chain interoperability, enabling devs to build secure solutions underpinned by CCIP’s level-5 security. Read the docs:
0:37

The Chainlink CCIP SDK is now live and ready to turbocharge cross-chain development: ⬡ Integrate CCIP in just a few lines of code ⬡ Use developer tools you already know ⬡ Leverage Chainlink’s defense-in-depth security The software development kit streamlines cross-chain development by allowing developers to use JavaScript to integrate Chainlink CCIP and create a token transfer frontend dApp with just a few lines of code. Developers are empowered to build with tools they already know and use daily, as the CCIP SDK UI components are built on top of the leading Web3 development framework Wagmi. This provides easy access to the standard for cross-chain interoperability, enabling devs to build secure solutions underpinned by CCIP’s level-5 security. Read the docs:

Chainlink

78,596 просмотров • 1 год назад

Grid Lines, drawn correctly in place based on the location on Earth (independent of map files). Also, GPS, the location here is spoofed as I don't live here (obviously), but the code is there for it to be live with one simple change. Location: Detroit, MI
0:48

Grid Lines, drawn correctly in place based on the location on Earth (independent of map files). Also, GPS, the location here is spoofed as I don't live here (obviously), but the code is there for it to be live with one simple change. Location: Detroit, MI

Shimazu.S

13,756 просмотров • 6 месяцев назад

I can’t describe how long the lines are to get into Trump’s final rally before he’s sworn in as 47. It’s incredible. People are traveling from all over the US—and the world—and waiting in line in the snow for a chance to see the rally and President Trump. Will the MSM show it?
0:45

I can’t describe how long the lines are to get into Trump’s final rally before he’s sworn in as 47. It’s incredible. People are traveling from all over the US—and the world—and waiting in line in the snow for a chance to see the rally and President Trump. Will the MSM show it?

Julia 🇺🇸

1,392,177 просмотров • 1 год назад

What if you could visualise how electricity flows from a wind farm in Scotland to your home in London? Ok it's not quite that simple, but it got me thinking that there must be a way of visualising the complexity of the electricity transmission grid in Great Britain. This is my first attempt… What you're seeing are the real power lines and substations on the transmission grid using OpenStreetMap data. The real connections between each substation are found using PyPSA-Eur as well as calculating the "electrical distance" or impedance between each substation. This is my incredibly basic and naive way of attempting to show the "speed" at which the energy is moving through the network, where the bottlenecks are, etc. I know it's not as simple as this, but it's a first attempt so… 😉 I've also added a way to pick any substation in the network and show how electricity pulses through the rest of the network from that point. This is the magic sauce that really brings to life how much infrastructure electricity has to pass through to get from point A to point B. Of course this isn't really what happens, and there's no demand to soak up the energy. However I think it still serves as an interesting visual of how complex the network actually is and the distances involved to get energy from one side of the country to the other. What do you think? Is this worth exploring further?
0:29

What if you could visualise how electricity flows from a wind farm in Scotland to your home in London? Ok it's not quite that simple, but it got me thinking that there must be a way of visualising the complexity of the electricity transmission grid in Great Britain. This is my first attempt… What you're seeing are the real power lines and substations on the transmission grid using OpenStreetMap data. The real connections between each substation are found using PyPSA-Eur as well as calculating the "electrical distance" or impedance between each substation. This is my incredibly basic and naive way of attempting to show the "speed" at which the energy is moving through the network, where the bottlenecks are, etc. I know it's not as simple as this, but it's a first attempt so… 😉 I've also added a way to pick any substation in the network and show how electricity pulses through the rest of the network from that point. This is the magic sauce that really brings to life how much infrastructure electricity has to pass through to get from point A to point B. Of course this isn't really what happens, and there's no demand to soak up the energy. However I think it still serves as an interesting visual of how complex the network actually is and the distances involved to get energy from one side of the country to the other. What do you think? Is this worth exploring further?

Robin Hawkes

144,544 просмотров • 6 месяцев назад

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.
4:35

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,211 просмотров • 1 год назад