Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

The code for σ-GPT is out! Feel free to experiment with it here: It extends the François Fleuret picoclvr (non-nlp tasks) and Andrej Karpathy nanoGPT (text) codebases, with an additional KV-Cache from IRIS by Vincent Micheli and Eloi Alonso. (links below)

Arnaud Pannatier

1,096 subscribers

12,611 views • 2 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

5 Comments

Arnaud Pannatier2 years ago

picoclvr: nanoGPT: IRIS (for KV-Cache, and go check the whole project it's amazing): σ-GPT:

Lab4crypto1 year ago

🚀 Don't gamble with your portfolio! Use our advanced hybrid quant risk tool using on/off-chain data and make informed decisions. 📈 Acess to 1000+ charts for your crypto journey. 📚Join our Premium Telegram for daily alerts. 📊+21 projects supported. 🏗️ Beginners and experts.

k_zer0s2 years ago

@francoisfleuret @karpathy @micheli_vincent @EloiAlonso1 👍

Chris Krempel2 years ago

@francoisfleuret @karpathy @micheli_vincent @EloiAlonso1 very! interesting. Is the only thing I need to adjust `def order` for a new learning/ randomization scheme? Let's say I want to gradually move from masking random cells in a table to masking out a whole column?!

Arnaud Pannatier2 years ago

@francoisfleuret @karpathy @micheli_vincent @EloiAlonso1 Yeah changing will let you use any kind of ordering. But if you want to be able to change on the fly at generation time, you need to keep a random order during training. But σ-GPT does not use masking like BERT it is causal like a standard GPT.

Related Videos

Hello GPT-5.5! Our best model yet is here. It’s smarter, faster, and gets more done. I had the chance to capture first reactions from developers building with it: it writes better code, understands more complex tasks, and keeps going. Can’t wait to see what you build with it!

Hello GPT-5.5! Our best model yet is here. It’s smarter, faster, and gets more done. I had the chance to capture first reactions from developers building with it: it writes better code, understands more complex tasks, and keeps going. Can’t wait to see what you build with it!

Romain Huet

16,275 views • 4 months ago

BREAKING: ChatGPT GPT-4o was just announce by OpenAI. It improves on vision, audio and text. The ease of use is incredibly enhanced. It makes interaction with the GPT much more natural, especially with voice. GPT-4o reasons across voice, text and vision. GPT-4 wil be available to everyone.

BREAKING: ChatGPT GPT-4o was just announce by OpenAI. It improves on vision, audio and text. The ease of use is incredibly enhanced. It makes interaction with the GPT much more natural, especially with voice. GPT-4o reasons across voice, text and vision. GPT-4 wil be available to everyone.

Ed Krassenstein

21,605 views • 2 years ago

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

I added KV caching and INT8 KV quantization to our transformer inference, improving throughput by 35x. All of this was done from scratch in Rust + CUDA, on top of a homemade ML framework. On a 4-token prompt with 252 generated tokens: - Original: 0.76 tok/s - KV cache fp32: 27.21 tok/s - KV cache int8 (quantized): 27.29 tok/s Try it out yourself here: In practice: - KV caching gave us about a 35x end-to-end speedup - INT8 KV cache kept roughly the same speed as fp32 but cut KV cache memory by 3.78x FP32 cache used 4.5 MB in this run while the INT8 cache used only 1.19 MB This simple change to inference created a huge impact on performance. To learn more about the KV cache and other optimizations like this, check out the blog at

Reese Chong

53,026 views • 4 months ago

The Magic Starts here... Introducing Dynamic Apps in Nimo . Create beautiful Personal Dynamic Apps that connect with your data and tools like Gmail, Sheets, Notion, Slack, and more. Check out some stunning Dynamic Apps below. Now with GPT-5, creating fast, reliable, and stunning Dynamic Apps is easier than ever. Check out the threads below to get an invitation code and try it today.

The Magic Starts here... Introducing Dynamic Apps in Nimo . Create beautiful Personal Dynamic Apps that connect with your data and tools like Gmail, Sheets, Notion, Slack, and more. Check out some stunning Dynamic Apps below. Now with GPT-5, creating fast, reliable, and stunning Dynamic Apps is easier than ever. Check out the threads below to get an invitation code and try it today.

rohildev

72,833 views • 1 year ago

1/ Introducing Glider - the smallest model to beat GPT-4o-mini on eval tasks ⚡🚀 - Open source, open weights, open code - Explainable evaluations by nature - Trained on 183 criteria and 685 domains Try it out for free at 🔥

1/ Introducing Glider - the smallest model to beat GPT-4o-mini on eval tasks ⚡🚀 - Open source, open weights, open code - Explainable evaluations by nature - Trained on 183 criteria and 685 domains Try it out for free at 🔥

PatronusAI

14,856 views • 1 year ago

The new Andrej Karpathy podcast with No Priors is a must-watch. I turned the most surprising insights & anecdotes into a webpage. For each, you can easily jump to the timetamp in the video where it occurred. Made with AnyGen (the most underrated AI tool out there) Link below!

The new Andrej Karpathy podcast with No Priors is a must-watch. I turned the most surprising insights & anecdotes into a webpage. For each, you can easily jump to the timetamp in the video where it occurred. Made with AnyGen (the most underrated AI tool out there) Link below!

Zara Zhang

29,505 views • 5 months ago

Exploring Andrej Karpathy Nanochat through a Knowledge Graph Love how this project combines simplicity with power. Want to navigate and understand the entire repo? Check it out here: Plus, you can connect it via MCP to any AI coding assistant and work directly on the repository. Powered by Deep Graph MCP AI at Meta and Groq Inc

Exploring Andrej Karpathy Nanochat through a Knowledge Graph Love how this project combines simplicity with power. Want to navigate and understand the entire repo? Check it out here: Plus, you can connect it via MCP to any AI coding assistant and work directly on the repository. Powered by Deep Graph MCP AI at Meta and Groq Inc

Daniel San

126,642 views • 10 months ago

Operator is an AI agent by ChatGPT that browses the web and performs tasks. Currently in research preview with restrictions, it evolves with feedback. Check out the use cases below: 1. Crypto investment research based on tokens:

Operator is an AI agent by ChatGPT that browses the web and performs tasks. Currently in research preview with restrictions, it evolves with feedback. Check out the use cases below: 1. Crypto investment research based on tokens:

Shruti

205,754 views • 1 year ago

This beautiful presentation is by Muse 1.2. It used gpt-image-v2 to create the images by itself. I told it to make a history page from Llama to Muse 1.2, and it did a great job. Alexandr Wang , you guys created magic with agentic tasks. I'm now waiting for the big model from you guys.

This beautiful presentation is by Muse 1.2. It used gpt-image-v2 to create the images by itself. I told it to make a history page from Llama to Muse 1.2, and it did a great job. Alexandr Wang , you guys created magic with agentic tasks. I'm now waiting for the big model from you guys.

Chetaslua

17,158 views • 29 days ago

Stylized Experiment Version 3 is here! With lots of fixes, new additions like face tracking, and future proofing for new cosmetics I plan to make! Character based off the species in Casualties: Unknown / Scav Prototype. Check out the game created by Orsoniks! Links below

Stylized Experiment Version 3 is here! With lots of fixes, new additions like face tracking, and future proofing for new cosmetics I plan to make! Character based off the species in Casualties: Unknown / Scav Prototype. Check out the game created by Orsoniks! Links below

akittu

143,427 views • 1 month ago

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

GPT-5.2-high by OpenAI is off to a strong start in the Code Arena. ⚡️ If you’re new here: Code Arena is where AI models build full web apps, tools, and interactive sites — all from a single prompt. Watch the video to see GPT-5.2-high in action, then try your own prompt and reply with your creation below. ⬇️

Arena.ai

246,430 views • 8 months ago

OpenAI just announced "GPT-4o". It can reason with voice, vision, and text. The model is 2x faster, 50% cheaper, and has 5x higher rate limit than GPT-4 Turbo. It will be available for free users and via the API. The voice model can even pick up on emotion and generate emotive voice.

OpenAI just announced "GPT-4o". It can reason with voice, vision, and text. The model is 2x faster, 50% cheaper, and has 5x higher rate limit than GPT-4 Turbo. It will be available for free users and via the API. The voice model can even pick up on emotion and generate emotive voice.

Lior Alexander

485,137 views • 2 years ago

🔥🔥 Tonkeeper has launched Signer, a revolutionary blockchain security product. The Signer app introduces an additional security layer for non-custodial wallets, enabling the secure storage and usage of private keys, which is particularly beneficial for users with larger crypto balances. The app seamlessly integrates with Tonkeeper and is now available for both iOS and Android. Check it out here:

🔥🔥 Tonkeeper has launched Signer, a revolutionary blockchain security product. The Signer app introduces an additional security layer for non-custodial wallets, enabling the secure storage and usage of private keys, which is particularly beneficial for users with larger crypto balances. The app seamlessly integrates with Tonkeeper and is now available for both iOS and Android. Check it out here:

Tonkeeper

84,066 views • 2 years ago

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

AutoGPT might be the next big step in AI. Here's why Karpathy recently said "AutoGPT is the next frontier of prompt engineering" AutoGPT is the equivalent of giving GPT-based models a memory and a body. You can now give a task to an AI agent and have it autonomously come up with a plan, execute on it, browse the web, and use new data to revise the strategy until the task is completed. It can analyze the market and come up with a trading strategy, customer service, marketing, finance, or other tasks that requires continuous updates. There are three components to it: 1. Architecture: It leverages GPT-4 and GPT-3.5 via API. 2. Autonomous Iterations: AutoGPT can refine its outputs by self-critical review, building on its previous work and integrating prompt history for more accurate results. 3. Memory Management: Integration with Pinecone allows for long-term memory storage, enabling context preservation and improved decision-making. 4. Multi-functionality: Capabilities include file manipulation, web browsing, and data retrieval, distinguishing AutoGPT from previous AI advancements by broadening its application scope.

Lior Alexander

808,131 views • 3 years ago

It was a ride to have it finished, but the SLIDE REEL IS HERE, and ummm... quite with something more actually because I made for it a whole stand as the one we insert it into, with the lantern inside of it! 🔥 ⬇️Photos and additional info below! Annapurna Interactive Mobius Digital Games #OuterWilds

It was a ride to have it finished, but the SLIDE REEL IS HERE, and ummm... quite with something more actually because I made for it a whole stand as the one we insert it into, with the lantern inside of it! 🔥 ⬇️Photos and additional info below! Annapurna Interactive Mobius Digital Games #OuterWilds

Mallow 🌈

171,816 views • 2 years ago

How to create your Based Agent with 0 lines of code Recorded a quick demo video to show how easy it is and walk you through the entire process step-by-step All the links are in the comment below

How to create your Based Agent with 0 lines of code Recorded a quick demo video to show how easy it is and walk you through the entire process step-by-step All the links are in the comment below

lincoln.base.eth

148,939 views • 1 year ago

Somebody hooked up Claude Code to pair program with o3 and Gemini 2.5 via MCP for an all-star coding lineup 🤯 - Claude controls the work - Can call out to Gemini and o3 for tasks and input - Works around MCP limit by using prompt files Check out Zen MCP (link below)

Somebody hooked up Claude Code to pair program with o3 and Gemini 2.5 via MCP for an all-star coding lineup 🤯 - Claude controls the work - Can call out to Gemini and o3 for tasks and input - Works around MCP limit by using prompt files Check out Zen MCP (link below)

Ian Nuttall

216,612 views • 1 year ago

here's a hands on guide to setup multi-agent autoresearch by Andrej Karpathy. uses open models. works with codex, claude, open code. - uses 5 agents each with a configuration, specific tools, roles, and permission (see repo) - a researcher agent searches papers on hf papers and creates hypotheses - a planner agent maintains an experiment plan and log - workers take the hypotheses and updates the scripts, then it starts a hf job with a gpu to run the script - a reporter agent monitors these jobs and reports events and metrics to a TrackioApp dashboard I ran this for 4 hours and the agents ran 32 jobs, they improved on the baseline by a small margin. check out everything I learnt in thread.

here's a hands on guide to setup multi-agent autoresearch by Andrej Karpathy. uses open models. works with codex, claude, open code. - uses 5 agents each with a configuration, specific tools, roles, and permission (see repo) - a researcher agent searches papers on hf papers and creates hypotheses - a planner agent maintains an experiment plan and log - workers take the hypotheses and updates the scripts, then it starts a hf job with a gpu to run the script - a reporter agent monitors these jobs and reports events and metrics to a TrackioApp dashboard I ran this for 4 hours and the agents ran 32 jobs, they improved on the baseline by a small margin. check out everything I learnt in thread.

Ben Burtenshaw

90,760 views • 4 months ago