正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it. And AI Models actually do learn during the context window, without changing the weights." ~ Anthropic CEO Dario Amodei (On the 2nd... show more

Rohan Paul

152,728 subscribers

631,729 次观看 • 10 个月前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

Google keeps sharing a ton of free content! Here is one of their most complete examples: a long-running AI agent that pauses, resumes, and never loses context. It simulates the onboarding of a new employee. There are the 3 architectural patterns you'll learn from this:

Google keeps sharing a ton of free content! Here is one of their most complete examples: a long-running AI agent that pauses, resumes, and never loses context. It simulates the onboarding of a new employee. There are the 3 architectural patterns you'll learn from this:

Santiago

20,040 次观看 • 2 个月前

in 9 minutes on the Tokyo stage, Angela Jang, head of product for the Claude platform, said the part most builders still haven't figured out: "a model is only as good as the context that you actually give it." that's the whole game now. not the prompt. the context. what your agent remembers. what skills it can pull. what it learns from its own past runs. anthropic now ships all three: memory, skills, and dreaming. agents that inspect their own trajectories and self-improve. demoed live this week. the prompt was never the bottleneck. the context was. most people are still tweaking words in a chat box. the ones who get this engineer the context their agents run on. watch the 9 minutes. then read the full breakdown below ⬇️

in 9 minutes on the Tokyo stage, Angela Jang, head of product for the Claude platform, said the part most builders still haven't figured out: "a model is only as good as the context that you actually give it." that's the whole game now. not the prompt. the context. what your agent remembers. what skills it can pull. what it learns from its own past runs. anthropic now ships all three: memory, skills, and dreaming. agents that inspect their own trajectories and self-improve. demoed live this week. the prompt was never the bottleneck. the context was. most people are still tweaking words in a chat box. the ones who get this engineer the context their agents run on. watch the 9 minutes. then read the full breakdown below ⬇️

Avid

34,516 次观看 • 2 个月前

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

New short course: LLMs as Operating Systems: Agent Memory, created with Letta, and taught by its founders Charles Packer and Sarah Wooders. An LLM's input context window has limited space. Using a longer input context also costs more and results in slower processing. So, managing what's stored in this context window is important. In the innovative paper MemGPT: Towards LLMs as Operating Systems, its authors (which include the instructors) proposed using an LLM agent to manage this context window. Their system uses a large persistent memory that stores everything that could be included in the input context, and an agent decides what is actually included. Take the example of building a chatbot that needs to remember what's been said earlier in a conversation (perhaps over many days of interaction with a user). As the conversation's length grows, the memory management agent will move information from the input context to a persistent searchable database; summarize information to keep relevant facts in the input context; and restore relevant conversation elements from further back in time. This allows a chatbot to keep what's currently most relevant in its input context memory to generate the next response. When I read the original MemGPT paper, I thought it was an innovative technique for handling memory for LLMs. The open-source Letta framework, which we'll use in this course, makes MemGPT easy to implement. It adds memory to your LLM agents and gives them transparent long-term memory. In detail, you’ll learn: - How to build an agent that can edit its own limited input context memory, using tools and multi-step reasoning - What is a memory hierarchy (an idea from computer operating systems, which use a cache to speed up memory access), and how these ideas apply to managing the LLM input context (where the input context window is a "cache" storing the most relevant information; and an agent decides what to move in and out of this to/from a larger persistent storage system) - How to implement multi-agent collaboration by letting different agents share blocks of memory This course will give you a sophisticated understanding of memory management for LLMs, which is important for chatbots having long conversations, and for complex agentic workflows. Please sign up here!

Andrew Ng

200,950 次观看 • 1 年前

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Introducing Claude Code Hook - Context Timeline (Saving this to try later) Install with: npx claude-code-templates@latest --hook monitoring/context-timeline Managing the context window and the subagents running in Claude Code is hard to keep track of That's why I built this hook... It starts the moment you open a session and shows a timeline with the main agent's context window and how subagents start working in their own separate context Every subagent you have running will show up in real time This way you can manage the context and the subagents you run, and see everything in a much simpler way than in the console

Daniel San

51,706 次观看 • 3 个月前

context engineering vs graph engineering. every few months the list gets a new word and everyone treats it as a replacement for the last one. these two are not on the same list. one decides what the model sees this turn, the other decides what exists at all. the cleanest way to tell them apart is to ask what a single unit of work looks like. > context engineering is the window the window opens empty, every single time. you assemble what goes in it. the prompt, the docs, the history, the tool results. the assembling is the work. the window only grows. it never shrinks on its own, so eventually something gets dropped. usually from the middle. usually without telling you. then the turn ends and the window is thrown away. not archived, thrown away. the next turn opens empty again and you re-explain what you already explained. good context engineering is knowing what to leave out, not what to pack in. the unit of work is one window. > graph engineering is the structure the same material arrives from the same sources. instead of packing it into a window, you pull entities out of it, resolve the duplicates into one node, and write typed edges between them. nothing here is stored as text you hope to find again. it is stored as a thing with a name and its connections to other things. when the turn ends, the graph is still there. the next turn does not start from zero. it starts by querying what already exists, and the query walks edges instead of guessing at similarity. good graph engineering is deciding what counts as the same thing twice. the unit of work is one relationship. > they are not alternatives the graph is what refills the window. context engineering decides what fits. graph engineering decides what there is to choose from. remove the graph and every session starts blind. remove the context work and the best structure in the world arrives as an unreadable dump. that also tells you which one broke. the answer drifted from what you actually said, or forgot something from this same session. that is the window. the answer is coherent but invents a connection that does not exist, or cannot join two facts it has clearly seen. that is the structure. people debug the prompt because the prompt is the easiest thing to edit. it keeps taking the blame for failures that live a layer down. save this - then read the full breakdown below

context engineering vs graph engineering. every few months the list gets a new word and everyone treats it as a replacement for the last one. these two are not on the same list. one decides what the model sees this turn, the other decides what exists at all. the cleanest way to tell them apart is to ask what a single unit of work looks like. > context engineering is the window the window opens empty, every single time. you assemble what goes in it. the prompt, the docs, the history, the tool results. the assembling is the work. the window only grows. it never shrinks on its own, so eventually something gets dropped. usually from the middle. usually without telling you. then the turn ends and the window is thrown away. not archived, thrown away. the next turn opens empty again and you re-explain what you already explained. good context engineering is knowing what to leave out, not what to pack in. the unit of work is one window. > graph engineering is the structure the same material arrives from the same sources. instead of packing it into a window, you pull entities out of it, resolve the duplicates into one node, and write typed edges between them. nothing here is stored as text you hope to find again. it is stored as a thing with a name and its connections to other things. when the turn ends, the graph is still there. the next turn does not start from zero. it starts by querying what already exists, and the query walks edges instead of guessing at similarity. good graph engineering is deciding what counts as the same thing twice. the unit of work is one relationship. > they are not alternatives the graph is what refills the window. context engineering decides what fits. graph engineering decides what there is to choose from. remove the graph and every session starts blind. remove the context work and the best structure in the world arrives as an unreadable dump. that also tells you which one broke. the answer drifted from what you actually said, or forgot something from this same session. that is the window. the answer is coherent but invents a connection that does not exist, or cannot join two facts it has clearly seen. that is the structure. people debug the prompt because the prompt is the easiest thing to edit. it keeps taking the blame for failures that live a layer down. save this - then read the full breakdown below

Hanako

19,160 次观看 • 24 天前

Dario Amodei directly argues that the US Department of War is misrepresenting the law to claim power that the government simply does not have. CBS News asked "Do you think Anthropic can survive this as a business?" Dario effectively says that the Department of War is using social media to trick the public into believing they have powers that Congress never actually gave them. Dario Amadoui confidently states that the impact of the "supply chain risk" designation is actually very limited and will not stop their success, and that Anthropic will be just fine. ---- Video from 'CBS News + CBS Evening News + CBS Mornings' YT channel (link in comment)

Dario Amodei directly argues that the US Department of War is misrepresenting the law to claim power that the government simply does not have. CBS News asked "Do you think Anthropic can survive this as a business?" Dario effectively says that the Department of War is using social media to trick the public into believing they have powers that Congress never actually gave them. Dario Amadoui confidently states that the impact of the "supply chain risk" designation is actually very limited and will not stop their success, and that Anthropic will be just fine. ---- Video from 'CBS News + CBS Evening News + CBS Mornings' YT channel (link in comment)

Rohan Paul

30,664 次观看 • 5 个月前

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

JUST IN: Google releases Gemini 1.5, a powerful MoE model. It's a huge breakthrough. The model has the longest context window ever seen: 1 million tokens. It can process 1 hour of video, 11 hours of audio, 30,000 lines of code, or 700,000 words in a single prompt. When tested on text, code, image, audio and video evaluations, 1.5 Pro outperforms 1.0 Pro on 87% of the benchmarks used for developing LLMs. You can can sign up in AI Studio to try it out.

Lior Alexander

83,432 次观看 • 2 年前

$Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.$

Introducing SubQ - a major breakthrough in LLM intelligence. It is the first model built on a fully sub-quadratic sparse-attention architecture (SSA), And the first frontier model with a 12 million token context window which is: - 52x faster than FlashAttention at 1MM tokens - Less than 5% the cost of Opus Transformer-based LLMs waste compute by processing every possible relationship between words (standard attention). Only a small fraction actually matter. Subquadratic finds and focuses only on the ones that do. That's nearly 1,000x less compute and a new way for LLMs to scale.

Alexander Whedon

13,442,792 次观看 • 3 个月前

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

We asked James Wang why Cerebras can run large models so quickly. His answer: Inference speed is mostly a memory problem. Instead of constantly pulling weights from external memory, Cerebras splits the model across multiple wafers and pipelines the layers together. “Inference is all about memory bandwidth.” “You don’t want to store weights external to the wafer, because the second it’s outside, it becomes much slower.” “We put the weights on the inside.” “The team built a new software stack that basically lets us split the models by layer and store them layer by layer across multiple wafers.” “That allows us to never have to read from external memory, and that’s what makes it so fast.” “For context, Claude is about 100 tokens per second.”

MTS

27,935 次观看 • 3 个月前

Google Engineer: "I never think about a context window anymore" tasks run 36 hours. the window empties and refills the whole time, and he designed for that nothing lives in the prompt. the rules hit through tool outputs, exactly when needed "we want to make every mistake something that is just not possible" bookmark this, then read the article below ↓

Google Engineer: "I never think about a context window anymore" tasks run 36 hours. the window empties and refills the whole time, and he designed for that nothing lives in the prompt. the rules hit through tool outputs, exactly when needed "we want to make every mistake something that is just not possible" bookmark this, then read the article below ↓

vartekx

82,917 次观看 • 6 天前

Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

Dario Amodei just dismantled the biggest myth in the AI industry. Open source AI isn’t free. It never was. Amodei: “It’s not free. You have to run it on inference and someone has to make it fast on inference.” For decades, open source meant something real. It meant a teenager in a basement could download the same tools as a Fortune 500 company. Could read the code. Could modify it. Could build something that competed with the giants. That was genuine democratization. That actually happened. AI is different. Fundamentally. Physically. In ways the ideology hasn’t caught up to yet. Downloading the weights is the easy part. The part that actually costs something is turning the weights into a running system. Into responses. Into intelligence operating in real time at scale. That requires compute. Power. Infrastructure. The kind measured in billions of dollars and years of construction. Amodei: “These are big models. They’re hard to do inference on. Ultimately you have to host it on the cloud. The people who host it on the cloud do inference.” The open source debate was never about who owns the model. It was always about who owns the cloud. And Amodei goes further. When a competitor drops a new open model, he doesn’t ask whether it’s open or closed. He doesn’t care about the licensing. He doesn’t engage the ideology. Amodei: “I don’t think it mattered that DeepSeek is open source. I think I ask, is it a good model? Is it better than us at the things that matter? That’s the only thing that I care about.” That’s the ruthless clarity of someone actually trying to win. While the media debates licensing frameworks, Amodei is asking one question. Is it better. Everything else is a distraction. Amodei: “I don’t think open source works the same way in AI that it has worked in other areas. Here we can’t see inside the model.” This isn’t Linux. You can’t read it. You can’t fork it. You can’t understand it the way generations of developers understood the tools they inherited. You can download it. And then you need a data center to run it. The teenager in the basement who was supposed to be empowered by this revolution needs a billion dollars of infrastructure before the empowerment starts. The era of the basement coder rewriting civilization on a laptop is over. The future belongs to whoever commands the compute, owns the power grid, and can actually turn the intelligence on. Open weights without infrastructure isn’t democratization. It’s a promise the physics of the universe won’t let us keep.

Dustin

687,563 次观看 • 5 个月前

MiniMax H3 is native in ComfyUI, day zero with the weights. Model highlights: → Text-to-video, prompt only → Image-to-video → First-and-last-frame - control the opening frame, the closing frame, or both → Reference-to-video - carry a subject, a motion, or a voice through the clip → In-place editing - modify a shot you already have instead of regenerating it The capability MiniMax (official) leads with is multimodal context understanding, and it's what collapses those five tasks into one model. H3 takes images, audio, and video together and resolves them against a prompt that explains how they relate. Download the workflows & learn more in our blog 👇

MiniMax H3 is native in ComfyUI, day zero with the weights. Model highlights: → Text-to-video, prompt only → Image-to-video → First-and-last-frame - control the opening frame, the closing frame, or both → Reference-to-video - carry a subject, a motion, or a voice through the clip → In-place editing - modify a shot you already have instead of regenerating it The capability MiniMax (official) leads with is multimodal context understanding, and it's what collapses those five tasks into one model. H3 takes images, audio, and video together and resolves them against a prompt that explains how they relate. Download the workflows & learn more in our blog 👇

ComfyUI

232,649 次观看 • 20 天前

you're running /compact every 10 minutes to fix a problem that disappears the second you give Claude one job instead of five. Anthropic shipped a clean 4-minute video on context management. /compact when the window fills up. /clear to reset. small files over big files. the practical tips work. the video covers the cleanup. it doesn't cover why your window keeps filling up in the first place, and the answer is usually that you're holding too many jobs in one prompt. look at what most people put in a single Claude call: > guess the right answer > choose between options > set the position size > watch the exit window > read the news for sentiment five tools' worth of state in one conversation. of course the window fills. i ran into this on a Polymarket bot: the prediction-layer version ate context constantly. /compact every 8-10 turns. i thought i needed better memory hygiene. i needed fewer jobs per call. i stripped the prompt down to one job: veto an entry i had already decided to take. the context window stopped filling. /compact went from every 8 turns to once a session, if ever. over 90 days that prompt blocked 128 trades. watch the 4 min, then ask why your window keeps needing it. clean context. lean prompt. nothing to compact.

you're running /compact every 10 minutes to fix a problem that disappears the second you give Claude one job instead of five. Anthropic shipped a clean 4-minute video on context management. /compact when the window fills up. /clear to reset. small files over big files. the practical tips work. the video covers the cleanup. it doesn't cover why your window keeps filling up in the first place, and the answer is usually that you're holding too many jobs in one prompt. look at what most people put in a single Claude call: > guess the right answer > choose between options > set the position size > watch the exit window > read the news for sentiment five tools' worth of state in one conversation. of course the window fills. i ran into this on a Polymarket bot: the prediction-layer version ate context constantly. /compact every 8-10 turns. i thought i needed better memory hygiene. i needed fewer jobs per call. i stripped the prompt down to one job: veto an entry i had already decided to take. the context window stopped filling. /compact went from every 8 turns to once a session, if ever. over 90 days that prompt blocked 128 trades. watch the 4 min, then ask why your window keeps needing it. clean context. lean prompt. nothing to compact.

Mnimiy

20,635 次观看 • 3 个月前

In software engineering we have so crossed the line. This is from Claude Code creator Boris Cherny (Boris Cherny ) "I don't write code; I prompt Claude. Actually, nowadays, what I'm mostly doing is having one Claude prompt other Claudes. So I don't even talk to Claude directly. I have a Claude that's talking to my other Claudes. I think in engineering, we've seen an explosion in the amount of leverage that a single person has. It's about how big of a business one person can build and how many products one person can support." ---- From "Alex Kantrowitz" (Alex Kantrowitz) YouTube channel, (full video link in comment)

In software engineering we have so crossed the line. This is from Claude Code creator Boris Cherny (Boris Cherny ) "I don't write code; I prompt Claude. Actually, nowadays, what I'm mostly doing is having one Claude prompt other Claudes. So I don't even talk to Claude directly. I have a Claude that's talking to my other Claudes. I think in engineering, we've seen an explosion in the amount of leverage that a single person has. It's about how big of a business one person can build and how many products one person can support." ---- From "Alex Kantrowitz" (Alex Kantrowitz) YouTube channel, (full video link in comment)

Rohan Paul

22,852 次观看 • 1 个月前

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

🚨David Friedberg: AI is starting to identify and solve problems on its own “I'll give you a science corner example: there's this Evo 2 model that they publish at the Arc Institute, which Patrick Collison, you know, is the main funder and chairman.” “So that Evo 2 model, they just ingested all the DNA data they could find in the world.” “Trillions and trillions of base paired data that they ingested and then they looked at patterns in DNA. And that's it.” “They had no context for what the DNA represented, they had no context for the concept of genes, none of the structured understanding of what that DNA does, what it is, and you know what it did?” “They fed in the BRCA gene variant and the thing output a warning saying, ‘I think that this is a pathogenic variant to DNA,’ without having any context.” “This is the breast cancer allele.” “And it didn't have any knowledge and it wasn't trained on that at all.” “It had no knowledge that there are pathogenic variants for cancer, and it identified that this was a genetic variant that can cause some sort of pathogenic outcome in the organism.” “That's a great example where there's a lack of understanding at the human level on what really drives some of the patterns in nature, the patterns in society, the patterns in behavior that are kind of emergent phenomena perhaps, that these AI models are starting to identify.”

The All-In Podcast

79,718 次观看 • 1 年前

Gemini-1.5 Pro has its spotlight stolen today, and people are poking fun at Sora vs Google memes. Well, I think it's the biggest boost in LLM capability so far in 2024. v1.5's 10M token context (1) excels at retrieval; (2) generalizes zero-shot to extremely long instructions like full tutorials and codebases; and (3) works across modalities such as text, audio, and video. Here's a stunning example: v1.5 learns to translate from English to Kalamang purely in context, following a full linguistic manual at inference time. Kalamang is a language spoken by fewer than 200 speakers in western New Guinea. Gemini has never seen this language during training and is only provided with 500 pages of linguistic documentation, a dictionary, and ~400 parallel sentences in context. It basically acquires a sophisticated new skill in the neural activations, instead of gradient finetuning. I talked about the Myth of Context Length many times before: don't get too excited by claims of 1M or even 1B context tokens. LSTMs already achieved literally infinite context length 25 yrs ago! What truly matters is how well the model actually uses the context to solve real-world problems, and Gemini-1.5 has surpassed the SOTA with flying colors. The paper is also well-written with lots of solid quantitative analysis on in-context memorization and generalization. Paper: “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context” Congrats to Jeff Dean Oriol Vinyals Sundar Pichai and team!

Gemini-1.5 Pro has its spotlight stolen today, and people are poking fun at Sora vs Google memes. Well, I think it's the biggest boost in LLM capability so far in 2024. v1.5's 10M token context (1) excels at retrieval; (2) generalizes zero-shot to extremely long instructions like full tutorials and codebases; and (3) works across modalities such as text, audio, and video. Here's a stunning example: v1.5 learns to translate from English to Kalamang purely in context, following a full linguistic manual at inference time. Kalamang is a language spoken by fewer than 200 speakers in western New Guinea. Gemini has never seen this language during training and is only provided with 500 pages of linguistic documentation, a dictionary, and ~400 parallel sentences in context. It basically acquires a sophisticated new skill in the neural activations, instead of gradient finetuning. I talked about the Myth of Context Length many times before: don't get too excited by claims of 1M or even 1B context tokens. LSTMs already achieved literally infinite context length 25 yrs ago! What truly matters is how well the model actually uses the context to solve real-world problems, and Gemini-1.5 has surpassed the SOTA with flying colors. The paper is also well-written with lots of solid quantitative analysis on in-context memorization and generalization. Paper: “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context” Congrats to Jeff Dean Oriol Vinyals Sundar Pichai and team!

Jim Fan

278,517 次观看 • 2 年前

Reviewing AI-generated code is the new bottleneck. Writing code was slow before. Now, in a few minutes, you can generate virtually unlimited lines of code using AI. But developers are now spending 70% of their time checking that code. That's the new bottleneck. I've been using Cline, and they have a feature that actually helps with this: You can click a button and have Cline generate an inline explanation for every single line of AI-generated code. Take a look at the attached video. What it does: 1. You can read the code and the explanation together 2. You don't lose context while reviewing code 3. You understand the "why" behind each change And here is the best part: It works on any git diff. You can use it to review PRs, understand commits from other people, or catch up on a codebase you haven't touched in a while.

Reviewing AI-generated code is the new bottleneck. Writing code was slow before. Now, in a few minutes, you can generate virtually unlimited lines of code using AI. But developers are now spending 70% of their time checking that code. That's the new bottleneck. I've been using Cline, and they have a feature that actually helps with this: You can click a button and have Cline generate an inline explanation for every single line of AI-generated code. Take a look at the attached video. What it does: 1. You can read the code and the explanation together 2. You don't lose context while reviewing code 3. You understand the "why" behind each change And here is the best part: It works on any git diff. You can use it to review PRs, understand commits from other people, or catch up on a codebase you haven't touched in a while.

Santiago

64,438 次观看 • 8 个月前

"What happens when we can access each other's AI memory in enterprises?" scott belsky, investor/founder of Behance: "If I was working as an intern for years, and the context window is accessible to the enterprise, can the enterprise (keep prompting) and asking me what I'd think or do, after I leave?" "Every company right now is trying to build this AI memory for each of its products and users. A lot of them are building connectors to form a collective memory." "In the social/consumer world, imagine your girlfriend saying she wants access to your context window or memory." "There are all kinds of weird questions and implications that arise from this."

"What happens when we can access each other's AI memory in enterprises?" scott belsky, investor/founder of Behance: "If I was working as an intern for years, and the context window is accessible to the enterprise, can the enterprise (keep prompting) and asking me what I'd think or do, after I leave?" "Every company right now is trying to build this AI memory for each of its products and users. A lot of them are building connectors to form a collective memory." "In the social/consumer world, imagine your girlfriend saying she wants access to your context window or memory." "There are all kinds of weird questions and implications that arise from this."

TBPN

44,908 次观看 • 1 年前

$Microsoft just dropped a 17-page paper - "Less Context, Better Agents" - proving the thing nobody building agents wants to admit: Your agent isn't failing because it needs more context. It's drowning in the context it already has. They ran GPT-5 four ways on one 50-task benchmark (via MCP): Full history → bloated, pricey, more errors Trim to the last 5 tool calls → 79% done Add light summarization → 91.6%, on a fraction of the tokens Less context. Fewer tokens. Higher completion. Everyone's racing to cram MORE into the window. Microsoft just published the receipt that the opposite wins for long-running agents. Rewired how I build agents this week. Paper ↓$

Microsoft just dropped a 17-page paper - "Less Context, Better Agents" - proving the thing nobody building agents wants to admit: Your agent isn't failing because it needs more context. It's drowning in the context it already has. They ran GPT-5 four ways on one 50-task benchmark (via MCP): Full history → bloated, pricey, more errors Trim to the last 5 tool calls → 79% done Add light summarization → 91.6%, on a fraction of the tokens Less context. Fewer tokens. Higher completion. Everyone's racing to cram MORE into the window. Microsoft just published the receipt that the opposite wins for long-running agents. Rewired how I build agents this week. Paper ↓

Archive

18,242 次观看 • 1 个月前