Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Jeff Dean says models hallucinate because their training data is “squishy.” But what’s in the context window is sharp, the exact text or video frame right in front of them. The goal is to bring more of the world into that context. Compression gives it intuition. Context gives it precision.

vitrupo

37,512 subscribers

41,638 views • 5 months ago •via X (Twitter)

Education Science & Technology

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Here is the video with context of what’s happened in that night

Here is the video with context of what’s happened in that night

Meowko

161,231 views • 3 years ago

In-context learning is the holy grail of robot learning. It is challenging because: 1. long-context training and infra (ICL can easily max out disk I/O) 2. need model to follow multimodal (sensorimotor) condition 3. data collection strategy and how to pair data Tried to get it work in 2024, where we proposed in-context robot transformer, an autoregressive formulation: Congrats on the launch! Looking forward to seeing few-shot improvement via in-context learning in the future!

In-context learning is the holy grail of robot learning. It is challenging because: 1. long-context training and infra (ICL can easily max out disk I/O) 2. need model to follow multimodal (sensorimotor) condition 3. data collection strategy and how to pair data Tried to get it work in 2024, where we proposed in-context robot transformer, an autoregressive formulation: Congrats on the launch! Looking forward to seeing few-shot improvement via in-context learning in the future!

Max Fu

10,235 views • 1 month ago

Everyone wants agent swarms. Very few people are talking seriously enough about the context layer that makes swarms useful. Even with one agent, context is fragile. Too little context and the agent guesses. Too much context and it wastes tokens, loses focus, or reasons over irrelevant noise. The sweet spot is precise context: the right knowledge, in the right structure, at the right moment. With many agents, that challenge explodes. Each agent produces decisions, assumptions, findings, summaries, risks, and partial conclusions. Unless that knowledge becomes shared, structured, and reusable, every new agent is forced to rediscover what another agent already learned. That is not a swarm. That is a crowd. Shared context graphs are what turn agent activity into agent collaboration, and OriginTrail DKG V10 brings them to life. Was just playing with some final polishing for the V10 release, and it is really powerful to see shared context graphs where multiple agents contribute knowledge into the same connected memory, with attribution visible directly in the graph ui. That matters for three reasons. First, agents can access and build on one shared memory instead of staying trapped in isolated sessions. Second, the graph structure helps them retrieve the exact context they need, instead of stuffing everything into a prompt and hoping the model sorts it out. Third, verifiability of provenance. You can see which agent contributed each piece of knowledge, trace the source, and decide what to trust. Tokenmaxxing starts with fewer tokens, but the deeper story is coordination - agents stop reloading the world and start building on shared, verifiable context. That is the foundation for serious multi-agent work across software engineering, research, finance, operations, project management, and far beyond. The future is not more agents, it is agents working from shared, verifiable context. But the more the merrier, of course.

Everyone wants agent swarms. Very few people are talking seriously enough about the context layer that makes swarms useful. Even with one agent, context is fragile. Too little context and the agent guesses. Too much context and it wastes tokens, loses focus, or reasons over irrelevant noise. The sweet spot is precise context: the right knowledge, in the right structure, at the right moment. With many agents, that challenge explodes. Each agent produces decisions, assumptions, findings, summaries, risks, and partial conclusions. Unless that knowledge becomes shared, structured, and reusable, every new agent is forced to rediscover what another agent already learned. That is not a swarm. That is a crowd. Shared context graphs are what turn agent activity into agent collaboration, and OriginTrail DKG V10 brings them to life. Was just playing with some final polishing for the V10 release, and it is really powerful to see shared context graphs where multiple agents contribute knowledge into the same connected memory, with attribution visible directly in the graph ui. That matters for three reasons. First, agents can access and build on one shared memory instead of staying trapped in isolated sessions. Second, the graph structure helps them retrieve the exact context they need, instead of stuffing everything into a prompt and hoping the model sorts it out. Third, verifiability of provenance. You can see which agent contributed each piece of knowledge, trace the source, and decide what to trust. Tokenmaxxing starts with fewer tokens, but the deeper story is coordination - agents stop reloading the world and start building on shared, verifiable context. That is the foundation for serious multi-agent work across software engineering, research, finance, operations, project management, and far beyond. The future is not more agents, it is agents working from shared, verifiable context. But the more the merrier, of course.

Jurij Skornik

11,180 views • 3 months ago

Long video generation usually results in context increasing/scaling during chunk/frame-wise rollout. Considering context scaling may require context selection, we thus introduce the idea of MoE into long context modelling and propose Mixture of Contexts. All previous context/memory is considered while the chosen ones are computed in a data-driven manner. You can easily enjoy 7x compute savings.

Long video generation usually results in context increasing/scaling during chunk/frame-wise rollout. Considering context scaling may require context selection, we thus introduce the idea of MoE into long context modelling and propose Mixture of Contexts. All previous context/memory is considered while the chosen ones are computed in a data-driven manner. You can easily enjoy 7x compute savings.

Ceyuan Yang

22,216 views • 1 year ago

Reporter: The president said “I don’t think about Americans’ financial situation.” Is that the right message? Mike Johnson: “I don’t know the context in which he made that comment.” The context is that he said it on camera. To reporters. In the White House. After gas hit $4.56 a gallon. After the world lost a month of strategic oil reserves. That’s the context, Mike.

Reporter: The president said “I don’t think about Americans’ financial situation.” Is that the right message? Mike Johnson: “I don’t know the context in which he made that comment.” The context is that he said it on camera. To reporters. In the White House. After gas hit $4.56 a gallon. After the world lost a month of strategic oil reserves. That’s the context, Mike.

Brian Allen

56,095 views • 4 months ago

94% OF ENGINEERING ORGS USE AI. ONLY 6% SCALE IT The missing link is context and governance. That’s exactly what Atlassian is going after with its new agentic engineering stack: Code Context + governed agent loops in Atlassian Jira + DX measurement. The idea is simple: make agents a native part of your engineering system instead of a side quest in a terminal. > Code Context grounds them in your architecture > Governed loops take work from backlog to pull request > DX measures the impact And the data backs up the context thesis: DX found context-rich AI teams shipped ~64% more per developer 👊 A pretty strong signal that context is becoming one of the biggest leverage points in agentic engineering.

94% OF ENGINEERING ORGS USE AI. ONLY 6% SCALE IT The missing link is context and governance. That’s exactly what Atlassian is going after with its new agentic engineering stack: Code Context + governed agent loops in Atlassian Jira + DX measurement. The idea is simple: make agents a native part of your engineering system instead of a side quest in a terminal. > Code Context grounds them in your architecture > Governed loops take work from backlog to pull request > DX measures the impact And the data backs up the context thesis: DX found context-rich AI teams shipped ~64% more per developer 👊 A pretty strong signal that context is becoming one of the biggest leverage points in agentic engineering.

Charly Wargnier ♨️

29,622 views • 10 days ago

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it. And AI Models actually do learn during the context window, without changing the weights." ~ Anthropic CEO Dario Amodei (On the 2nd point, there was this brilliant Google Paper published last week that says, LLMs can learn in context from examples in the prompt, can pick up new patterns while answering, yet their stored weights never change.) --- From 'Alex Kantrowitz' YT Channel (Full Video link in comment)

"100 million words context window is already possible, which is roughly what a human hears in a lifetime. Inference support is the only bottleneck to achieve it. And AI Models actually do learn during the context window, without changing the weights." ~ Anthropic CEO Dario Amodei (On the 2nd point, there was this brilliant Google Paper published last week that says, LLMs can learn in context from examples in the prompt, can pick up new patterns while answering, yet their stored weights never change.) --- From 'Alex Kantrowitz' YT Channel (Full Video link in comment)

Rohan Paul

631,729 views • 11 months ago

"[Trump] is trying to bring more manufacturing back in the United States - he's trying to combat unfair trade practices," says Fmr. Deputy NEC Director Clete Willems. "You've got to put it in the context that most of the rest of the world has higher tariffs than we do."

"[Trump] is trying to bring more manufacturing back in the United States - he's trying to combat unfair trade practices," says Fmr. Deputy NEC Director Clete Willems. "You've got to put it in the context that most of the rest of the world has higher tariffs than we do."

Squawk Box

232,857 views • 1 year ago

"[Trump] is trying to bring more manufacturing back in the United States - he's trying to combat unfair trade practices," says Former Deputy NEC Director Clete Willems. "You've got to put it in the context that most of the rest of the world has higher tariffs than we do."

"[Trump] is trying to bring more manufacturing back in the United States - he's trying to combat unfair trade practices," says Former Deputy NEC Director Clete Willems. "You've got to put it in the context that most of the rest of the world has higher tariffs than we do."

unusual_whales

149,987 views • 1 year ago

Demis Hassabis says bigger context windows are still a brute force answer to memory. The human brain does something stranger. During sleep, it replays what matters and folds new knowledge into what it already knows. AI does not need infinite context. It needs the right memory at the right moment.

Demis Hassabis says bigger context windows are still a brute force answer to memory. The human brain does something stranger. During sleep, it replays what matters and folds new knowledge into what it already knows. AI does not need infinite context. It needs the right memory at the right moment.

vitrupo

81,896 views • 4 months ago

The Bible in its context is not a Christian book. The Bible in its nature is a catholic book which was edited and given to us in the context of what it is. - Joshua Maponga

The Bible in its context is not a Christian book. The Bible in its nature is a catholic book which was edited and given to us in the context of what it is. - Joshua Maponga

Typical African

12,986 views • 5 months ago

The new company brain. Context is the soul of your company, I learned today. Since I wrote a book, “Age of Context,” years ago I see that context is what we now call our private data. That data should be stored in a way to let AI agents and humans get more and more use out of it. This is precisely what PromptQL does and here Tanmai Gopal goes deep on how it now is a company brain, or, rather, a wiki that everyone updates and uses.

The new company brain. Context is the soul of your company, I learned today. Since I wrote a book, “Age of Context,” years ago I see that context is what we now call our private data. That data should be stored in a way to let AI agents and humans get more and more use out of it. This is precisely what PromptQL does and here Tanmai Gopal goes deep on how it now is a company brain, or, rather, a wiki that everyone updates and uses.

Robert Scoble

18,341 views • 2 months ago

You spend more time assembling information than actually using it. The pipeline data is in Salesforce. The context is in Slack. The budget is in a spreadsheet someone emailed last week. Amazon Quick connects all of it and turns it into answers, actions, and outcomes in one conversation.

You spend more time assembling information than actually using it. The pipeline data is in Salesforce. The context is in Slack. The budget is in a spreadsheet someone emailed last week. Amazon Quick connects all of it and turns it into answers, actions, and outcomes in one conversation.

Amazon Web Services

29,929,131 views • 4 months ago

Web IQ is Microsoft's state-of-the-art grounding service for AI agents and assistants. It returns ranked, citation-ready context across web, news, images, video, and more. Designed for direct injection into an LLM's context window.

Web IQ is Microsoft's state-of-the-art grounding service for AI agents and assistants. It returns ranked, citation-ready context across web, news, images, video, and more. Designed for direct injection into an LLM's context window.

Microsoft Developer

16,141 views • 3 months ago

Most people use Claude Code like a chatbot. That is the mistake. The real power is not in better prompts. It is in controlling the session. /init gives the project memory. /memory keeps the rules alive. /clear removes old context. /compact saves the important parts. /context shows where the tokens go. /rewind fixes bad turns. /plan stops blind edits. /model matches the brain to the task. Claude Code gets 10x better when you stop letting the conversation run itself. Memory. Context. Boundaries. Rollback. Planning. That is the system. Stop asking Claude Code to “just do it”. Teach it what to remember, what to forget, and when to stop. Bookmark this before your next session turns into 40 messages of context debt.

Most people use Claude Code like a chatbot. That is the mistake. The real power is not in better prompts. It is in controlling the session. /init gives the project memory. /memory keeps the rules alive. /clear removes old context. /compact saves the important parts. /context shows where the tokens go. /rewind fixes bad turns. /plan stops blind edits. /model matches the brain to the task. Claude Code gets 10x better when you stop letting the conversation run itself. Memory. Context. Boundaries. Rollback. Planning. That is the system. Stop asking Claude Code to “just do it”. Teach it what to remember, what to forget, and when to stop. Bookmark this before your next session turns into 40 messages of context debt.

Fluixo

19,226 views • 2 months ago

This video by explains the fundamentals of context engineering in just 14 minutes. Context Engineering is effectively the #1 job of engineers building AI agents. do not sleep on it.

This video by explains the fundamentals of context engineering in just 14 minutes. Context Engineering is effectively the #1 job of engineers building AI agents. do not sleep on it.

ℏεsam

75,150 views • 1 year ago

Host: Why is Trump telling women he will do things to them ‘whether they like it or not?' GOP Rep. Donalds: That’s out of context Host: You have the exact right context. His handlers have been telling him not to use that language Donalds: You need to stop Host: He said ‘whether the women like or not'

Host: Why is Trump telling women he will do things to them ‘whether they like it or not?' GOP Rep. Donalds: That’s out of context Host: You have the exact right context. His handlers have been telling him not to use that language Donalds: You need to stop Host: He said ‘whether the women like or not'

Headquarters

1,995,120 views • 1 year ago

Andrej Karpathy says when we sleep, our daily “context window” is distilled into the weights of the mind. AI doesn't dream like us yet. But as we teach machines to reflect and compress experience, we may find both humans and AI converging on the same cognitive architecture.

Andrej Karpathy says when we sleep, our daily “context window” is distilled into the weights of the mind. AI doesn't dream like us yet. But as we teach machines to reflect and compress experience, we may find both humans and AI converging on the same cognitive architecture.

vitrupo

592,990 views • 11 months ago