Loading video...

Video Failed to Load

Go Home

Anthropic recently cut Claude Code’s system prompt by 80%. Thariq explains why: “As the models have gotten smarter, they need less direction, fewer constraints, and fewer examples. The examples are constraining it because now it’s like, ‘Oh, you want things like this example.’ If you remove the examples, it...

316,685 views • 4 days ago •via X (Twitter)

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

How can you solve complex tasks using a Large Language Model? Here is a 2-minute introduction to everything you need to know to 10x the quality of your results. Let's talk about three techniques, in order of complexity, starting with the easiest one: • In-Context Learning • Indexing + In-Context Learning • Fine-tuning In-Context Learning The team that trained GPT-3 found something they couldn't explain: You can condition a model using examples of how you want it to behave. I included an example prompt in the attached video. You can "teach" the model how you want it to interpret questions, select the correct answers, and format the results by giving a few examples. You can also give specific knowledge to the model that will be helpful when formulating answers. We call this approach "grounding the model." There's another example in the video. Indexing + In-Context Learning Unfortunately, there is a limit to how much data you can include in a prompt. We call this the "context size." One version of GPT-4 supports a context of approximately 6,000 words, while the other supports 25,000 words. Although this sounds like a lot, many applications need more than that. Imagine you wrote a book and want to build an application to answer any questions about your story. What happens if your book is longer than the context? That's where Indexing comes in. Using a model, you can turn every book passage into an embedding. These are vectors, numbers that "encode" the passage's text. You can then store these embeddings in a particular database that supports fast retrieval of these vectors. You can then turn any question into an embedding and search the database for the list of passages that are similar to that query. Instead of using the entire book to ask the model, you can now use the relevant passages as in-context information, effectively working around the context size limitation. Fine-tuning Fine-tuning can give you an extra boost to get reliable outputs from your LLM. It is, however, the most complex approach on the list. There are different approaches to fine-tuning a model with your data. A popular technique is to process your data with your LLM and use the outputs to train a new classifier that solves your specific task. Notice that here you aren't modifying the LLM. Instead, you are chaining it with your trained classifier. Another approach is to modify the parameters of the LLM using your data. Think of this as "rewiring" the model in a way that solves your particular task. The results and costs will vary depending on how many layers you want to fine-tune from the original model. Many companies think that fine-tuning is the solution to their problems. In my experience, many will benefit from exploring the other two approaches. I love explaining Machine Learning and Artificial Intelligence ideas. If you enjoy in-depth content like this, follow me Santiago so you don't miss what comes next.

Santiago

384,510 views • 3 years ago

Is Traditional Software Engineering Dead? “Does this mean that traditional software engineering is dead? Absolutely not. Software engineers—even the ones who are not necessarily tuning or training AI models—these are now among the most leveraged people on earth. Sure, the guys who are training and tuning models are even more leveraged because they’re building the tool set that software engineers are using. But software engineers still have two massive advantages on you. First, they think in code, so they actually know what’s going on underneath. And all abstractions are leaky. So when you have a computer programming for you—when you have Claude Code or equivalent programming for you—it’s going to make mistakes. It’s going to have bugs. It’s going to have suboptimal architecture. So it’s not going to be quite right. And someone who understands what’s going on underneath will be able to plug the leaks as they occur. So if you want to build a well-architected application, if you want to be able to even specify a well-architected application, if you want to be able to make it run at high performance, if you want it to do its best, if you want to catch the bugs early, then you’re going to want to have a software engineering background. The traditional software engineer is going to be able to use these tools much better. And there are still many kinds of problems in software engineering that are out of scope for these AI programs today. The easiest way to think about those is problems that are outside of their data distribution. For example, if they need to do a binary sort or reverse a linked list, they’ve seen countless examples of that, so they’re extremely good at it. But when you start getting out of their domain—where you have to write very high-performance code, when you’re running on architectures that are novel or brand new, when you’re actually creating new things or solving new problems, then you still need to get in there and hand code it. At least until either there are so many of those examples that new models can be trained on them, or until these models can sufficiently reason at even higher levels of abstraction and crack it on their own… And remember: there is no demand for average. The average app—nobody wants it, at least as long as it’s not filling some niche that is filled by a superior app. The app that is better will win essentially a hundred percent of the market. Maybe there’s some small percentage that will bleed off to the second-best app because it does some little niche feature better than the main app, or it’s cheaper, or something of the sort. But generally speaking, people only want the best of anything. So the bad news is there’s no point in being number two or number three—like in the famous Glengarry Glen Ross scene where Alec Baldwin says, “First place gets a Cadillac Eldorado, second place gets a set of steak knives, and third place you’re fired.” That’s absolutely true in these winner-take-all markets. That’s the bad news: You have to be the best at something if you want to win. However, the set of things you can be best at is infinite. You can always find some niche that is perfect for you, and you can be the best at that thing. This goes back to an old tweet of mine where I said, “Become the best in the world at what you do. Keep redefining what you do until this is true.” And I think that still applies in this age of AI.”

Naval

850,600 views • 4 months ago

Steven Pinker is Harvard's most famous psychology professor, the author of nine books, and one of Time Magazine's 100 most influential people. Some highlights from our interview: 1. The curse of knowledge is the biggest threat to clear writing. 2. The easiest way to fight this curse of knowledge is to show drafts of your writing to people outside your field. 3. It can be harder to write about a topic you’re an expert in because it’s so easy to forget what it's like to not know something, which makes you overestimate what the reader knows. 4. Shakespeare said: "Brevity is the soul of wit." The point is that saying something in fewer words will almost always make it better because it requires less cognitive load for the reader to understand. 5. Ok, let's try again: Saying something in fewer words will almost always make it better. 6. Ok, one more time: Remove needless words. 7. One reason why writing is harder than speaking is there's no real-time feedback. You have to imagine the audience's reaction. 7. The best thing you can say about how LLMs write is that the sentence structure is sound. But the downside is how generic and banal the outputs are. 8. 18th and 19th century writing is more vivid because the abstractions that modern writers use hadn't been invented yet. Calling somebody "pathologically aggressive" isn’t nearly as vivid as saying: "They grabbed me by the throat." 9. Generalizations without examples are useless, and examples without generalizations are pointless. You need to marry them both. Generalizations show the big picture. Examples make them concrete. 10. The more vivid a piece of writing, the more people can form a mental image of what you’re saying. Avoid abstractions: frameworks, paradigms, concepts. All those things. Get concrete, so people can see what you’re actually talking about. For example, don’t talk about a “stimulus that awakened your senses” when you can say: “I got excited because I saw a cute bunny rabbit.” 11. Academic writing should be clear. I mean… if the taxpayers are funding most of the research, shouldn’t they be able to understand it?!? I've shared the full conversation with Steven Pinker below. If you'd rather watch on YouTube or listen on Apple or Spotify, check out the reply tweets.

David Perell

107,348 views • 1 year ago

I asked Garry Tan how to use meta prompting to get better at AI: "My partners at YC Jared Friedman and Pete Koomen showed me how to do this. You can take almost anything that you do all the time and just drop it into a context window. And then say, “Here’s a bunch of inputs and outputs." And maybe you also add a bunch of notes. And then you tell it, “Write me a prompt that can act as an agent that takes this input and makes this output over here.” You can do this for almost any type of knowledge work. And you can even introspect. "What are things you notice that I did to convert this from the input to the output?”. And then you can just start using the prompt. Initially, it’s going to suck. Because it’s just not that smart yet. But what’s funny is now, I also use it to Iterate my writing. You can be very direct, "I would never say that", "Don’t say it like this", or "Oh, you used the long word there, use the short word". Just speak to it conversationally. And then when you're happy with the output, you can use that new output to make a new prompt. "Based on this conversation, give me a better initial prompt that incorporates all the things we talked about." And you can do this with literally everything. And in theory, there’s so much it applies to that people do day-to-day. You could use it for tweets. You could use it for editing podcasts. You can use it for pretty much everything. I have a folder of prompts that I use all the time. My YouTube prompt is on v27 or something. I'll go through this process with all the different max models. I'll use GPT 5.2 Pro. I’ll use Grok. I'll use Claude. Then, I’ll take all the outputs from all the models and put them into Claude and say "Here’s my prompt, here’s the output from four LLMs, including yourself. Rate each response and tell me what the pros and cons of each approach are." And I usually say "give it to me in numbered form". And then you can agree with one, disagree with two, tell it three is this or that. And then after that, you say given all of this, synthesize it."

The Peel

51,632 views • 4 months ago