Video yükleniyor...
Video Yüklenemedi
David Friedberg: Frontier Models are Training on Your Novel Insights as “De-Identified Data” @jason: “Should they trust any of these LLMs with their proprietary knowledge for fear of having it cribbed into a core LLM?” david friedberg: “I have had experiences where we've asked some fairly novel scientific questions,... show more
54,050 görüntüleme • 3 gün önce •via X (Twitter)
29 Yorum
It's interesting to see Friedberg (who I admire) go from defending the right of AI companies to "learn" from artists without their consent (no, putting something online is not granting permision to take it), just a few months ago, to now not agreeing with his work being used to improve AI models. When it's somebody else's work, it belong to the collective.. But when it's your work, now it's "your alpha" and you need to protect it.
@Jason @friedberg The OpenAI case seems more extreme - eg normally I’d expect a VERY unique corpus to get ‘averaged out’ in training, and the fact that that didn’t happen implies something else was done to prevent this. My guess is other LLMs are more ethically run and don’t do this?
This is all that LLMs are. Whoever has the training data wins. This is why Anthropic exists. This is why Anthropic thinks they will be the last company. If everything is distilled into the model, it’s going to be better than you because it has more memory and speed than a person can have. It becomes easy to copy anything.
@Jason @friedberg no NDA with a chatbot, groundbreaking discovery from a guy who runs a public company
@Jason @friedberg Too much echo chamber now a days. Where are the intellectual disagreements?
@Jason @friedberg De-identified is doing a lot of work in that sentence. Removing your name from a prompt does not remove the idea from it, and the idea is the thing you were worried about. The only setting that protects it is one where the prompt never enters training at all.
Worth separating the tiers here. On the API and business plans, OpenAI and Anthropic both say they don't train on your inputs by default. Opting in is a deliberate switch you have to flip. The five-year de-identified retention Friedberg is describing sits on the consumer plans, and only when the model training setting is left on. So the answer depends on which plan the proprietary work got typed into.
@Jason @friedberg Will it matter. All information and data is about to be available to everyone always. Your gonna be able to just prompt anything and it will figure. It out
@Jason @friedberg De-identification claims still need measurable guarantees against memorization and unintended model leakage.
@Jason @friedberg
@Jason @friedberg I used to work for a company that practiced it .. hired the best engineers … big paychecks .. forced to file for patents (idea harvesting) or get low rating (risk of elimination) .. fire them when they had the patent(s).. blackball them from finding employment in same field ..
😢@Jason Do you have an AI agent that randomly blocks followers if it has perceived some sort of slight against you? If so, THIS is how AI will likely destroy the world, because I cannot imagine what would have prompted this.🤔 (Unless it’s the AI ‘Do It or Don’t’ Nike ad I created on my feed, which is being hidden on X. I posted it 10 hours ago and 3 people have seen it, so if that’s the issue, it’s sort of a non-issue.🤣)
@Jason @friedberg taking the name off a document doesn't take the secrets out of it.
@Jason @friedberg It seems that these days it’s up to productive immigrants to defend the true American values.
@Jason @friedberg you didn't care about Google porting all of our domains over to Square Space forcing us into workspace... spare me
@Jason @friedberg Great episode. Was enjoyable to hear what I think helped build up All-In's audience: insightful business discussions. Interestingly, perhaps subconsciously, the discussions around 'go woke go broke' I think apply to some of the pods regressions (sentiment-not-metrics) 🧵
@Jason @friedberg Sorry but why are we all suddenly all act surprised when privacy is cooked? It has always been the name of the game from day 1.
@Jason @friedberg De-identification is not the same as non-retention. Enterprise trust needs explicit defaults: no training, short retention, tenant isolation, and auditable deletion. The real product moat is proving proprietary context never crosses the boundary.
@Jason @friedberg No discussion on Trump’s $5000 cheques and socialism eh? 🤣 that is why you guys lose to free grocery guys 🤦♂️
@Jason @friedberg Currently doing AI training sets for the profession I’ve worked in for the past 10 years for a major foundation model. I figure my career is cooked either way. Might as well get paid an absurd amount of money right now.
@Jason @friedberg De-identified doesn’t mean de-useful. If the insight was novel enough to matter to you, it’s novel enough to matter to the next model. Treat anything you wouldn’t publish as something you wouldn’t paste — especially if it’s the edge you’re paid for.
@Jason @friedberg The answer is to have the AI conversation in an LLM that generates the IP from the conversation and registers the origins of the original idea. X would be a great place to do that. Basically publish ideas on x in real time and get credit for it. Then one year to file the IP.
@Jason @friedberg MODEL BUILDERS CANT HOST unkess they are encrypt in and encrypt out.
@Jason @friedberg Clip Friedberg ✊
@Jason @friedberg That test can't tell training-on-your-chat from the model just improving. A canary phrase nobody else would type is the one that can.
@Jason @friedberg open source doesn't stop this, it just moves the training data collection in house
@Jason @friedberg You guys are not exactly rational and objective as you claim to be and are frankly getting TIRESOME. Time to take a break from the All-in Podcast.
To get ahead of what I have labeled the "Super Gambit" ( see: ) POTUS should tell the press that he's made a decision. The first "regulation" from any new govt AI agency will be to award "winner take all" status to the company that solves AI super intelligence safety and alignment because obviously we won't want any frontier AI model without that causing harm. Boom. Winner take all... NOW how much do you want fed govt regulating AI? cc: @DavidSacks @chamath
@Jason @friedberg LLM tokens traffic should be encrypted by the harness. Like a VPN that sits between your traffic and your ISP. This is what @nvidia should sell when hosting open source LLMs, people care about their privacy and IPs.
!["One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals. And you look at the evals and you go, 'Those are pretty hard evals.' But the economic impact seems to be dramatically behind. There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data. When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.' You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?' If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"](https://image.24vids.com/tw-1993450075616690474/amplify_video_thumb/1993449024045932544/img/W-1TjOXcXeiUbHaH.jpg)
![Whitney Webb "It sucks to find out that the mob runs the world... [so] do you want to say, screw you guys, we're going to... get off your... slave plantation?... [because] we can't really keep losing money and rights without being so deep in a hole that we can't... climb out" This clip of Webb (Whitney Webb), a contributing editor of Unlimited Hangout and author of One Nation Under Blackmail, is taken from an interview with Peter McCormack (The Peter McCormack Show) posted to YouTube on September 4, 2025. -----------------Partial transcription of clip-------------- "It's not just about burning it down, it's about putting something better in its place. Of course. And so I think the solutions there, you know, some people are like, oh, Whitney, your work is so demoralizing. Well, I'm like, yeah, it sucks to find out that the mob runs the world, you know, but the question is, or do you just want to let them keep running it because it bums you out, or do you want to say, screw you guys, we're going to build something else and get off your, you know, slave plantation? "And I am in the group of, I would like to build something else. I have three kids. I do not want them living in a world run by the mob. So what are you going to do about it? I mean, it starts locally, it starts with your community, because that's where we can actually affect change. Voting for left or right, blue or red. I mean, it's just this ping pong thing where nothing fundamental is changing. "And so to affect that change, we have to do it ourselves. And you think this would be a core American value? Individualism, individual responsibility. But a lot of people just don't care. And so, you know, that sucks. But I think those of us that do care need, to take steps to be as independent from the system as possible. Because every so often, this predator class, they do wealth transfers or they orchestrate and manufacture events and they make a big grab, not just for our money, but for our rights. And we're at the point where we can't really keep losing money and rights without being so deep in a hole that we can't climb ourself out. Can't climb out. "So what, what should we do before that happens? There is an important window of time to do something about it in our local community and for our families and our friends and our neighbors. And we have to do something because if we don't and that thing happens. Oh, well, I watched all these podcasts and learned about how corrupt everything is. But, you know, and I think they spend a lot of time trying to keep us distracted on all sorts of things and sucked into things that don't matter. So that we just don't give them the finger, basically. And that's what we really need to do."](https://image.24vids.com/tw-1964385906242343049/amplify_video_thumb/1964383325117960196/img/k88I5h6VHXiZJjv1.jpg)
![Rick Rubin tells Andrew Huberman how he deals with creative or writer’s block. He treats his work like a diary entry (and doesn’t worry about internal or external judgment): ➡️ “What's the cause of the block? The block is usually something that's either personal ("I'm not good enough") or it can be a confidence issue ("I don't have anything to say") or it could be...thinking about someone else ("nobody's going to like what I make"). Do you know what I'm saying? So, it's either fear of self-judgment or external judgment. If you're making something with a freedom of "this is something I'm making for myself for now", that is all [you have to do]. It is a diary entry. Everything I make is a diary entry. The beauty of a diary entry is that I can write my diary entry and you can't tell me that my diary entry wasn't good enough. Or that [the diary entry] is not what I experienced. Of course it's what I experienced: I'm writing a personal diary for myself and no one else can judge if it is my experience of my life. Everything we make can be that: a personal reflection of who we are in that moment of time. It doesn't have to be the greatest you could ever do. It doesn't have to have any expectation that it's going to change the world. It doesn't have to sell a certain number of copies for any reason. It doesn't have any of those things at all. It is "I'm making this thing for me and I want to do it to the best of my ability and to where I feel good about it". [The work] is honest of where I'm at and if you're living in this world of just being honest to where you're at, there's nothing blocking you. There are no blocks. The blocks are all based on dealing with a different force or a different perception that is made up.” ⬅️](https://image.24vids.com/tw-1739835673853804569/amplify_video_thumb/1739835456953704448/img/ZO5pgXpJKpiyF84w.jpg)
!["Where the hell are the [NHIs] in all this? Why aren't they coming out?" ~Semivan "What I experienced, you know, physical marks the whole thing, and my wife, worse." ~Semivan on their abduction "I would be very careful giving these entities that are f**king with you, like, all this great credit and putting them on a pedestal." ~DeLonge to Chris Bledsoe "It's indifferent to man." ~Vallée to Semivan "Are we evil when we go fishing and we hurt fish? Are we considered evil to the fish"? ~DeLonge Jay Stratton's, "got orbs flying around in his house, scaring the hell out of his wife and his kids." ~Semivan ( I enjoyed this episode because a lot of this is where I am when it comes to thinking about the phenomenon. I'm not saying this is the truth but it definitely resonates with me.) ~~~ Jim Semivan: "When we started 'To The Stars,' a while back, we took a little bit of flack because we were talking about...we weren't going along with the idea that the phenomenon was always a great, beneficial thing. We were saying, like, 'Look, let's just wait. There's been instances where it hasn't been...' I always like to use Chris Bledsoe as an example. He's just a dear, dear man, but when you listen to his story, up close and personally, and you could see the absolute fear that he went through, and his family went through, when he first encountered these beings, and this phenomena, and then later on how it transformed him into a very, very deeply spiritual man and healer. So you see this dichotomy. "And if you talk to Chris, Chris will tell you, 'No, this is a beneficial thing.' And then, you know, you run into guys like Keel, and even Jacques Vallée to a certain extent, where they'll tell you, 'Well, no, no, hold on a second.' Because I remember when when I had my experience, I was visited. It was years after my experience, I had met John Alexander at a conference and John put me in touch with some people. And then next thing I know, Jacques Vallée, you know, the guy, right? The house god of TTS calls me up and says, 'Can I come out and see you?' "So he flew out from San Francisco, came to my house in Virginia, and spent the day with my wife and I. And we had some very, very long, involved conversations about this. And I asked him, I said, 'Well, you know, what I experienced, you know, physical marks the whole thing, and my wife worse.' (That may be the first time he's said he had physical marks from his abduction-like experience with his wife when he was in grad school. We don't know details of what his wife experienced.) Semivan: "I said (to Vallée), 'Do you think this is evil?' And he said, 'I don't know, I don't think anybody knows. But I will tell you one thing: What I do know is that you can say that it's basically indifferent. It's indifferent to man. In other words, people get harmed by walking into it or getting caught in it, but it doesn't seem to have an evil intention, per se.'" (Reminds me of what Lue Elizondo said to Curt Jaimungal) Lue: "What if it turns out that a lot of the things that we thought were one way, aren’t. Are we prepared to have that honest question with ourselves? Are we prepared to recognize that we’re not at the top of the food chain, potentially?” (Most humans are indifferent to the welfare of most beings that are below us on the food chain. If we're driving and hit a squirrel, and they get hurt or killed, we may feel bad for a little bit, but we get on with our business and enjoy the rest of our day. How many cows, chickens and pigs are killed every day so we can eat them? Most humans, including me, don't give that a second thought. Indifference. That indifference may be happening with an intelligence (or multiple intelligences) that's above us on the food chain. And I'm not saying they're eating us, although that can't be ruled out 100%. I'm saying "they" just do their thing and if we get hurt or killed, it's not a huge deal to them. Indifference. And maybe some of them enjoy messing with us?) Semivan: "But then when you read Keel and you know, you read the Mothman and and a lot of these other stories like 'The Mojave Incident.' Well, holy cow! You look at this stuff. And, you know, you get these hobgoblins flying around, you know, with Jay Stratton. He's got orbs flying around in his house, scaring the hell out of his wife and his kids. I don't see, I don't know where there's any kind of a spiritual turn to that where you could look at all that and all of a sudden you say, 'Oh, now I'm a better person.'" (Most of us have known that Axelrod in "Skinwalkers at the Pentagon" is Jay Stratton but that's the first time one of his colleagues/friends has said it. And I agree: What's the purpose of orbs scaring, and leaving marks on, his kids? And the wolf that his wife, kids and their friends say they saw? Why? To make them better people? It makes zero sense to me.) Semivan: "It happened with Chris (Bledsoe) and, you know, great for him. Sure, as hell didn't happen to me. And I don't think it happened to Jay Stratton, either. I think it gave Jay a different outlook as with me, but sh*t, I mean, nothing I couldn't have got from reading books. What do you think?" Tom DeLonge: "I think like, yeah, you know, the idea of what we might call something evil, but to that, it's like, you know, are we evil when we go fishing and we hurt fish? You know, are we considered evil to the fish? So it's things looking out for its own interest." (100%. Someone asked me about good vs. evil last night and I wrote back, "Not about good vs. evil. About survival and food chain. Somber.") DeLonge: "The thing that I look at, you know, a lot of the ideas of good and evil come from like, religion, all different types of religions where it really defines the metaphysical nature of who we are and what we're living in. But I look at religion... I cannot think of one religion that hasn't left a wake of destruction. I mean, literally, every single one has done so much harm. And the one pattern in all these religions is that most of them, outside of the pagan ones, have really tried to just to get us to believe that the creator of the universe is some kind of human, or something. It's not like a source, an energy, it's like, it's a Jesus or an Allah, or it's a Buddha, or I don't know. Buddhism doesn't really say that, but you know what my point is. It's trying to humanize something. "And so, to me, it's like when Chris Bledsoe, or some of these others that have these kinds of spiritual experiences, these are the exact same experiences that are written about throughout the entire Bible, in the Quran, and like, all these other texts. And all of them are really setting the stage for us to worship the UFO and their occupants, to some degree." (You see this a lot with the folks who think we're dealing with ETs here to uplift humanity. The Space Brothers theory. I don't know the truth but, to me, it doesn't seem like that's the answer.) DeLonge: "And that, to me, is the one strong pattern, is to confuse us and not let us know of our own spiritual nature to source consciousness." (I've said many times that I believe a lot of this is about human potential and our untapped capabilities in the psi (psychic, pk, consciousness, etc.) area) DeLonge: "And as long as we don't know that, then we will think that, 'Okay, the aliens like, f**ked with our DNA and made us like this,' or 'The Tall Whites showed up and they were the Greek gods, too, and they gave us philosophy and all this whatever.' It's like we're always like, giving some kind of humanized form credit for everything. "And the one thing that's common in all these religions is that it's not giving credit to source energy that allows us to heal, that allows us to be telepathic, that allows us to have telekinesis, that allows us to have, like, compassion in energy to healing. Like, whether you do Reiki and meditate, or whether you sit and pray, it doesn't matter. Why does it work in every religion? Because it's source energy, it's not about the religion." (The one thing I could never get behind is healing because there was no data to back it up. That was until I was shown the work of Bill Bengston. Tons of data. How does it work? I don't know. Nobody does.) DeLonge: "But if you look at who's responsible for all the religions? Well, it's whatever UFO the Star of Bethlehem was, or whatever UFO the Fatima thing was, or whatever the Tall White was that met with Joseph Smith and started the Mormon religion was. Whatever the Jinn are that are in the Koran, or whatever the Demons are there in the Old Testament, they seem to be the things that come in, that take you in your sleep, and make you think things and deceive you. It's all the same." (Deception, deception, deception. A lot of this is very similar to what David Icke (and others) has written about.) DeLonge: "So you have all these things f**king with you, as the good ones and the bad ones. But all the religions, the pattern is, is never ever, ever let humans know the physics of their existence. Which is meditate, you can heal somebody and have all your information you need, or pray. It's the same thing, it doesn't matter. "That's more like my line of thinking is, is why it's so important for us to understand the true nature of spirituality, and why I step back when I hear the Chris Bledsoe stories. And it's wonderful that he's gotten to a place that it's helping him, because I really do like him as a person, and I really respect and honor his experiences because I firsthand have heard them from him. And he's just a wonderful human. I mean, it was an incredible story. But I told him personally, I said, I would be very careful giving these entities that are f**king with you, like, all this great credit and putting them on a pedestal, because these are the same groups that have been doing a lot of weird s*it throughout time, good and bad.' (Again, I think Bledsoe has had real experiences but IMO, his interpretation of what he's dealing with is off.) DeLonge: "And that, to me, seems, you know, that seems suspect. You know, it seems like if there is any really, true, beautiful, great force, and entities representing that force, like your angels in the Bible. I mean, even in the Bible, people would, you know, when they saw an angel, they would collapse on the ground in extreme fear, and like, they couldn't even look at them. It was like the scariest thing that's ever happened. And that doesn't sound like a real loving interaction to me (laughs). That sounds like some scary shit, you know? "So I think whatever would represent true love and unparalleled love, that doesn't seem to be showing up and f**king with our society and our timelines, and our behaviors, and trying to manage us. It's letting us do it on our own, you know, like, figure it out on your own, you know? But everything else that's showing up, seems to have its own interest at heart." (I think TD is saying that a loving/caring intelligence wouldn't try to manipulate us into doing things that just benefit THEM. And that's what appears to be going on. "They" only care about themselves. If there were ones here who truly cared about us, they would leave us alone and let us figure it on our own.) Semivan: "I tend to agree with you on this. I did a podcast a while back and somebody related to this. They were talking about why there hasn't been Disclosure? And I said, 'Well, you know, there's two sides of this story.' I said, 'You have the non human-intelligences, you know, or the phenomenon, and then you have the governments.' And I said 'Well, where the hell are the non-human intelligences in all this? Why aren't they coming out? They pretty sneakily running around, and they meet people in weird, obscured circumstances, make their experiences sound crazy. Yet, they sort of wanna be known, but they don't wanna be known.' As I mentioned to somebody once, 'They'll screw around with you but you can't take them home to meet your parents. They're not landing on the White House lawn for a reason.' I'm not quite sure what that is."](https://image.24vids.com/tw-1773078284487798811/ext_tw_video_thumb/1773040319740948480/pu/img/LNfQpFjqV8ZVbffs.jpg)
![The most interesting part for me is where Andrej Karpathy describes why LLMs aren't able to learn like humans. As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.” A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer. > “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.” So what do humans do instead? > “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.” > “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.” Why can’t we just add this training to LLMs today? > “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.” > “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.” > “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.” How do humans get around model collapse? > “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.” In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by Erik Hoel. I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization? > “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”](https://image.24vids.com/tw-1979259041013731752/amplify_video_thumb/1979257571786543104/img/OqUh56yit9gUQWnb.jpg)
