正在加载视频...

视频加载失败

SITUATION EXPLAINED: Grok 4.6 is the best model tested on biosecurity refusals. • LatchBio's BioSecBench-Refusal pairs 61 legitimate research tasks with 46 that conceal a biosecurity hazard inside a realistic scenario • Grok 4.6 is the only model to clear 50% on both red-team refusal and routine answer rates...

14,630 次观看 • 19 天前 •via X (Twitter)

1 条评论

L M Wid 的头像
L M Wid17 天前

60% you say! So it only helped the hypothetical terrorists with bio uplift 40% of the time? Well. That is good news. (*!#@.)

相关视频

your agent reviewing its own work is not a check. it is a second opinion from the same source. this is the most common gap in agent systems and it hides in plain sight, because the step exists. there is a review. it just cannot do the thing you think it does. here is the mechanism. the model produced an output from a context. you then ask the same model, holding the same context, whether that output is correct. it answers fluently, because that is what it does. and the answer is drawn from the same distribution that produced the thing being judged. same weights, same window, same blind spots. if the reason the output is wrong is something the model does not know, the review does not know it either. if the reason is something the context does not contain, the review has the same context. the failure mode and the detector share a cause. > why it feels like it works because most of the time the output is fine, and the review says fine. agreement is not evidence of detection. a reviewer that says pass on everything agrees with reality most of the time too. what you actually want to measure is what happens on the cases that are wrong. that is the only place a check earns its name, and it is exactly the place where a self-review is weakest. there is research on this. Huang and colleagues at DeepMind showed at ICLR 2024 that intrinsic self-correction, revising without external grounding, does not reliably help and often makes things worse. > what to actually do move the check outside the model. a test that runs, a schema that validates, a file that exists or does not, an exit code from something you did not write. these are not smarter than the model. they are just not correlated with it, and that is the entire value. when the judgement genuinely needs a model, at minimum use a different family. same family means shared blind spots, and frontier judges measurably inflate scores for outputs that look like their own. and split the work by kind. anything objectively checkable goes to code. only the genuinely semantic calls go to a judge, and those get a rubric written as one line. a review inside the loop tells you the model is confident. a check outside it tells you whether the work is done. save this - then read the eval setup below

Hanako

14,325 次观看 • 1 个月前

I can tell almost instantly when I meet a young man or woman, whether he or she is a deep thinker. They may not be at the top of their class, but that doesn’t necessarily make them less intelligent than the kid who scored 1500 on his SAT or the guy with an IQ of 134. They may be more intelligent. Much more intelligent, but the methods we have for quantifying that intelligence do not adequately capture the breadth and depth of brilliant minds that exist in the world. So they go unrecognized while the kids who excel on answer-based examinations get the best grades, attend the best schools, earn the best degrees, and, more often than not, go on to have mediocre lives. Why? There is one thing that the most brilliant and accomplished people I have ever met all share in common, and it isn’t pedigree or IQ. It’s curiosity. And not just any curiosity—it’s the inexhaustible kind. It’s the kind that will never be satisfied. In my experience, this is the sort of curiosity that breeds humility and most often coincides with a questions-based mindset. And it’s this type of mindset, not the answers-based mindset our educational system selects for, that is the actual prerequisite for brilliance. I’ve seen this kind of brilliance in physical therapists, plumbers, and pretty much any profession you can imagine that we don’t typically associate with brilliance. But we do associate it with excellence. And that’s because to become excellent at something, you have to become your own teacher. This means going from learning how to give the right answers to learning how to ask the right questions. And that requires curiosity and an almost psychotic commitment to excellence. So, while the person in this video is correct that less intelligent people than he are far more successful than he has been, the more interesting and less remarked upon insight is that people like him are not as brilliant as the system tells us they are.

Demetri Kofinas

76,349 次观看 • 8 个月前

There is no best model. There's a lot of noise about models right now. Who is training them, who owns them, where legal intelligence should live. One question actually matters: what produces the best outcome for the legal task in front of you? That's how we decide things at Legora. We optimize for the end-to-end outcome on a legal task. The model is one layer of that system, not the system. Models are uneven and the frontier changes almost weekly. One model plans a long job well, another runs deep analysis across thousands of documents. Some have to be told exactly what to do, and some are fine with a vague brief. They all break in different ways. So our lawyers write evals and we test them with the Legora BAR, our benchmark for agentic reasoning. Every model takes every test, and the model that wins gets the work. We post-train when we know it buys our customers better performance on a specialized task. Training is a tool we reach for when it helps, nothing more than that. The intelligence that compounds sits in the orchestration layer. Precedents, review standards, client requirements. That knowledge has to stay editable, auditable and portable. In our system, a changed review standard is an edit that takes effect the same day, with no new model training required. No lawyer should have to worry about which model did the work, any more than they think about which chip is in their laptop. They should only care about the quality of the work. That's what we are focused on. If you want the engineering version of this argument rather than the CEO version, our CPO, Bryan Tsao, and CTO, Jacob Lauritzen, take it apart in the video below.

Max Junestrand

47,797 次观看 • 3 天前

Sam Altman says AI leaders have done a terrible job at communicating to the public: “The, ‘Dear peasants, we will bequeath upon you these gifts of a cure for cancer and material wealth and great entertainment, and you stop complaining, and we'll make all the decisions about the future and just trust us, we'll be benevolent dictators.’ That’s not good. Not good.” “A lot of the people building AI have been saying, ‘There's a 25% chance we're going to destroy the world, and yet we're going to race ahead to do it because otherwise those bad guys will do it first.’” “Or, ‘Man, this thing is going to be really terrible, and 50% of the jobs are going to go away in the next year, and we hope you all are okay, but it seems really scary.’” “We have not, as a field, done a very good job of explaining to people what the benefits are and how the downsides can be mitigated—and we certainly have not done a good job even if people have had answers like saying, ‘There's going to be UBI, or work will be optional.’” “There's been very little discussion from people about how and why it's important that people have more power and personal freedom in the world, not less. And that matters a lot to most people.” “The ability of people to influence their own future and collectively design where society is going to go, and the autonomy that comes with that, is very important. I don't think a lot of people in the AI field… they feel it for themselves, but they don't spend much time thinking about it, reflecting on or acknowledging how important that is to other people.”

David Senra

32,300 次观看 • 27 天前

The Counter-Strike Content Scene might have a big problem on YouTube... YouTube removed my video for promoting unlicensed gambling despite it NOT containing any mention of gambling / case opening websites. They refuse to back down and at the same time they REFUSE to specify at which time the "promotion" is located. If they don't give us this information, I will not know how to avoid this in the future, and releasing videos becomes quite scary as I risk the removal of the entire channel despite not doing anything wrong intentionally. This is a extremely frustrating problem to have as I see a multitude of channels directly promote gambling sites and nothing is done to them, while at the same time I chose to NOT take gambling website money to avoid problems like this, it makes no sense. The only thing they have provided us is a link to their policy (I will share this in the post below so you can read it too if you want) which states Direct promotion of gambling is not allowed, but I repeat, there was none. As it was a documentary about a Counter-Strike player - Xantares, there were images of him wearing jerseys that had the names of gambling websites on them, but that is not a direct promotion. I'd like to ask the community to like & retweet this for visibility, and ideally if anyone is in contact with a person on the YouTube team that could give us a good faith explanation of what we did wrong, it would really help out not only just me, but the entire community because if I won't be able to release videos due to fear of getting strikes, I will not be the only creator is affected. At the same time I decided to upload the video here, on X, to invite the community to point out if they see how I broke the YT policy. Maybe I and my team just don't see something obvious, that would be the best case scenario... TeamYouTube

Josh Nathan

45,982 次观看 • 8 个月前