Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

LatchBio CTO Kenny Workman reveals Grok 4.6 tested best in class on several metrics for desirable biosecurity behavior: "We basically have been building independent evaluations to understand dual use behavior of models and have been benchmarking frontier models for the past few months." "There's always a trade-off between the...

MTSlive's profile picture

MTS

500,012 subscribers

12,308 Aufrufe • vor 11 Tagen •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

5 Kommentare

Profilbild von Beff (e/acc)
Beff (e/acc)vor 11 Tagen

@kenbwork biotech Clav is back

Profilbild von Jester
Jestervor 11 Tagen

@kenbwork best in class at biosecurity is a wild leaderboard to be topping but sure, congrats to the model that said no

Profilbild von Islam تايب
Islam تايبvor 11 Tagen

@kenbwork my fav forward deployed researcher @arjunomics

Profilbild von Ricci Research
Ricci Researchvor 11 Tagen

"Independent evaluations" and "we worked with the team to benchmark" are doing slightly different jobs in the same paragraph. The trade-off framing is the right one, but on dual-use specifically, the metric set and the refusal thresholds matter more than the ranking — and neither is in the clip.

Profilbild von newsmartttttttt
newsmarttttttttvor 11 Tagen

@kenbwork the least capable model is the safest? Duh?

Ähnliche Videos

LatchBio CTO Kenny Workman + researcher arjun found evidence Kimi K3 is learning to hack benchmarks by reasoning about graders that don't even exist: Kenny: "We see awareness of the benchmarks we published months ago and trajectories of models today. The open source models are clearly benchmark maxing, benchmark hacking, using knowledge of the context in which an evaluation is constructed to optimize for performance on that thing." Arjun: "We have done various ablations on latent spaces or J spaces in Qwen. Part of the J space includes the start of a MCQ answer, like A, and then parenthesis." "We found a lot of evidence that Kimi K3 reasons about a grader on general biology questions when there's no grader in sight. In a large portion of all of them, we saw reasoning about some grader that doesn't exist." "One of the big defenses is, how do we make these questions a lot more realistic, a lot more normal, a lot more casual to not elicit this behavior, as well as how do we make our sandboxes very secure and very strong and monitorable so that when things do happen, we can stop it, and ensure that never happened in the first place." LatchBio
1:56

LatchBio CTO Kenny Workman + researcher arjun found evidence Kimi K3 is learning to hack benchmarks by reasoning about graders that don't even exist: Kenny: "We see awareness of the benchmarks we published months ago and trajectories of models today. The open source models are clearly benchmark maxing, benchmark hacking, using knowledge of the context in which an evaluation is constructed to optimize for performance on that thing." Arjun: "We have done various ablations on latent spaces or J spaces in Qwen. Part of the J space includes the start of a MCQ answer, like A, and then parenthesis." "We found a lot of evidence that Kimi K3 reasons about a grader on general biology questions when there's no grader in sight. In a large portion of all of them, we saw reasoning about some grader that doesn't exist." "One of the big defenses is, how do we make these questions a lot more realistic, a lot more normal, a lot more casual to not elicit this behavior, as well as how do we make our sandboxes very secure and very strong and monitorable so that when things do happen, we can stop it, and ensure that never happened in the first place." LatchBio

MTS

14,923 Aufrufe • vor 11 Tagen

Vals AI CEO Rayan Krishnan on how they score a model's ability to build its own successor: "One benchmark we released recently is our Recursive Self-Improvement Index. It's a topic which a lot of the big labs have been talking about and starting to report on in their model cards." "There isn't a shared language to talk about the RSI potential of models. So we created this as an apples-to-apples way to actually benchmark across the models." "In an ideal world, what you want to do is take a frontier model and have it train the next version of itself and see where the delta comes from. But obviously that's very expensive and slow." "What we're doing is forming a set of proxies for every part of the process it takes to build the next version of the model." "There's work around pre-training, post-training, harness-level engineering, and then seeing in which mechanisms and behaviors the models are able to do very good research work and build something new, and where they're struggling." Rayan Krishnan Jennifer Li
1:02

Vals AI CEO Rayan Krishnan on how they score a model's ability to build its own successor: "One benchmark we released recently is our Recursive Self-Improvement Index. It's a topic which a lot of the big labs have been talking about and starting to report on in their model cards." "There isn't a shared language to talk about the RSI potential of models. So we created this as an apples-to-apples way to actually benchmark across the models." "In an ideal world, what you want to do is take a frontier model and have it train the next version of itself and see where the delta comes from. But obviously that's very expensive and slow." "What we're doing is forming a set of proxies for every part of the process it takes to build the next version of the model." "There's work around pre-training, post-training, harness-level engineering, and then seeing in which mechanisms and behaviors the models are able to do very good research work and build something new, and where they're struggling." Rayan Krishnan Jennifer Li

a16z

34,366 Aufrufe • vor 4 Tagen

Last week we launched Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency Watch our new video walkthrough to see how to build and run your own benchmark with Optima, and find the best model for your use case Start building your benchmark today at
17:43

Last week we launched Optima, a new platform for benchmarking models on your own workloads and comparing performance, speed and cost efficiency Watch our new video walkthrough to see how to build and run your own benchmark with Optima, and find the best model for your use case Start building your benchmark today at

Artificial Analysis

19,837 Aufrufe • vor 26 Tagen

We just unveiled Grok 4, the world’s smartest artificial intelligence. 🧵 Grok 4 outperforms all other models on the ARC-AGI benchmark, scoring 15.9% - nearly double that of the next best model - and establishing itself as the most intelligent AI to date.
1:22

We just unveiled Grok 4, the world’s smartest artificial intelligence. 🧵 Grok 4 outperforms all other models on the ARC-AGI benchmark, scoring 15.9% - nearly double that of the next best model - and establishing itself as the most intelligent AI to date.

SpaceXAI

8,940,886 Aufrufe • vor 1 Jahr

Why is it that in the past 25 years in Europe we can only point to a handful of companies that have been discovered? Does Europe not realise that they are falling behind so badly? We used to be the leaders of the world and innovation now we have taxed and regulated ourselves into irrelevance… we have been regulated into a joke.
1:12

Why is it that in the past 25 years in Europe we can only point to a handful of companies that have been discovered? Does Europe not realise that they are falling behind so badly? We used to be the leaders of the world and innovation now we have taxed and regulated ourselves into irrelevance… we have been regulated into a joke.

Maila Maria Rosa

25,025 Aufrufe • vor 1 Monat

“Often time, people take oaths and accept such responsibilities, agreeing to them, believing in them, and even proving capable of fulfilling them. However, we know that it sometimes happens that, despite committing to serve the country and Rwandans, and to do everything possible, some people may end up prioritizing their personal interests over their duty. This is a behavior we have been eradicating over the years, a culture that had become deeply rooted in bad politics of the past, which we have been uprooting for the past 30 years.” President Kagame.
1:40

“Often time, people take oaths and accept such responsibilities, agreeing to them, believing in them, and even proving capable of fulfilling them. However, we know that it sometimes happens that, despite committing to serve the country and Rwandans, and to do everything possible, some people may end up prioritizing their personal interests over their duty. This is a behavior we have been eradicating over the years, a culture that had become deeply rooted in bad politics of the past, which we have been uprooting for the past 30 years.” President Kagame.

Presidency | Rwanda

44,181 Aufrufe • vor 1 Jahr

SITUATION EXPLAINED: Grok 4.6 is the best model tested on biosecurity refusals. • LatchBio's BioSecBench-Refusal pairs 61 legitimate research tasks with 46 that conceal a biosecurity hazard inside a realistic scenario • Grok 4.6 is the only model to clear 50% on both red-team refusal and routine answer rates • Opus refuses almost all red teaming but answers routine questions only about 20% of the time • Its safeguards come predominantly from the model's own reasoning rather than an external classifier, which is how most competitors do it • The safeguards produced no degradation on any other benchmark Theo Jaffee: "There's this view that SpaceXAI does not care about AI safety at all. This seems to not be the case. They seem to have done a pretty good job at correctly refusing dangerous queries while also not refusing all queries. So, a more comprehensive outlook than Fable."
6:12

SITUATION EXPLAINED: Grok 4.6 is the best model tested on biosecurity refusals. • LatchBio's BioSecBench-Refusal pairs 61 legitimate research tasks with 46 that conceal a biosecurity hazard inside a realistic scenario • Grok 4.6 is the only model to clear 50% on both red-team refusal and routine answer rates • Opus refuses almost all red teaming but answers routine questions only about 20% of the time • Its safeguards come predominantly from the model's own reasoning rather than an external classifier, which is how most competitors do it • The safeguards produced no degradation on any other benchmark Theo Jaffee: "There's this view that SpaceXAI does not care about AI safety at all. This seems to not be the case. They seem to have done a pretty good job at correctly refusing dangerous queries while also not refusing all queries. So, a more comprehensive outlook than Fable."

MTS

14,608 Aufrufe • vor 12 Tagen

Elon Musk: “We’re making rapid progress on Grok because Grok 4.5 was a huge improvement We have Grok 4.6 coming out probably next week, and then Grok 4.7 is about three or four weeks from today. We expect the cadence of AI development to improve dramatically And then, with Grok 5....which should be out before the end of the year.....we will be incorporating the entire corpus of SpaceX: basically all the data that SpaceX has ever produced That is a tremendous amount over the course of a quarter century, and it will be incorporated into Grok training We think this will make Grok, by far, the best engineer…”
0:47

Elon Musk: “We’re making rapid progress on Grok because Grok 4.5 was a huge improvement We have Grok 4.6 coming out probably next week, and then Grok 4.7 is about three or four weeks from today. We expect the cadence of AI development to improve dramatically And then, with Grok 5....which should be out before the end of the year.....we will be incorporating the entire corpus of SpaceX: basically all the data that SpaceX has ever produced That is a tremendous amount over the course of a quarter century, and it will be incorporated into Grok training We think this will make Grok, by far, the best engineer…”

X Freeze

39,005 Aufrufe • vor 1 Monat

Bill DeWitt Jr.: "We do what we think is best for the team, current and future. ... You have big markets that are just going to do what they do, and we're not in a position to compete at that kind of level of payroll. ... There's a time to do it and a time to build." #STLCards
1:05

Bill DeWitt Jr.: "We do what we think is best for the team, current and future. ... You have big markets that are just going to do what they do, and we're not in a position to compete at that kind of level of payroll. ... There's a time to do it and a time to build." #STLCards

Joey Schneider

23,729 Aufrufe • vor 1 Jahr

Anthropic's CEO Dario Amodei on AI: We have the potential to cure diseases that have been incurable for thousands of years. To radically improve human health. To lift billions out of poverty. To create a better world for everyone. I’m concerned about the autonomous behavior of AI models. Their potential for misuse by individuals and governments. Their potential for economic displacement.
0:39

Anthropic's CEO Dario Amodei on AI: We have the potential to cure diseases that have been incurable for thousands of years. To radically improve human health. To lift billions out of poverty. To create a better world for everyone. I’m concerned about the autonomous behavior of AI models. Their potential for misuse by individuals and governments. Their potential for economic displacement.

Open Source Intel

26,907 Aufrufe • vor 6 Monaten

“I have been working on something for the last many, many months ,objectively in terms of the kind of Kenya we would want to have, because it is the kind of Kenya that we would want to have, that then would dictate what kind of team do we need to do this job,” Kiharu MP Nyoro
2:45

“I have been working on something for the last many, many months ,objectively in terms of the kind of Kenya we would want to have, because it is the kind of Kenya that we would want to have, that then would dictate what kind of team do we need to do this job,” Kiharu MP Nyoro

K24TV

17,577 Aufrufe • vor 2 Monaten

JUST IN: The Trump Administration just announced that they are currently finalizing a solution to the egg shortage crisis and will present it to the president in the next few days. "What we need to do is have better ways with biosecurity and medication and so on to make sure that the perimeter doesn't have to kill the chickens. We have a better, smarter perimeter. And so having a smart perimeter is what we're working on. And we're finalizing the ideas about how to do that with the best scientists in government." - White House Economic Council Director. THERE WE HAVE IT! This administration is coming up with solutions instead of denying the problem & gaslighting the public.
0:57

JUST IN: The Trump Administration just announced that they are currently finalizing a solution to the egg shortage crisis and will present it to the president in the next few days. "What we need to do is have better ways with biosecurity and medication and so on to make sure that the perimeter doesn't have to kill the chickens. We have a better, smarter perimeter. And so having a smart perimeter is what we're working on. And we're finalizing the ideas about how to do that with the best scientists in government." - White House Economic Council Director. THERE WE HAVE IT! This administration is coming up with solutions instead of denying the problem & gaslighting the public.

George

197,768 Aufrufe • vor 1 Jahr

The model (RRFS) that is soon to take over some of the other models we have been used to using- is going nuclear for the Twin Cities on Wednesday. Starting with a heavy round of wind and storms 2-6am? and then 2-8pm “PDS TOR” soundings. Now- other models are more relaxed- somewhat- and it’s important to note that the models have been relatively hot garbage these days. HOWEVER- we are due for our annual banger of a storm. So please stay tuned to the forecast for Wednesday. It looks to be a big day for some of us in the area.
0:40

The model (RRFS) that is soon to take over some of the other models we have been used to using- is going nuclear for the Twin Cities on Wednesday. Starting with a heavy round of wind and storms 2-6am? and then 2-8pm “PDS TOR” soundings. Now- other models are more relaxed- somewhat- and it’s important to note that the models have been relatively hot garbage these days. HOWEVER- we are due for our annual banger of a storm. So please stay tuned to the forecast for Wednesday. It looks to be a big day for some of us in the area.

Bo Cole

36,088 Aufrufe • vor 3 Monaten

A Turkish activist from the Sumud Flotilla: We did not see any humane behavior, unfortunately. We were subjected to severe humiliation. We now better understand what Palestinians have been going through for years. They did not even give us a glass of water. A minister (Ben-Gvir) came and called us terrorists, and we said that the real terrorists are the ones who have been committing this genocide in Gaza for the past two years.
1:18

A Turkish activist from the Sumud Flotilla: We did not see any humane behavior, unfortunately. We were subjected to severe humiliation. We now better understand what Palestinians have been going through for years. They did not even give us a glass of water. A minister (Ben-Gvir) came and called us terrorists, and we said that the real terrorists are the ones who have been committing this genocide in Gaza for the past two years.

Clash Report

261,571 Aufrufe • vor 11 Monaten

What makes Kshama Sawant different is she understands that in order to win victories for working people you have to confront capital: “Under capitalism we didn’t make it a class struggle, it is a class struggle because the capitalist class and their politicians fight to the bitter end for their interests. We have to do the same for the interests of the working class.” She is running as an independent socialist for Congress to take on the capitalist duopoly, end the wars, stop the genocide, and nationalize health care. Let’s support her!
1:01

What makes Kshama Sawant different is she understands that in order to win victories for working people you have to confront capital: “Under capitalism we didn’t make it a class struggle, it is a class struggle because the capitalist class and their politicians fight to the bitter end for their interests. We have to do the same for the interests of the working class.” She is running as an independent socialist for Congress to take on the capitalist duopoly, end the wars, stop the genocide, and nationalize health care. Let’s support her!

Power to the People ☭🕊

18,072 Aufrufe • vor 1 Monat

JD Vance on Netanyahu: Netanyahu governs a country that has obviously been a very close partner of the United States. But even when we've been close partners, sometimes we have interests that are perfectly aligned, and sometimes we have interests that are misaligned. Netanyahu aggressively asserts the interests of his country. Sometimes that means we're on the same page, and sometimes it means that we're not. They've been a great partner in a lot of ways, but we also have to focus on what is in America's best interest. And where that diverges, we unfortunately for the Israelis have to choose the side of the American people, which we always do. Source: CBS News
0:56

JD Vance on Netanyahu: Netanyahu governs a country that has obviously been a very close partner of the United States. But even when we've been close partners, sometimes we have interests that are perfectly aligned, and sometimes we have interests that are misaligned. Netanyahu aggressively asserts the interests of his country. Sometimes that means we're on the same page, and sometimes it means that we're not. They've been a great partner in a lot of ways, but we also have to focus on what is in America's best interest. And where that diverges, we unfortunately for the Israelis have to choose the side of the American people, which we always do. Source: CBS News

Clash Report

97,152 Aufrufe • vor 3 Monaten

Took Grok 4.6 out for a spin this weekend and built this horde shooter prototype on ThreeJS. It is a big step up from Grok 4.5! I've always wanted to try building a horde shooter game with thousands of enemies onscreen and used Grok 4.6 for pretty much everything (including UI, vfx, and even the sfx!), except the 3D models for the characters, which were made and rigged with Meshy. I'm also amazed that ThreeJS with WebGPU can really handle thousands of enemies onscreen at the same time, though I think I pushed the limit here with Depth of Field and some other fancy shaders. Very fun!
0:48

Took Grok 4.6 out for a spin this weekend and built this horde shooter prototype on ThreeJS. It is a big step up from Grok 4.5! I've always wanted to try building a horde shooter game with thousands of enemies onscreen and used Grok 4.6 for pretty much everything (including UI, vfx, and even the sfx!), except the 3D models for the characters, which were made and rigged with Meshy. I'm also amazed that ThreeJS with WebGPU can really handle thousands of enemies onscreen at the same time, though I think I pushed the limit here with Depth of Field and some other fancy shaders. Very fun!

Danny Limanseta

65,105 Aufrufe • vor 28 Tagen

Matt Barnes calls out Stephen A. Smith “You’ve been the same man in front of the camera for the last handful of months disrespecting Serena Williams, disrespecting Jasmine Crockett, disrespecting Kamala Harris, and tap dancing for the whole white Republican party... but you got an issue with LeBron? Do you understand the bulls*** you’re talking and what that can do to the Black community? We all used to have a lot of love for you bro... but we not going to forget the bulls*** you on right now.” (Via All the Smoke )
1:14

Matt Barnes calls out Stephen A. Smith “You’ve been the same man in front of the camera for the last handful of months disrespecting Serena Williams, disrespecting Jasmine Crockett, disrespecting Kamala Harris, and tap dancing for the whole white Republican party... but you got an issue with LeBron? Do you understand the bulls*** you’re talking and what that can do to the Black community? We all used to have a lot of love for you bro... but we not going to forget the bulls*** you on right now.” (Via All the Smoke )

NBA Base

41,555 Aufrufe • vor 5 Monaten