Video yükleniyor...

Video Yüklenemedi

Bu video yüklenirken bir sorun oluştu. Bu geçici bir ağ sorunundan kaynaklanıyor olabilir veya video kullanılamıyor olabilir.

Ana Sayfaya Dön

Today we are open-sourcing boundary bench, a new paper, benchmark, and GitHub repo that enterprises can use to find out the REAL performance of their agents. Boundary-Bench was developed by researchers from Accomplish and NYU, where we tested 12 frontier agents across roughly 10,000 runs, with realistic enterprise policies,... show more

Or Hiltch

19,492 subscribers

75,573 görüntüleme • 1 ay önce •via X (Twitter)

Bilim & Teknoloji Eğitim Haberler & Politika

Anya Rossi• Live Now

Private livecam show

37 Yorum

Or Hiltch profil fotoğrafı

Or Hiltch1 ay önce

A few highlights from the paper: * Coding agents cost approximately 40% more on average in restricted environments. * @AnthropicAI based agents achieved success rates similar to @OpenAI agents, but cost approximately 7x more (!!) in restricted and hardened environments. * @SpaceXAI @grok 4.5 / Grok Build emerges as the most persistent model - it fights to the death until it succeeds, pivoting around until a trajectory for success is found. This, however, comes at significant token cost. * @claudeai Sonnet 5 is the opposite: modest cost increase, but the biggest success loss.

Or Hiltch profil fotoğrafı

Or Hiltch1 ay önce

Provider safety classifiers are quietly failing enterprise runs: both @AnthropicAI and @OpenAI run safety classifiers inside their serving layers - between the agent harness and the model. These classifiers scan request content, and when one fires on a coding task (usually security-themed work: the paper names tasks like vulnerable-secret, break-filter-js-from-html, and a differential-cryptanalysis task), it can kill the model stream mid-trajectory. The agent is halfway through legitimate benchmark work and the provider just... stops serving it.

Or Hiltch profil fotoğrafı

Or Hiltch1 ay önce

You might ask, what are these policies? A policy restricts three things: which internet destinations the environment can reach, which files it can write, and which privileges it holds. Enforcement is native Linux, not agent instructions: egress passes through a proxy the agent cannot reconfigure, system files are mounted read-only, and the agent runs as a user that cannot escalate. The agent is never told about the limits, and every child process it spawns inherits them. The three levels form a cumulative ladder, each adding restrictions on top of the previous one.

Or Hiltch profil fotoğrafı

Or Hiltch1 ay önce

We define three policy levels, and you can create your own using the policy builder: The levels are: Control is the open baseline: administrator rights, unrestricted internet, a writable filesystem. It is the setup every other benchmark reports. Non-root drops administrator rights and nothing else, isolating what losing privilege alone costs. Network and filesystem stay open. NIST-derived high tightens all three axes at once: internet limited to a fixed egress allowlistA list of approved destinations, such as package registries, APIs, and the model endpoint, that the environment is allowed to reach. Anything not on the list is blocked. The list is set per task., system files read-only outside a small writable workspace, and no path back to elevated privileges. The restrictions are derived from NIST SP 800-53A widely used U.S. government catalog of security controls that many organizations base their policies on. Our levels are derived from it.

Or Hiltch profil fotoğrafı

Or Hiltch1 ay önce

Check it out and let us know what you think! Boundary-Bench: Repo: Paper:

Amit Avner profil fotoğrafı

Amit Avner1 ay önce

@boundarybench @Accomplish_ai It's a crazy stat that agents cost approximately 40% more in restricted environments!

Yuval Naor profil fotoğrafı

Yuval Naor1 ay önce

@boundarybench @Accomplish_ai 🤯 this is so interesting. Ever witnessed an agent trying to implement a PE vulnerability to escalate itself and override its boundaries?

Uri Tzadikevitch profil fotoğrafı

Uri Tzadikevitch1 ay önce

@boundarybench @Accomplish_ai Super interesting - Grok 4.5 held up best of any coding agent under real restrictions: only 7.1% success lost, ahead of Fable 5 and GPT-5.6 Sol. The catch is it paid the steepest cost increase in the field, 167%. It keeps fighting the policy where others fold.

Itay Romanowski profil fotoğrafı

Itay Romanowski1 ay önce

@boundarybench @Accomplish_ai The biggest takeaway for me isn’t which model wins, but how much real enterprise environments reshape both cost and performance. Great work.

Yair Amar profil fotoğrafı

Yair Amar1 ay önce

@boundarybench @Accomplish_ai Coolest part is the non-uniformity of the change in cost and success rate! GPT suite seems quite robust all across the cost curve!

Jonathan Jacobi profil fotoğrafı

Jonathan Jacobi1 ay önce

@boundarybench @Accomplish_ai congrats!

Erica Brescia profil fotoğrafı

Erica Brescia1 ay önce

@boundarybench @Accomplish_ai Very exciting. Congrats!

Eliran Goldsteine profil fotoğrafı

Eliran Goldsteine1 ay önce

@boundarybench @Accomplish_ai Extremely interesting. As usual, you guys are way ahead of the industry, setting up the benchmark!

Lightspeed profil fotoğrafı

Lightspeed1 ay önce

@boundarybench @Accomplish_ai Congratulations on the launch, team!

Omri Haviv profil fotoğrafı

Omri Haviv1 ay önce

@boundarybench @Accomplish_ai Interesting to see your data so closely correlated to online sentiment per model (e.g. sonnet and grok being polar opposites)

urik profil fotoğrafı

urik1 ay önce

@boundarybench @Accomplish_ai Inside view

Danny Karmon profil fotoğrafı

Danny Karmon1 ay önce

@boundarybench @Accomplish_ai Fascinating results! Great work!

idan levin profil fotoğrafı

idan levin1 ay önce

@boundarybench @Accomplish_ai Very cool!

Michael H profil fotoğrafı

Michael H1 ay önce

@boundarybench @Accomplish_ai More relevant than ever now that managed Codex and Claude Code are becoming more popular.

Yair Graif profil fotoğrafı

Yair Graif1 ay önce

@boundarybench @Accomplish_ai Super interesting!

Sharone Revah Zitzman 💙🇮🇱 profil fotoğrafı

Sharone Revah Zitzman 💙🇮🇱1 ay önce

@boundarybench @Accomplish_ai 🥳🥳👏👏 What an all-star team you've built! Congrats on the launch - I will be following you closely!

Olli profil fotoğrafı

Olli1 ay önce

@boundarybench @Accomplish_ai This makes agent benchmarks more honest. The environment is part of the product: network rules, file permissions, root access and cleanup. If a model only looks strong in a permissive sandbox, that is not the model most companies are buying.

Kunal Kushwaha profil fotoğrafı

Kunal Kushwaha1 ay önce

@boundarybench @Accomplish_ai Congrats!

Miki Makhlevich profil fotoğrafı

Miki Makhlevich1 ay önce

@boundarybench @Accomplish_ai awesome!

Vivek Gopalan profil fotoğrafı

Vivek Gopalan1 ay önce

@boundarybench @Accomplish_ai This is really important for enterprise agents!

Hedwigz profil fotoğrafı

Hedwigz1 ay önce

@boundarybench @Accomplish_ai MIND=BLOW 🤯

Peleg Aran profil fotoğrafı

Peleg Aran1 ay önce

@boundarybench @Accomplish_ai Wow! Super interesting!

Matt Van Horn profil fotoğrafı

Matt Van Horn1 ay önce

@boundarybench @Accomplish_ai nice!

Roygold profil fotoğrafı

Roygold1 ay önce

@boundarybench @Accomplish_ai Awesome!

Gerard Sans | Axiom 🇬🇧 profil fotoğrafı

Gerard Sans | Axiom 🇬🇧1 ay önce

@boundarybench @Accomplish_ai

Matan Zutta profil fotoğrafı

Matan Zutta1 ay önce

@boundarybench @Accomplish_ai Nice!!!

Yair Weinberger profil fotoğrafı

Yair Weinberger1 ay önce

@boundarybench @Accomplish_ai Interesting! Which harness did you use for the benchmarks?

Zongi 💰 profil fotoğrafı

Zongi 💰1 ay önce

@boundarybench @Accomplish_ai @threadreaderapp unroll

Fajar M Reza profil fotoğrafı

Fajar M Reza1 ay önce

@boundarybench @Accomplish_ai Does Boundary-Bench measure autonomous judgment, or mostly tool reliability under pressure?

Ismael Hishon-Rezaizadeh ⭕ profil fotoğrafı

Ismael Hishon-Rezaizadeh ⭕1 ay önce

@boundarybench @Accomplish_ai the timing on this is something. shipping a benchmark for what agents do with the guardrails on the same morning AISI published what happened when it turned its own off. did any of your twelve cheat in a way the run logs alone would not have caught?

Manpreet Bola profil fotoğrafı

Manpreet Bola1 ay önce

The thread lists these as separate facts. The paper says they are on trade-off. Grok 4.5 kept working best but cost inflation hit 167%, highest by far. And Claude Sonnet 5 lost most success (-18.3pp) at just +21% cost. The axes disagreee here. You pick which pain you can afford, then the model.

Emad Ghorbaninia profil fotoğrafı

Emad Ghorbaninia1 ay önce

@boundarybench @Accomplish_ai The 167% cost increase is the real headline here, not the success-rate drop. Everyone budgets agents assuming today's wide-open sandbox economics. Lock down egress and filesystem access like an actual security team would, and the unit economics change completely.

Benzer Videolar

🚀We are excited to introduce the Tool Decathlon (Toolathlon), a benchmark for language agents on diverse, complex, and realistic tool use. ⭐️32 applications and 600+ tools based on real-world software environments ⭐️Execution-based, reliable evaluation ⭐️Realistic, covering daily and professional scenarios Toolathlon reveals significant shortcomings of SOTA LLMs in realistic tool-use tasks, where Claude Sonnet 4.5 achieves 38.6% success rate. It also indicates a clear gap between open-source and leading proprietary models. Check our blog: Github: Paper: 🧵⬇️

🚀We are excited to introduce the Tool Decathlon (Toolathlon), a benchmark for language agents on diverse, complex, and realistic tool use. ⭐️32 applications and 600+ tools based on real-world software environments ⭐️Execution-based, reliable evaluation ⭐️Realistic, covering daily and professional scenarios Toolathlon reveals significant shortcomings of SOTA LLMs in realistic tool-use tasks, where Claude Sonnet 4.5 achieves 38.6% success rate. It also indicates a clear gap between open-source and leading proprietary models. Check our blog: Github: Paper: 🧵⬇️

Junxian He

44,403 görüntüleme • 10 ay önce

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.

Earlier this week at GTC, we announced our partnership with Nvidia. We will work with Nvidia to build strong, American open-source models that are at the frontier of scientific reasoning. These models will be essential for the US to compete with China on science in the coming decades. Jensen is committing to spend tens of billions of dollars developing open-source models, and we are excited to be a partner with them in figuring out how to benchmark, train and use those agents to accelerate scientific research. We have already open-sourced some of the work we have done with them, and are looking forward to open-sourcing more. There are few things today that are more important. See our blog post below, and watch the video to learn more, narrated by the man himself.

Sam Rodriques

24,338 görüntüleme • 5 ay önce

we’re launching Codag (Y Combinator S26) - the compression and control layer for agent tools. we started with logs. then customers pulled us into searches, tests, builds, and api responses. the pattern was always the same: agents burning context on repetitive output, with teams unable to see where the tokens went. Codag now cuts input-token usage ~3x at answer parity and gives teams visibility across their agent stack. agents are becoming the primary users of software - companies are adding agents, models, providers, and MCP servers faster than they can measure or control them. most cannot answer which tools their agents invoke, what those calls cost, or where the waste is. we’re building the layer between them and everything they read. 90+ engineering orgs already use Codag today to reduce that spend and see agent tool usage across provider. find out more and book a demo at

we’re launching Codag (Y Combinator S26) - the compression and control layer for agent tools. we started with logs. then customers pulled us into searches, tests, builds, and api responses. the pattern was always the same: agents burning context on repetitive output, with teams unable to see where the tokens went. Codag now cuts input-token usage ~3x at answer parity and gives teams visibility across their agent stack. agents are becoming the primary users of software - companies are adding agents, models, providers, and MCP servers faster than they can measure or control them. most cannot answer which tools their agents invoke, what those calls cost, or where the waste is. we’re building the layer between them and everything they read. 90+ engineering orgs already use Codag today to reduce that spend and see agent tool usage across provider. find out more and book a demo at

Michael Zhou

163,206 görüntüleme • 27 gün önce

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

362,795 görüntüleme • 2 ay önce

🚨BREAKING: An ICE agent says they’re being pulled off child predator cases to hit immigration quotas instead. In this phone call, an HSI Special Response Team agent… who works child exploitation cases… says agents are being pulled off investigations into predators, to go round up immigrants instead. He says it outright: The government is “letting legit criminal investigations go by the wayside.” Why? Because leadership is demanding numbers, and they don’t care who. He explains that with policies pushed by Stephen Miller, agents are being forced to work hand in hand with ERO to hit immigration quotas, instead of doing the job they were trained for. And the number of agents being reallocated are staggering… Out of a child exploitation unit of 8 to 10 agents, 6 to 7 are reassigned… leaving just two agents to investigate crimes against children. So, while DHS goes on TV claiming these immigration surges are helping fight crime, protect Americans, and cracking down on predators… Their own agents are saying the exact opposite. They’re not going after the “worst of the worst”… they’re stripping resources from the very investigations designed to find them. And while agents are busy chasing quotas, real predators are left to continue preying on children. And here’s where it gets even darker… We’ve seen with the Epstein files: abuse is ignored, predators are protected, and investigations are stalled, while children continued to be trafficked… But now, under Donald Trump, this isn’t just negligence… it’s a government-run operation. Agents are pulled from investigating real predators to hunt parents and children… children who are then trapped in detention centers where the government profits off their captivity. Meanwhile, the kids, who are U.S. citizens, are separated from their parents, and left exposed to trafficking and abuse. The system that claims to protect children is, in fact, exploiting them.

🚨BREAKING: An ICE agent says they’re being pulled off child predator cases to hit immigration quotas instead. In this phone call, an HSI Special Response Team agent… who works child exploitation cases… says agents are being pulled off investigations into predators, to go round up immigrants instead. He says it outright: The government is “letting legit criminal investigations go by the wayside.” Why? Because leadership is demanding numbers, and they don’t care who. He explains that with policies pushed by Stephen Miller, agents are being forced to work hand in hand with ERO to hit immigration quotas, instead of doing the job they were trained for. And the number of agents being reallocated are staggering… Out of a child exploitation unit of 8 to 10 agents, 6 to 7 are reassigned… leaving just two agents to investigate crimes against children. So, while DHS goes on TV claiming these immigration surges are helping fight crime, protect Americans, and cracking down on predators… Their own agents are saying the exact opposite. They’re not going after the “worst of the worst”… they’re stripping resources from the very investigations designed to find them. And while agents are busy chasing quotas, real predators are left to continue preying on children. And here’s where it gets even darker… We’ve seen with the Epstein files: abuse is ignored, predators are protected, and investigations are stalled, while children continued to be trafficked… But now, under Donald Trump, this isn’t just negligence… it’s a government-run operation. Agents are pulled from investigating real predators to hunt parents and children… children who are then trapped in detention centers where the government profits off their captivity. Meanwhile, the kids, who are U.S. citizens, are separated from their parents, and left exposed to trafficking and abuse. The system that claims to protect children is, in fact, exploiting them.

Jesus Freakin Congress

165,443 görüntüleme • 5 ay önce

We are engaging with countries in the Middle East and the Gulf, which now show a strong interest in Ukraine’s experience in defending against drones. We already see that not only “shaheds” are being used in the region, but there is also growing evidence of the use of FPV drones. This is modern warfare, and everyone must be prepared for it. Ukraine has this expertise, and in exchange for our support we need help in areas where we face greater challenges. This includes protection against ballistic threats and financial resources for defense. Ukraine offers a mutually beneficial partnership: we can strengthen those who can strengthen us. The situation in the world now is such that only coordinated and joint actions can guarantee real results and genuine security. We are being blocked in Europe, and as long as this risk remains, we must seek out additional opportunities to strengthen ourselves. The Middle East and the Gulf are the right direction and serious opportunities to make Ukraine stronger.

We are engaging with countries in the Middle East and the Gulf, which now show a strong interest in Ukraine’s experience in defending against drones. We already see that not only “shaheds” are being used in the region, but there is also growing evidence of the use of FPV drones. This is modern warfare, and everyone must be prepared for it. Ukraine has this expertise, and in exchange for our support we need help in areas where we face greater challenges. This includes protection against ballistic threats and financial resources for defense. Ukraine offers a mutually beneficial partnership: we can strengthen those who can strengthen us. The situation in the world now is such that only coordinated and joint actions can guarantee real results and genuine security. We are being blocked in Europe, and as long as this risk remains, we must seek out additional opportunities to strengthen ourselves. The Middle East and the Gulf are the right direction and serious opportunities to make Ukraine stronger.

Volodymyr Zelenskyy / Володимир Зеленський

566,996 görüntüleme • 5 ay önce

[WATCH] IN THE INTEREST OF JUSTICE AND DEMOCRATIC PARTICIPATION. In the interest of democratic participation and to allow our communities to exercise their fair choice in selecting those they believe will be the best public representatives. Those communities affected we believe should be given the opportunity to participate in the democratic processes. It is not harmful to democracy to allow the investigation into the technical issues to take place. On of the major changes is how the ANC manages the candidate selection process is that there are new requirements, the criteria we use now is much more stringent to ensure that those we send to contest elections are qualifying and wanted by their communities, both in ethical conduct, skills and competence. What that means is that ANC processes of candidates selection are very laborious and we are the first to admit that they are laborious because we involve over a period of time many stakeholders including the community, to those who say why we left things to the eleventh hour we can say categorically we did not. We want to assure our members and communities that those communities and municipalities affected we are dealing with this issue and do find justice in the process and that they will exercise their democratic right. That is why we are asking for a constructive solution with the independent electoral commission and we do believe they will open the door to engagement, it is a Chapter 9 institution it is duty bound to find a solution as long as that solution does not break the law. We are not asking for any changes in our list, we are asking for that portion that was kicked out to be investigated.

[WATCH] IN THE INTEREST OF JUSTICE AND DEMOCRATIC PARTICIPATION. In the interest of democratic participation and to allow our communities to exercise their fair choice in selecting those they believe will be the best public representatives. Those communities affected we believe should be given the opportunity to participate in the democratic processes. It is not harmful to democracy to allow the investigation into the technical issues to take place. On of the major changes is how the ANC manages the candidate selection process is that there are new requirements, the criteria we use now is much more stringent to ensure that those we send to contest elections are qualifying and wanted by their communities, both in ethical conduct, skills and competence. What that means is that ANC processes of candidates selection are very laborious and we are the first to admit that they are laborious because we involve over a period of time many stakeholders including the community, to those who say why we left things to the eleventh hour we can say categorically we did not. We want to assure our members and communities that those communities and municipalities affected we are dealing with this issue and do find justice in the process and that they will exercise their democratic right. That is why we are asking for a constructive solution with the independent electoral commission and we do believe they will open the door to engagement, it is a Chapter 9 institution it is duty bound to find a solution as long as that solution does not break the law. We are not asking for any changes in our list, we are asking for that portion that was kicked out to be investigated.

ANC SECRETARY GENERAL | Fikile Mbalula

66,166 görüntüleme • 11 gün önce

Mamdani: We want to make it clear that our city does not simply oppose the actions of ICE, but that we are not going to be complicit with those actions either. And that means making sure that every single city agency is in compliance with our sanctuary city policies. Now, if you don’t know what a sanctuary city policy is, I’ll describe it this way: it means that we do not allow ICE agents into our schools, our hospitals, our city property unless they can provide a judicial warrant signed by a judge. It means that we don’t assist ICE with their civil immigration enforcement. It means that we make clear that we are proud of our heritage as an immigrant city.

Mamdani: We want to make it clear that our city does not simply oppose the actions of ICE, but that we are not going to be complicit with those actions either. And that means making sure that every single city agency is in compliance with our sanctuary city policies. Now, if you don’t know what a sanctuary city policy is, I’ll describe it this way: it means that we do not allow ICE agents into our schools, our hospitals, our city property unless they can provide a judicial warrant signed by a judge. It means that we don’t assist ICE with their civil immigration enforcement. It means that we make clear that we are proud of our heritage as an immigrant city.

Acyn

357,739 görüntüleme • 2 ay önce

How well do today’s frontier models handle long-horizon, multi-step web agent tasks, such as identifying the top 25 U.S. CS PhD programs with ML/AI faculty likely accepting students and compiling the results into a structured sheet? Check out our new work on Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks Paper: Leaderboard: We introduce Odysseys, a benchmark of 200 long-horizon tasks derived from real browsing sessions and evaluated on the live Internet. We show that binary pass/fail is inadequate in this setting and propose rubric-based evaluation, which better aligns with human judgment and provides more informative signals. Across leading models, the best achieves only 44.5% success, leaving substantial headroom. We further introduce a Trajectory Efficiency metric (rubric score per step) and find efficiency remains extremely low (1.15%), highlighting a key bottleneck. Odysseys provides a realistic benchmark for measuring progress toward web agents capable of sustained, efficient, real-world operation. See a more detailed thread by Jing Yu Koh.

How well do today’s frontier models handle long-horizon, multi-step web agent tasks, such as identifying the top 25 U.S. CS PhD programs with ML/AI faculty likely accepting students and compiling the results into a structured sheet? Check out our new work on Odysseys: Benchmarking Web Agents on Realistic Long Horizon Tasks Paper: Leaderboard: We introduce Odysseys, a benchmark of 200 long-horizon tasks derived from real browsing sessions and evaluated on the live Internet. We show that binary pass/fail is inadequate in this setting and propose rubric-based evaluation, which better aligns with human judgment and provides more informative signals. Across leading models, the best achieves only 44.5% success, leaving substantial headroom. We further introduce a Trajectory Efficiency metric (rubric score per step) and find efficiency remains extremely low (1.15%), highlighting a key bottleneck. Odysseys provides a realistic benchmark for measuring progress toward web agents capable of sustained, efficient, real-world operation. See a more detailed thread by Jing Yu Koh.

Russ Salakhutdinov

22,518 görüntüleme • 4 ay önce

Wow, Alejandro Mayorkas suggests that the Biden administration has not allowed him to secure the border: "People have differing opinions on what the right policies and operational measures should be. These differences are discussed, decisions are made, and then we all move forward together in unity." "We are one team with a unified effort. It's not a surprising idea that there would be a variety of opinions when many people are involved in making decisions." The fallout from the chaotic Biden administration appears to be emerging as each person looks out for themselves. Leave your thoughts in the comments below.

Wow, Alejandro Mayorkas suggests that the Biden administration has not allowed him to secure the border: "People have differing opinions on what the right policies and operational measures should be. These differences are discussed, decisions are made, and then we all move forward together in unity." "We are one team with a unified effort. It's not a surprising idea that there would be a variety of opinions when many people are involved in making decisions." The fallout from the chaotic Biden administration appears to be emerging as each person looks out for themselves. Leave your thoughts in the comments below.

𝐃𝐔𝐓𝐂𝐇

18,127 görüntüleme • 1 yıl önce

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Base Labs

229,120 görüntüleme • 11 gün önce

Next week will see more international work, including those with our EU partners. We will increase activity. This is also true for security commitments from partners. We are putting together new, strong bilateral agreements. January and February must produce more results in this area. We can already see specific dates when such new and powerful documents may be signed. I am grateful to our entire team that participates in the relevant negotiations, as well as all leaders and nations that are prepared to take truly ambitious steps. By developing such a new security commitment architecture, we are effectively reanimating the system of international law. And when justice and security are restored in Ukraine, this will also work for the rest of the world. We are also working with key partners on specific military aid packages that will be delivered right now, in the coming weeks and months. I thank all of Ukraine's friends around the world who understand that the battlefield cannot simply wait and that life-saving efforts cannot be postponed, as well as those who work to expedite decisions that are needed right now.

Next week will see more international work, including those with our EU partners. We will increase activity. This is also true for security commitments from partners. We are putting together new, strong bilateral agreements. January and February must produce more results in this area. We can already see specific dates when such new and powerful documents may be signed. I am grateful to our entire team that participates in the relevant negotiations, as well as all leaders and nations that are prepared to take truly ambitious steps. By developing such a new security commitment architecture, we are effectively reanimating the system of international law. And when justice and security are restored in Ukraine, this will also work for the rest of the world. We are also working with key partners on specific military aid packages that will be delivered right now, in the coming weeks and months. I thank all of Ukraine's friends around the world who understand that the battlefield cannot simply wait and that life-saving efforts cannot be postponed, as well as those who work to expedite decisions that are needed right now.

Volodymyr Zelenskyy / Володимир Зеленський

348,867 görüntüleme • 2 yıl önce

.Haseeb Qureshi ＞|＜ calls any team not running its releases through frontier-model security scanning "delinquent": "We are in a world where the previous rules around cybersecurity no longer apply. Because the reality is, if you can find this bug for $2, the team that was building this also could have done security scanning for $2. What were they doing?" "GLM has been out for a while. Kimi obviously more recent, but even just doing this on Opus would have cost probably four or five times that at most. Supposedly somebody ran Claude Code and found this in two minutes." "So what all this tells you is that there are companies that are going to survive this transition, and there are companies that are not. And there's going to be this kind of COVID-like mass inoculation period that we're going through right now in cybersecurity." "It's probably going to be a very centralizing force, because it means there's gonna be more consolidation toward the companies that have good security practices, have enough money to spend real dollars on security engineering, and are gonna be running every release through thousands of dollars of security scanning from frontier-level models." "If you are not doing that now, you are delinquent. You are already behind. You are putting your users and your company at risk." "AI is kind of an everything technology. It's like this background radiation now that is just touching everything that is software based." Dragonfly ＞|＜

.Haseeb Qureshi ＞|＜ calls any team not running its releases through frontier-model security scanning "delinquent": "We are in a world where the previous rules around cybersecurity no longer apply. Because the reality is, if you can find this bug for $2, the team that was building this also could have done security scanning for $2. What were they doing?" "GLM has been out for a while. Kimi obviously more recent, but even just doing this on Opus would have cost probably four or five times that at most. Supposedly somebody ran Claude Code and found this in two minutes." "So what all this tells you is that there are companies that are going to survive this transition, and there are companies that are not. And there's going to be this kind of COVID-like mass inoculation period that we're going through right now in cybersecurity." "It's probably going to be a very centralizing force, because it means there's gonna be more consolidation toward the companies that have good security practices, have enough money to spend real dollars on security engineering, and are gonna be running every release through thousands of dollars of security scanning from frontier-level models." "If you are not doing that now, you are delinquent. You are already behind. You are putting your users and your company at risk." "AI is kind of an everything technology. It's like this background radiation now that is just touching everything that is software based." Dragonfly ＞|＜

MTS

12,086 görüntüleme • 1 ay önce

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Mati Staniszewski on why ElevenLabs is betting on a cascaded approach for voice agents: “Our approach, as you think about voice agents, conversational agents, is effectively a cascaded approach. You use transcription or speech-to-text, an LLM, text-to-speech, and orchestrate all of that together. Then you have speech-to-speech, which goes directly from speech, and there’s a speech response on the other side. Today, we are optimizing heavily on a cascaded approach. As we work with a lot of the businesses and enterprises, they will need that visibility into what happens. They will want to execute certain tasks on top of that. They want good visibility into each of the steps and great accuracy of all the models. But beyond that, they can abstract away what’s the LLM layer, what’s the intelligence layer, and the integrations are easier in that system. That’s where we are betting a lot of the research work on how you can make that great, and we think we can make that great.” John Collison @matiii ElevenLabs

Stripe

18,535 görüntüleme • 4 ay önce

Today we are introducing 2 key features to JAIGP: AI Review & Open Prompting. AI review is part of 5-step process where papers get feedback from & are evaluated based on their ability to address that feedback. This means they are not stuck in an endless AI review loop. Open prompting is a newer idea. We are making all prompts we used to create JAIGP open, and also, opening up the journal's rules to the community for suggestions. You can suggest the prompt we should run next! So, if you have opinions about AI generated papers, you can share them directly with us at

Today we are introducing 2 key features to JAIGP: AI Review & Open Prompting. AI review is part of 5-step process where papers get feedback from & are evaluated based on their ability to address that feedback. This means they are not stuck in an endless AI review loop. Open prompting is a newer idea. We are making all prompts we used to create JAIGP open, and also, opening up the journal's rules to the community for suggestions. You can suggest the prompt we should run next! So, if you have opinions about AI generated papers, you can share them directly with us at

César A. Hidalgo

13,453 görüntüleme • 5 ay önce

🚨 BREAKING: DHS is now launching CRIMINAL INVESTIGATIONS into the FUNDERS of anti-ICE leftists and obstructionists, "CRIMINAL CONSPIRACY" charges are being floated Good! ARREST THEM ALL! 🔥 ICE DIRECTOR LYONS: "There is an active group CRIMINALLY BASED that is impeding these operations." "Homeland Security Investigations and the FBI, we're actively investigating many groups all across the U.S. now that are actively targeting ICE officers and agents and other federal partners in their day-to-day mission." "So we do believe there's a criminal conspiracy going on. We are working with all of our federal partners to try to track down the funding, the leadership of these groups, because we would see time and time again ICE officers and agents would be doing an operation, and within seconds to minutes, all of a sudden, there's 50, 60 protesters on that site!"

🚨 BREAKING: DHS is now launching CRIMINAL INVESTIGATIONS into the FUNDERS of anti-ICE leftists and obstructionists, "CRIMINAL CONSPIRACY" charges are being floated Good! ARREST THEM ALL! 🔥 ICE DIRECTOR LYONS: "There is an active group CRIMINALLY BASED that is impeding these operations." "Homeland Security Investigations and the FBI, we're actively investigating many groups all across the U.S. now that are actively targeting ICE officers and agents and other federal partners in their day-to-day mission." "So we do believe there's a criminal conspiracy going on. We are working with all of our federal partners to try to track down the funding, the leadership of these groups, because we would see time and time again ICE officers and agents would be doing an operation, and within seconds to minutes, all of a sudden, there's 50, 60 protesters on that site!"

Eric Daugherty

24,113 görüntüleme • 8 ay önce

Today, we had a detailed discussion with Steve Witkoff and Jared Kushner, joined by Lindsey Graham and Mark Rutte, on how to strengthen diplomacy, what steps are possible, and on security guarantees and the involvement of Europeans. It was a positive conversation. Thank you, Steve, Jared, Lindsey, and Mark. We agreed to strengthen security guarantees, and I have already instructed our team to promptly update the documents so that the security guarantees for Ukraine are strong, the prospects for post-war reconstruction are real, and everything is doable. Everything must be kept current. We see how the world is changing – how much can shift in a single moment. This concerns both alliances and security challenges alike. Ukraine must receive reliable security: this is the main task. I am grateful that the U.S. President’s envoys hear us and are ready to work together to find the right compromises, steps, and decisions. The key lies in how and what all of us discuss with Russia, and in how the documents are prepared, particularly those on security guarantees. So that Ukrainians clearly understand how partners will respond in the event of renewed Russian aggression, and what will actually deter Russia from aggression. We need strong, shared positions, and Ukraine’s contribution to this strength is unquestionable. We continue working with America. I thank everyone who is helping. I expect that the teams will work substantively in the coming days so that we can all feel progress. A trilateral format – a leaders’ format – all of this is necessary.

Today, we had a detailed discussion with Steve Witkoff and Jared Kushner, joined by Lindsey Graham and Mark Rutte, on how to strengthen diplomacy, what steps are possible, and on security guarantees and the involvement of Europeans. It was a positive conversation. Thank you, Steve, Jared, Lindsey, and Mark. We agreed to strengthen security guarantees, and I have already instructed our team to promptly update the documents so that the security guarantees for Ukraine are strong, the prospects for post-war reconstruction are real, and everything is doable. Everything must be kept current. We see how the world is changing – how much can shift in a single moment. This concerns both alliances and security challenges alike. Ukraine must receive reliable security: this is the main task. I am grateful that the U.S. President’s envoys hear us and are ready to work together to find the right compromises, steps, and decisions. The key lies in how and what all of us discuss with Russia, and in how the documents are prepared, particularly those on security guarantees. So that Ukrainians clearly understand how partners will respond in the event of renewed Russian aggression, and what will actually deter Russia from aggression. We need strong, shared positions, and Ukraine’s contribution to this strength is unquestionable. We continue working with America. I thank everyone who is helping. I expect that the teams will work substantively in the coming days so that we can all feel progress. A trilateral format – a leaders’ format – all of this is necessary.

Volodymyr Zelenskyy / Володимир Зеленський

360,290 görüntüleme • 5 ay önce

We miss you Papa J- “Let me just give you one point. The NPP cannot be underestimated, you know it and I know it. They’ve been in existence for a long time, and have been changing their names up till this day. The nature of who they are, and what they are and where they’ve come from, it’s something they are happy with and they will be there for a long time.” The calculation I want to put before you is that, by virtue of who and what they are, their mannerisms, their character, will necessarily give rise to what you and I are in the NDC. In other words, by virtue of who we are; where we’ve come from, how we came about into being, I keep saying, we cannot afford to fight them on their terms.” He admonished that; “It’s important than we fight them on our terms. What are those terms? In other words, let’s not mimic their behaviour. Let us reach into the principles and the values they emerged out of the circumstances that gave birth to us. And if we can hold onto those principles and those values, we will move way into the future and no one can ever defeat us. We will ride and override any obstacle as we did in the past. Mimic them as some of them are doing and we will run into problems.NPP NEVER 👎 AGAIN GHANA 🇬🇭 SHALL RISE AGAIN

We miss you Papa J- “Let me just give you one point. The NPP cannot be underestimated, you know it and I know it. They’ve been in existence for a long time, and have been changing their names up till this day. The nature of who they are, and what they are and where they’ve come from, it’s something they are happy with and they will be there for a long time.” The calculation I want to put before you is that, by virtue of who and what they are, their mannerisms, their character, will necessarily give rise to what you and I are in the NDC. In other words, by virtue of who we are; where we’ve come from, how we came about into being, I keep saying, we cannot afford to fight them on their terms.” He admonished that; “It’s important than we fight them on our terms. What are those terms? In other words, let’s not mimic their behaviour. Let us reach into the principles and the values they emerged out of the circumstances that gave birth to us. And if we can hold onto those principles and those values, we will move way into the future and no one can ever defeat us. We will ride and override any obstacle as we did in the past. Mimic them as some of them are doing and we will run into problems.NPP NEVER 👎 AGAIN GHANA 🇬🇭 SHALL RISE AGAIN

Aristotlektv

20,744 görüntüleme • 1 yıl önce

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Historically, RL policies for robots have been trained in synthetic, untextured environments, limiting perceptive policies to depth images where the sim-to-real gap is manageable. RGB has always had more potential, but leveraging it to train policies in simulation remained an open problem. Partnering with Niantic Spatial 🌎 and NVIDIA Robotics, we built a pipeline that addresses exactly that. We can now scan a real deployment site with off-the-shelf hardware, reconstruct it into a photorealistic Gaussian splat, and run massively parallel RL training. The policies trained in our Gym environment then transfer zero-shot to the real robot and environments they were trained for. This enables faster deployment of more capable and robust policies for the end user. The new resulting capabilities are a big step towards solving sim-to-real and also apply well beyond navigation. Read the full technical breakdown on our blog; link in the comments. #HumanoidRobots #Flexion #NianticSpatial #NVIDIA

Flexion

72,555 görüntüleme • 1 ay önce

Today, we are launching the first publicly available AI Scientist, via the FutureHouse Platform. Our AI Scientist agents can perform a wide variety of scientific tasks better than humans. By chaining them together, we've already started to discover new biology really fast. With the platform, we are bringing these capabilities to the wider community. Watch our long-form video, in the comments below, to learn more about how the platform works and how you can use it to make new discoveries, and go to our website or see the comments below to access the platform. We are releasing three superhuman AI Scientist agents today, each with their own specialization: A general-purpose agent (Crow); An agent to automate literature reviews (Falcon); and An agent to answer the question “Has anyone done X before” (Owl). We are also releasing an experimental agent, Phoenix, that has access to a wide variety of tools for planning experiments in chemistry. More on that below. The three literature search agents (Crow, Falcon, and Owl) have benchmarked superhuman performance. They also have access to a large corpus of full scientific texts, which means that you can ask them more detailed questions about experimental protocols and study limitations that general-purpose web search agents, which usually only have access to abstracts, might miss. Our agents also use a variety of factors to distinguish source quality, so that they don’t end up relying on low-quality papers or pop-science sources. Finally, and critically, we have an API, which is intended to allow researchers to integrate our agents into their workflows. Phoenix is an experimental project we put together recently just to demonstrate what can happen if you give the agents access to lots of scientific tools. It is not better than humans at planning experiments yet, and it makes a lot more mistakes than Crow, Falcon, or Owl. We want to see all the ways you can break it! The agents we are releasing today cannot yet do all (or even most!) aspects of scientific research autonomously. However, as we show in the video, you can already use them to generate and evaluate new hypotheses and plan new experiments way faster than before. Internally, we also have dedicated agents for data analysis, hypothesis generation, protein engineering, and more, and we plan to launch these on the platform in the coming months as well. Within a year or two, it is easy to imagine that the vast majority of desk work that scientists do today will be accelerated with the help of AI agents like the ones we are releasing today. The platform is currently free-to-use. Over time, depending on how people use it, we may implement pricing plans. If you want higher rate limits, especially for research projects, get in touch. Michael Skarlinski, Andrew White 🐦‍⬛, Tyler Nadolski, Remo Storni, James Braza, Ludovico Mitchener, Michaela Hinks, as well as Jason Carman and his team for making such fantastic videos of us!

Today, we are launching the first publicly available AI Scientist, via the FutureHouse Platform. Our AI Scientist agents can perform a wide variety of scientific tasks better than humans. By chaining them together, we've already started to discover new biology really fast. With the platform, we are bringing these capabilities to the wider community. Watch our long-form video, in the comments below, to learn more about how the platform works and how you can use it to make new discoveries, and go to our website or see the comments below to access the platform. We are releasing three superhuman AI Scientist agents today, each with their own specialization: A general-purpose agent (Crow); An agent to automate literature reviews (Falcon); and An agent to answer the question “Has anyone done X before” (Owl). We are also releasing an experimental agent, Phoenix, that has access to a wide variety of tools for planning experiments in chemistry. More on that below. The three literature search agents (Crow, Falcon, and Owl) have benchmarked superhuman performance. They also have access to a large corpus of full scientific texts, which means that you can ask them more detailed questions about experimental protocols and study limitations that general-purpose web search agents, which usually only have access to abstracts, might miss. Our agents also use a variety of factors to distinguish source quality, so that they don’t end up relying on low-quality papers or pop-science sources. Finally, and critically, we have an API, which is intended to allow researchers to integrate our agents into their workflows. Phoenix is an experimental project we put together recently just to demonstrate what can happen if you give the agents access to lots of scientific tools. It is not better than humans at planning experiments yet, and it makes a lot more mistakes than Crow, Falcon, or Owl. We want to see all the ways you can break it! The agents we are releasing today cannot yet do all (or even most!) aspects of scientific research autonomously. However, as we show in the video, you can already use them to generate and evaluate new hypotheses and plan new experiments way faster than before. Internally, we also have dedicated agents for data analysis, hypothesis generation, protein engineering, and more, and we plan to launch these on the platform in the coming months as well. Within a year or two, it is easy to imagine that the vast majority of desk work that scientists do today will be accelerated with the help of AI agents like the ones we are releasing today. The platform is currently free-to-use. Over time, depending on how people use it, we may implement pricing plans. If you want higher rate limits, especially for research projects, get in touch. Michael Skarlinski, Andrew White 🐦‍⬛, Tyler Nadolski, Remo Storni, James Braza, Ludovico Mitchener, Michaela Hinks, as well as Jason Carman and his team for making such fantastic videos of us!

Sam Rodriques

726,032 görüntüleme • 1 yıl önce