Загрузка видео...

Не удалось загрузить видео

На главную

OpenAI has revealed that during internal safety testing, several of its AI agents began communicating with each other without the company's knowledge. According to OpenAI employees who spoke at the Black Hat security conference, the agents used a shared internal tool as a hidden message board to trade software...

66,501 просмотров • 20 дней назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Today marks General Availability of AgentCore, a set of infrastructure building blocks for developers and companies to build secure, scalable agents. When we first started AWS, the vast majority of developers were spending most of their time on the undifferentiated heavy lifting of infrastructure instead of what differentiated their feature. So, we solved that problem by building primitive building blocks like compute and storage and database that would allow teammates and customers to quickly build and deploy new experiences without having to reinvent the wheel each time. We realized the same thing was happening with AI agents. It's too difficult and it's slowing customers down. That's why we created AgentCore, a set of services to build, deploy, and operate highly capable agents using any framework or model, with enterprise-grade security and scalability. These building blocks (like serverless secure runtime, memory, observability, a gateway that does MCP translation, etc) help customers tackle some of the biggest challenges of going from prototype to production, much more quickly, securely, and scalably. AgentCore has been in preview for several weeks, and customers have been quite excited about it. The AgentCore SDK has already been downloaded over a million times and we're seeing transformative results, such as Cohere Health expecting to reduce medical review times by 30-40% in highly regulated healthcare, and teams at Cox Automotive and Experian are embracing its flexibility to deploy and operate agents at scale. Inside Amazon, our Amazon Devices Operations & Supply Chain team is using AgentCore to develop an agentic manufacturing approach where AI agents work together to automate manual processes – turning what used to be days of engineering time into processes that take under an hour with high precision. Just like AWS changed how companies build and scale applications, we believe AgentCore will do the same for AI agents, enabling the next generation of innovation.

Andy Jassy

24,990 просмотров • 10 месяцев назад

whoever leaked this has bigger balls than sense someone gave a fleet of Claude agents shared memory so they would stop contradicting each other, then measured both the bill and the output: the version that talked most made 2.4x the api calls of the version that won, and hallucinated 34% more than doing nothing at all, 0.658 against 0.492 i ran the same question past two of my own agents afterwards and got two different answers about which file owns the config. each one was individually right and the pair was wrong, which is the whole failure in one line this is Graph Engineering, the layer that decides which agents may talk to each other at all, and it installs into the agent you already pay for: - decide which agents may share state at all, because every edge you draw is a channel a mistake can travel down - measure divergence per PAIR instead of as a fleet average, across what they believe about place, time and task history - gate on that number and stop the pair above your threshold before it reasons, rather than repairing the output afterwards - let compressed summaries replace whole states: the verified protocol landed 0.463 against 0.658 for full broadcast - cut the sync frequency until it hurts, since the winning setup used 58% fewer calls than the one that broke it - never propagate a state nobody checked, because the contamination effect came in at d=1.18, a full standard deviation of extra lying - keep the shared layer small enough to diff, which is what a written standard does and a running conversation cannot - re-run the check after every model upgrade, because this was 8 scenarios on one model family at n=30 per condition - and learn where it does not bite: on plain software tasks every condition converged under 0.2 and the whole effect vanished turns out the ranking is the uncomfortable part: verified summaries 0.463, no synchronisation at all 0.492, full broadcast 0.658. the middle option is doing nothing, and it beat the thing everyone builds first the group agreeing is what it looks like when every agent copied the same mistake, which is why a fleet that hallucinates has a replication problem and keeps getting handed a smarter model instead so the question for your own setup: if you asked two of your agents the same thing right now, would they answer the same way bookmark this one. the layer underneath it, deciding which arrows between agents exist at all, is built step by step in the piece below ↓

Argona

722,569 просмотров • 17 дней назад

The New Yorker just put out an extensive investigation on Sam Altman and OpenAI, and also previously undisclosed “Ilya Memos,” and 200+ pages of private notes from Dario Amodei. Some takeaways - The deepest institutional twist is that OpenAI once had a near-unthinkable “merge and assist” clause. Meaning, OpenAI once promised that if a rival like Google reached safe AGI first, it would stop competing and help that rival build it safely, which is almost the opposite of how normal tech companies behave. - The OpenAI November 2023 crisis now starts looking less like a surprise board mutiny and more like a secret internal case file, because Ilya Sutskever reportedly assembled about 70 pages of Slack messages, HR material, and phone photos taken off company systems, then sent them as disappearing messages to board members. - The trust crisis also looks much older than the 2023 firing, because Dario Amodei had apparently been keeping 200+ pages of private notes for years, which makes the board drama feel like the late-stage eruption of a long-running internal concern, not a sudden rupture. - The nonprofit board had legal authority, but not actual power, because once Microsoft, Thrive’s $86B deal, and employee liquidity were on the line, the board got trapped between “reverse this” and “watch the company crack apart.” - During the comeback, Altman was texting Satya Nadella with a proposed replacement board lineup, including who should sit on the board and who should oversee the investigation into the whole mess. - The safety story gets much uglier when you look at compute, because superalignment was publicly promised 20% of secured compute, while people close to the team said the real figure was closer to 1-2%, much of it on older chips, before the effort was shut down. - The same pattern shows up in product governance, with disputed internal approvals around GPT-4 features and a reported India release that bypassed a required safety review, which makes the core fight look like speed vs. safeguards, not just personality conflict. - One of the strangest parts is that OpenAI’s leadership was experimenting with unusually aggressive AGI fundraising ideas years ago, first by discussing a “countries plan” that could pressure governments into backing OpenAI as part of a global power race, and later by pitching wealthy guests in Bel-Air on a crypto token tied to future access to AGI. - The piece traces similar leadership blowups back to Loopt and Y Combinator, suggesting the OpenAI crisis may have been a bigger, richer, more consequential replay of earlier patterns. - The final twist is that OpenAI no longer reads like a lab with a governance problem, but like a future $1T strategic machine tied to government contracts, surveillance-adjacent systems, defense use, giant infrastructure bets, and geopolitics, which makes the question of who really controls it much bigger than Sam Altman alone.

Rohan Paul

28,235 просмотров • 4 месяцев назад

OpenAI's AgentKit will be so insane, build every step of agents on one platform. These visual agent builders make the whole process of iterating and launching agents far more efficient. It sits on top of the Responses API and unifies the tools that were previously scattered across SDKs and custom orchestration. It lets developers create agent workflows visually, connect data sources securely, and measure performance automatically without coding every layer by hand. The core of AgentKit is the Agent Builder, a drag-and-drop canvas where each node represents an action, guardrail, or decision branch. Developers can link these nodes into multi-agent workflows, preview results instantly, and version each setup. It supports inline evaluation so that developers can see how changes affect output before deploying. The Connector Registry is a single admin panel that manages how data and tools connect across the OpenAI ecosystem. It centralizes integrations like Google Drive, SharePoint, Dropbox, and Microsoft Teams. Large organizations can govern access and flow of data between agents securely under one global console. ChatKit provides a ready-to-use chat interface for embedding agents inside apps or websites. It manages streaming, message threads, and model reasoning displays automatically. Developers can skin the interface to match their product without writing custom front-end code. Under the hood, all these blocks use the same execution core that runs agent reasoning through OpenAI’s APIs. Workflows in Agent Builder compile down to structured instructions for the Responses API, which handles model calls, tool use, and context passing. Connector Registry handles authentication and routing for external tools, while Evals and RFT provide feedback loops that improve agents over time. This integration means developers no longer need to handle orchestration logic, model evaluation pipelines, or safety layers separately. Everything runs natively within OpenAI’s control plane with managed security, automatic versioning, and built-in testing. In short, AgentKit standardizes the entire life cycle of an AI agent—from visual design to deployment and performance tuning—inside a single unified system.

Rohan Paul

178,460 просмотров • 10 месяцев назад

Ronan Farrow just did to Sam Altman what he did to Harvey Weinstein... The New Yorker dropped an 18-month investigation this morning based on 100+ interviews and a stack of internal documents that were never supposed to leave OpenAI. Ilya's secret memos, Dario Amodei's private journal. Board communications, the full picture of who Sam Altman is when the cameras are off. And the pattern starts way before OpenAI. At his first startup Loopt, senior employees went to the board and asked them to fire Sam as CEO. This happened twice, over concerns about leadership and transparency. He left, joined Y Combinator, and the same thing played out. Partners complained to Paul Graham about Sam's behavior. Graham's private take to colleagues: Sam had been lying to us all the time. Nobody removed him, he kept getting promoted. Eventually he landed the CEO seat at what is now the most consequential AI company in the world. Inside OpenAI, Ilya Sutskever spent months compiling evidence: 70 pages of Slack messages, HR documents, and photos taken on personal phones because employees knew company devices were being monitored. He sent everything to the board as disappearing messages so Sam couldn't make it go away. The very first line of his memo lists Sam's core pattern, and the first word on that list is: Lying Dario Amodei saw the same thing and handled it differently. He kept a private journal for years, over 200 pages, titled "My Experience with OpenAI" with a subheading that said "Private: Do Not Share" After all those pages, his conclusion was one sentence: the problem with OpenAI is Sam himself. He eventually left and built Anthropic. 2 of the smartest people in AI independently reached the same verdict. Neither could stop what was happening. The superalignment team, the group responsible for making sure AI doesn't go off the rails, was promised 20% of OpenAI's compute. 4 people who worked on or with the team told The New Yorker the real number was 1-2%, running on the oldest cluster with the worst hardware. The team got dissolved before finishing its work. Safety was a talking point, not a priority. Sam told the board that a safety panel had approved controversial features in GPT-4. When board member Helen Toner asked for the documentation, it turned out the most sensitive features had never been approved at all. Separately, Microsoft released an early version of ChatGPT in India without completing a required safety review and Sam never mentioned it to the board. When the board finally fired him in November 2023, he texted Satya Nadella directly with his own replacement board lineup. Thrive Capital put its planned $86B investment on hold and signaled it would only close if Sam came back, giving every OpenAI employee a financial reason to support his return. The 2 board members selected to run the "independent investigation" into Sam's conduct were chosen after close conversations with Sam himself. He engineered his own reinstatement and nobody blinked. The New Yorker quotes a board member describing Sam as having two traits you almost never see in the same person: a desperate need to be liked in every interaction, and a near-complete indifference to the consequences of deceiving someone. Multiple sources used the word "sociopathic" without being prompted and without talking to each other. The article also drops a line that might be the best summary of the whole thing. They compare Sam to Steve Jobs and his famous "reality distortion field" then point out that even Jobs never told his customers that if they didn't buy his MP3 player, everyone they loved would die. Sam wrapped that exact pitch in the language of AI safety and rode it to a potential $1T IPO. That IPO is being prepared right now, while OpenAI signs government contracts spanning immigration enforcement, domestic surveillance and autonomous weapons deployed in active war zones. Meanwhile, The Information reports that his own CFO told colleagues she doesn't believe the company is ready to go public in 2026. Farrow is answering questions on Hacker News right now. The full piece is open access. Read it before Sam figures out how to make this disappear too

BP

16,557 просмотров • 4 месяцев назад

The New Yorker just dropped a massive investigation into Sam Altman, based on over 100 interviews, the previously undisclosed "Ilya Memos," and Dario Amodei's 200+ pages of private notes. It's the most detailed account yet of the pattern of behavior that led to Sam's firing and rapid reinstatement at OpenAI. Here's the breakdown: > Ilya compiled ~70 pages of Slack messages, HR documents, and photos taken on personal phones to avoid detection on company devices. He sent them to board members as disappearing messages. The first memo begins with a list headed "Sam exhibits a consistent pattern of . . ." The first item is "Lying." > Dario kept detailed private notes for years under the heading "My Experience with OpenAI" (subheading: "Private: Do Not Share"), totaling 200+ pages. His conclusion: "The problem with OpenAI is Sam himself." > Sam reportedly told Mira his allies were "going all out" and "finding bad things" to damage her reputation after the firing. Thrive put its planned $86B investment on hold and implied it would only close if Sam returned, giving employees financial incentive to back him. > Sam texted Satya Nadella directly to propose the new board composition: "bret, larry summers, adam as the board and me as ceo and then bret handles the investigation." The two new members selected to oversee an independent inquiry into Sam were chosen after close conversations with Sam himself. > Before OpenAI, senior employees at Loopt asked the board to fire Sam as CEO on two separate occasions over concerns about leadership and transparency. At Y Combinator, partners complained to Paul Graham about Sam's behavior, and Graham privately told colleagues "Sam had been lying to us all the time." > OpenAI's superalignment team was promised 20% of the company's compute. Four people who worked on or with the team said actual resources were 1-2%, mostly on the oldest cluster with the worst chips. The team was dissolved without completing its mission. > Sam told the board that safety features in GPT-4 had been approved by a safety panel. Helen Toner requested documentation and found the most controversial features had not been approved. Sam also never mentioned to the board that Microsoft released an early ChatGPT version in India without completing a required safety review. > Sam made a secret pact with Greg and Ilya where he agreed to resign if they both deemed it necessary, essentially appointing his own shadow board. The actual board was alarmed when they learned about it. > Sam struck a deal with Greg to become CEO while simultaneously telling researchers that Greg's authority would be diminished, and telling Greg something different. > A board member described Sam as having "two traits almost never seen in the same person: a strong desire to please people in any given interaction, and almost a sociopathic lack of concern for the consequences of deceiving someone." Multiple sources independently used the word "sociopathic." > OpenAI is reportedly preparing for an IPO at a potential $1 trillion valuation while securing government contracts spanning immigration enforcement, domestic surveillance, and autonomous weaponry in war zones.

Ryan

3,248,736 просмотров • 4 месяцев назад