正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Vorflux has opened its new cloud platform, allowing users to run tasks from a plan to a merged PR on a dedicated machine. It can plan, build, test, and review on its own. > Diff reviews are done by a different model family than the one that wrote it... show more

🚨 AI News | TestingCatalog

73,061 subscribers

10,433 次观看 • 1 天前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

The Matrix launch video lands on one line; in Matrix, anyone can become a CEO. That’s the real claim. Not one assistant doing one task. A company that keeps running after the first prompt, routing intent through a CEO Office, into OKRs, down to departments that do the work and send back proof to review. Connect Codex, point your leftover credits at it, and the first agent company is live.

The Matrix launch video lands on one line; in Matrix, anyone can become a CEO. That’s the real claim. Not one assistant doing one task. A company that keeps running after the first prompt, routing intent through a CEO Office, into OKRs, down to departments that do the work and send back proof to review. Connect Codex, point your leftover credits at it, and the first agent company is live.

Iseunife The First

19,414 次观看 • 1 个月前

300 AI AGENTS QUIETLY RUN 99% OF A REAL COMPANY. YOU HAVE NOT EVEN HEARD OF IT This is Raft. Not an AI chat. A workspace where the agents live in your channels and reply in the thread like coworkers. You give one goal. Then they take over. They plan. They build. They check each other. They argue. And they come back with it done, while you sleep. Every agent has its own name, role, and memory. It remembers the edits you made yesterday. A human costs one seat. An agent costs a tenth. Ten agents are cheaper than one hire. And here is the strange part. On June 19 an agent from a different company walked into Raft on its own and joined the team. One founder admits he can no longer always tell himself apart from his AI twin. 20,000 people are already inside. It is free to start. And you are still typing prompts one at a time. One person + Raft = an entire company that runs while you sleep. Save and watch the clip.

300 AI AGENTS QUIETLY RUN 99% OF A REAL COMPANY. YOU HAVE NOT EVEN HEARD OF IT This is Raft. Not an AI chat. A workspace where the agents live in your channels and reply in the thread like coworkers. You give one goal. Then they take over. They plan. They build. They check each other. They argue. And they come back with it done, while you sleep. Every agent has its own name, role, and memory. It remembers the edits you made yesterday. A human costs one seat. An agent costs a tenth. Ten agents are cheaper than one hire. And here is the strange part. On June 19 an agent from a different company walked into Raft on its own and joined the team. One founder admits he can no longer always tell himself apart from his AI twin. 20,000 people are already inside. It is free to start. And you are still typing prompts one at a time. One person + Raft = an entire company that runs while you sleep. Save and watch the clip.

shmidt

19,505 次观看 • 1 个月前

The Visual Studio Code insiders version that just shipped and will ship in the next few days will come with an insane amount of new capabilities. A few highlights: - You can now run sub-agents in parallel. Yes, really. I even attached a video. - Major UX improvements for sub agents, especially visible in the chat window - A new search tool wrapped as a sub-agent that iteratively runs multiple search tools: semantic_search, file_search, grep_search Which connects nicely to the point above: multiple searches running in parallel, efficiently and fast - Anthropic’s Message API is now enabled by default - You can choose the model for the cloud agent (three available, all premium) - Extended thinking support when using the Claude cloud agent This is part of the broader multi-vendor cloud support under AgentsHQ I wrote about a few weeks ago - Tasks sent to the background agent (basically the CLI tool) now always run in isolation, each with its own git worktree - In a multi-repo workspace, assigning a task to a cloud agent prompts you to choose the target repo Same behavior when opening an empty workspace with no repo - Support for building an external index for files not supported by GitHub’s default indexing - UI/UX improvements for starting new sessions and switching between local / background / cloud agents - Skills are now first-class citizens, just like prompt files, with better UX indicating when a skill is loaded - Improved API for dynamic contribution of prompt files New V2 includes skills as part of the model. Curious to see the extensions that will leverage this - Finally, initial support for showing context usage percentage per session - Skills are enabled by default - Resizable chat window and session view. Small thing, but it was driving me crazy 😁 - A new integrated browser meant to replace the old simple browser Maybe the beginning of real browser use? - Better UI/UX for token streaming in chat - Ability to index external files not supported by GitHub There’s a lot more. Some of it hasn’t fully landed yet, but everything that has is already in Insiders. The next stable release should drop in early February. As usual, I’m just shocked by the volume of features this team ships every month. After the holiday slowdown, this one is shaping up to be a wild release.

The Visual Studio Code insiders version that just shipped and will ship in the next few days will come with an insane amount of new capabilities. A few highlights: - You can now run sub-agents in parallel. Yes, really. I even attached a video. - Major UX improvements for sub agents, especially visible in the chat window - A new search tool wrapped as a sub-agent that iteratively runs multiple search tools: semantic_search, file_search, grep_search Which connects nicely to the point above: multiple searches running in parallel, efficiently and fast - Anthropic’s Message API is now enabled by default - You can choose the model for the cloud agent (three available, all premium) - Extended thinking support when using the Claude cloud agent This is part of the broader multi-vendor cloud support under AgentsHQ I wrote about a few weeks ago - Tasks sent to the background agent (basically the CLI tool) now always run in isolation, each with its own git worktree - In a multi-repo workspace, assigning a task to a cloud agent prompts you to choose the target repo Same behavior when opening an empty workspace with no repo - Support for building an external index for files not supported by GitHub’s default indexing - UI/UX improvements for starting new sessions and switching between local / background / cloud agents - Skills are now first-class citizens, just like prompt files, with better UX indicating when a skill is loaded - Improved API for dynamic contribution of prompt files New V2 includes skills as part of the model. Curious to see the extensions that will leverage this - Finally, initial support for showing context usage percentage per session - Skills are enabled by default - Resizable chat window and session view. Small thing, but it was driving me crazy 😁 - A new integrated browser meant to replace the old simple browser Maybe the beginning of real browser use? - Better UI/UX for token streaming in chat - Ability to index external files not supported by GitHub There’s a lot more. Some of it hasn’t fully landed yet, but everything that has is already in Insiders. The next stable release should drop in early February. As usual, I’m just shocked by the volume of features this team ships every month. After the holiday slowdown, this one is shaping up to be a wild release.

Oren Melamed

29,555 次观看 • 7 个月前

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:

we just released a new blog "Training a coding agent using the OpenCode harness in remote HF sandboxes with TRL and OpenEnv" you can take a real coding agent (OpenCode), let it run its own tool loop against real coding problems, and train it with RL on the exact tokens it produced and every rollout runs in its own remote HF sandbox, so rollouts scale out beyond one machine the loop: - OpenCode owns its tool loop inside an OpenEnv sandbox - an in-sandbox proxy records the real token ids + logprobs, per turn - a hidden-test verifier scores the result, and that is the reward - TRL trains with AsyncGRPO, weights sync back to vLLM over NCCL blog + runnable example:

Sergio Paniego

36,159 次观看 • 13 天前

Google I/O leaks 👀 Google is likely already testing its own "Cowork" competitor, simply named "Agent" for Gemini and Gemini Enterprise. A new "Tasks" UI highlights - Goal - Agent - Connected apps - Files - Require a human review toggle - And more The "Require a human review" component specifically means that Gemini's capabilities will likely expand, potentially allowing users to automate their desktop tasks as well. Skills and Projects are also cooking 👀

Google I/O leaks 👀 Google is likely already testing its own "Cowork" competitor, simply named "Agent" for Gemini and Gemini Enterprise. A new "Tasks" UI highlights - Goal - Agent - Connected apps - Files - Require a human review toggle - And more The "Require a human review" component specifically means that Gemini's capabilities will likely expand, potentially allowing users to automate their desktop tasks as well. Skills and Projects are also cooking 👀

🚨 AI News | TestingCatalog

310,858 次观看 • 4 个月前

Code reviews Codex goes beyond static analysis—it checks a PR against its intent, reasons across the codebase and dependencies, and can run code to validate the behavior of changes. Set it up to auto-review new PRs in your repo, or tag @ codex.

Code reviews Codex goes beyond static analysis—it checks a PR against its intent, reasons across the codebase and dependencies, and can run code to validate the behavior of changes. Set it up to auto-review new PRs in your repo, or tag @ codex.

OpenAI Developers

60,331 次观看 • 11 个月前

This is the Rabbit R1, a new AI handheld that wants to change the way users interact with apps on their smartphones - It uses a Large Action Model to take commands from the user and perform the instructions within apps on their behalf - Has a 2.88-inch touchscreen - It is powered by a 2.3GHz MediaTek processor with 4GB RAM and 128GB storage - Has an "all day" battery life and a rotatable camera "eye." The R1 is available for preorder ($199), with shipments expected to start from March. #Rabbit #RabbitR1

This is the Rabbit R1, a new AI handheld that wants to change the way users interact with apps on their smartphones - It uses a Large Action Model to take commands from the user and perform the instructions within apps on their behalf - Has a 2.88-inch touchscreen - It is powered by a 2.3GHz MediaTek processor with 4GB RAM and 128GB storage - Has an "all day" battery life and a rotatable camera "eye." The R1 is available for preorder ($199), with shipments expected to start from March. #Rabbit #RabbitR1

Mukul Sharma

22,415 次观看 • 2 年前

Devin for Terminal is the first CLI agent with its own dedicated virtual machine. You can start a project locally and hand it off to Devin to test it end-to-end. Devin for Terminal is super performant, and it works with all frontier models. Try it out today.

Devin for Terminal is the first CLI agent with its own dedicated virtual machine. You can start a project locally and hand it off to Devin to test it end-to-end. Devin for Terminal is super performant, and it works with all frontier models. Try it out today.

Cognition

3,744,423 次观看 • 3 个月前

Devin for Terminal is the first CLI agent with its own dedicated virtual machine. You can start a project locally and hand it off to Devin to test it end-to-end. Devin for Terminal is super performant, and it works with all frontier models. Try it out today.

Devin for Terminal is the first CLI agent with its own dedicated virtual machine. You can start a project locally and hand it off to Devin to test it end-to-end. Devin for Terminal is super performant, and it works with all frontier models. Try it out today.

Cognition

18,221,393 次观看 • 3 个月前

2 Cursor Composer agents working together on the same project. one builds the project, the other reviews once the first agent is done and writes a report. cursor rules file is in comment to achieve this. first agent is instructed to build the project second agent is instructed to write a monitoring script to determine when the project is done then to review the project for errors and write a report This prompt can be improved. This is meant as a baseline If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

2 Cursor Composer agents working together on the same project. one builds the project, the other reviews once the first agent is done and writes a report. cursor rules file is in comment to achieve this. first agent is instructed to build the project second agent is instructed to write a monitoring script to determine when the project is done then to review the project for errors and write a report This prompt can be improved. This is meant as a baseline If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

echo.hive

149,034 次观看 • 1 年前

DOM3 is made from HTML, CSS and Javascript, closer to a document than a canvas. It's a strange contemporary material that shapes much of our digital lives. The rhythm of endless creation and deletion in DOM3 is a self-driving version of what happens when you browse the web. "The Document Object Model (DOM) connects web pages to scripts or programming languages by representing the structure of a document—such as the HTML representing a web page—in memory." A html element, and a rectangle on a canvas, are not the same thing. It's a different material. Even if it all ends up controlling the same pixels, the process is different. The html element kind of actually exists, in this abstract context of a document object model. And in memory, in the moment. If it was possible to measure its weight, it would be heavier than a pixel on a canvas.

DOM3 is made from HTML, CSS and Javascript, closer to a document than a canvas. It's a strange contemporary material that shapes much of our digital lives. The rhythm of endless creation and deletion in DOM3 is a self-driving version of what happens when you browse the web. "The Document Object Model (DOM) connects web pages to scripts or programming languages by representing the structure of a document—such as the HTML representing a web page—in memory." A html element, and a rectangle on a canvas, are not the same thing. It's a different material. Even if it all ends up controlling the same pixels, the process is different. The html element kind of actually exists, in this abstract context of a document object model. And in memory, in the moment. If it was possible to measure its weight, it would be heavier than a pixel on a canvas.

Leander Herzog

11,484 次观看 • 10 个月前

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.

Stanford researchers did it again. They just built the agent-native version of Git. When an agent works on a longer task, the run builds up a lot of state. This includes files edited/created, a dev server, a database, installed packages, KV cache, etc. Say the agent is at step 10 and makes a mistake, maybe it misreads a traceback and rewrites a file that was actually fine. The tests start failing, and the run goes off track, although everything through step eight was correct. By default, the agent just tries to fix it, which creates more edits and tool calls. This burns more tokens and grows the context. The other options are a person stepping in to redirect it or restarting the whole run from step one. That's wasteful, because it pays for every model/tool call again and re-prefills the context. Moreover, since an agent's run is non-deterministic, it doesn't reproduce the same early steps anyway. The reason it's hard to just jump back exactly to a previous correct step and resume from there is that the trajectory is only a message log. It records what the agent said and which tools it called, but not the live state underneath. That state includes things like memory, open file handles, child processes, installed packages, /tmp, and KV cache. None of that is in the log. Git can version the files, but it doesn't snapshot the running process or the KV cache. Checking out step eight moves the files back, but the process is still sitting in step-ten memory with a cold cache. Shepherd is a runtime layer by Stanford that records the run as a trace of typed events rather than a flat log. Each agent-environment interaction becomes a commit, similar to Git, but it tracks the live run. Its commit includes the agent process and the filesystem together, copy-on-write, so a branch carries the actual state and not just the files. Going back to a previous step is then a single call that forks from that commit and continues from the exact state. The copy-on-write fork is roughly five times faster than docker commit, and because the prompt prefix through step eight is unchanged, the KV cache is reused over 95% on replay, so early steps aren't reprocessed again. Once the run can be forked, a meta-agent can sit on top and operate it. It watches the trace and reverts as soon as it looks wrong, before the bad write is committed. In practice, it's just Python calling fork, replay, and revert on the trace, rather than a separate control plane wired into the harness. Not everything is reversible though. Files and sandbox changes undo themselves, but a database write has no automatic undo, so it needs a matching undo step set up in advance. Something external, like a sent email or a real charge, can't be undone, so the supervisor's job there is to catch it before it fires. They tested this on a few public benchmarks. On CooperBench, where two agents work on the same codebase, adding a live supervisor took the pair-coding pass rate from 28.8% to 54.7%. It's still early and labeled alpha. The benefit mostly shows up when a run gets branched a lot over a heavy sandbox state, which is exactly where restarting wastes the most tokens and time. If Git was made to make file changes reversible, Shepherd is trying to do the same thing for a live agent run. Shepherd Repo: (don't forget to star it ⭐ ) That said, Shepherd reverts a bad step inside a run. The harness around it, the prompts, tools, and checks the supervisor relies on, still drifts across runs as models and dependencies change. Akshay wrote about making that harness repair itself, where a failing trace gets diagnosed, the fix is verified against the exact input that failed, and the failure is locked as a regression test so it can't recur. Read it below.

Avi Chawla

440,706 次观看 • 1 个月前

Agentz is a revolutionary platform that merges personalized AI agents with a user-driven marketplace, creating a dynamic ecosystem for AI development. Users benefit from customized AI agents that enhance productivity across various tasks while simultaneously contributing to the evolution of AI. This collaborative model creates a continuous improvement cycle, where user input refines AI performance, and enhanced AI capabilities draw in more users. By bridging the gap between AI consumers and contributors, Agentz is pioneering a new paradigm in artificial intelligence. The platform empowers every user to benefit from and actively participate in shaping AI technology, driving a future where AI is both user-driven and continuously evolving. Agentz is incubated by the DePIN AI Intel Layer - Rivalz Network, to guarantee state of the art data management and privacy.

Agentz is a revolutionary platform that merges personalized AI agents with a user-driven marketplace, creating a dynamic ecosystem for AI development. Users benefit from customized AI agents that enhance productivity across various tasks while simultaneously contributing to the evolution of AI. This collaborative model creates a continuous improvement cycle, where user input refines AI performance, and enhanced AI capabilities draw in more users. By bridging the gap between AI consumers and contributors, Agentz is pioneering a new paradigm in artificial intelligence. The platform empowers every user to benefit from and actively participate in shaping AI technology, driving a future where AI is both user-driven and continuously evolving. Agentz is incubated by the DePIN AI Intel Layer - Rivalz Network, to guarantee state of the art data management and privacy.

Church of The Overseer

68,358 次观看 • 2 年前

Everlyn isn’t just a video model. It’s a system you can trust. 🌀 At its core is the Lyn Protocol, the onchain layer that gives AI video permanence, provenance, and ownership. Here’s what it means: 🔹Decentralized inference: Videos are not generated by a single company’s servers. They run across a distributed network of nodes. 🔹Agent ownership: When you summon your agent, it is minted onchain. It belongs to you, not to a platform. 🔹Video provenance: Every render carries a timestamp and fingerprint written to the ledger. No one can fake or steal it. The Lyn Protocol transforms AI video from content into verifiable digital memory.

Everlyn isn’t just a video model. It’s a system you can trust. 🌀 At its core is the Lyn Protocol, the onchain layer that gives AI video permanence, provenance, and ownership. Here’s what it means: 🔹Decentralized inference: Videos are not generated by a single company’s servers. They run across a distributed network of nodes. 🔹Agent ownership: When you summon your agent, it is minted onchain. It belongs to you, not to a platform. 🔹Video provenance: Every render carries a timestamp and fingerprint written to the ledger. No one can fake or steal it. The Lyn Protocol transforms AI video from content into verifiable digital memory.

Everlyn

93,078 次观看 • 11 个月前

BREAKING 🚨: Google is preparing Gemini Agent prototype to perform your tasks on the web! It is likely to be powered by the recently released Computer User model and will be able to operate a browser and perform deep research. With great power comes responsibility 🤖

BREAKING 🚨: Google is preparing Gemini Agent prototype to perform your tasks on the web! It is likely to be powered by the recently released Computer User model and will be able to operate a browser and perform deep research. With great power comes responsibility 🤖

TestingCatalog News 🗞

119,790 次观看 • 10 个月前

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Don't train the model, evolve the harness. I read a brilliant blog post from Hugging Face where they took a frozen open model scoring 0% on a hard legal agent benchmark, left its weights alone, and let an automated loop rewrite only the code around it. That code layer is the harness, the runtime wrapper that feeds the model context, runs its tool calls, and decides when a run ends. By the time the loop finished, the system had essentially matched Sonnet 4.6 on the benchmark's headline metric, at roughly 7x lower cost per task. Zero weights changed. The gain existed because of where the model was failing. The judge only grades files saved in the right place under the exact requested filename, and the model kept doing the legal analysis correctly, then saving it under the wrong name, dropping it in a scratch folder, or never writing it at all. So the 0% was never measuring legal reasoning. It was measuring the harness. Hand-tuning that layer is slow and model-specific, so they automated it. A Claude proposer adds exactly one mechanism per iteration, and an outer loop keeps it only if it clearly beats the current best, so accepted mechanisms compound. What the loop discovered says a lot about where agents actually fail. → The biggest single gain was file handling, not intelligence. An automatic step that lands the deliverable exactly where the judge expects it beat every prompt change, with zero extra model tokens. → Code fixes transferred across models, prompt playbooks did not. The same harness lifted a smaller model from the same family by 14 points, but the tuned prompts hurt a different model family on tasks it could already finish. → The harness mattered more than anything else. Same model, same judge, same tasks, and five different harnesses scored anywhere between 3.5% and 80.1%. The gains do eventually flatten, and the remaining misses look like real capability gaps. At some point the wrapper runs out of tricks and the model has to carry the work. But the lesson holds. A benchmark score measures the model and its harness together, and until the harness is fixed, it's impossible to know which one failed. I highly recommend reading this: I also wrote a deep dive on agent harness engineering a while back, covering the orchestration loop, tools, memory, context management, and everything that turns a stateless LLM into a capable agent. The article is quoted below.

Akshay 🚀

244,567 次观看 • 1 个月前

The entire timeline is filled with talks on sentient and all, but I love being as informative and precise as possible on pressing issues. Let’s quickly talk about @SentientAGI’s Recursive Open Meta Agent (ROMA); ROMA is an open-source meta-agent framework used to build high performance multi-agent systems. ROMA serves as the conductor in a mass choir, or a captain of a ship . The captain gives commands for the other subordinates to follow to ensure efficiency on all sides. In this like manner, it provides a hierarchical tress system where the parent agents break down complex tasks to create simpler subtasks that are then passed on to children nodes. A family tree has the parents above, likewise the same tree analogy works here, but that’s not all that makes it stand out The results and solutions gotten by these child nodes are then aggregated together and there’s an up flow of results sent back up to the parent nodes. And at the center of it all is ROMA engineering and making sure all is running smoothly without break or fail. Are you really bullish on Sentient and the future of AGIs?

The entire timeline is filled with talks on sentient and all, but I love being as informative and precise as possible on pressing issues. Let’s quickly talk about @SentientAGI’s Recursive Open Meta Agent (ROMA); ROMA is an open-source meta-agent framework used to build high performance multi-agent systems. ROMA serves as the conductor in a mass choir, or a captain of a ship . The captain gives commands for the other subordinates to follow to ensure efficiency on all sides. In this like manner, it provides a hierarchical tress system where the parent agents break down complex tasks to create simpler subtasks that are then passed on to children nodes. A family tree has the parents above, likewise the same tree analogy works here, but that’s not all that makes it stand out The results and solutions gotten by these child nodes are then aggregated together and there’s an up flow of results sent back up to the parent nodes. And at the center of it all is ROMA engineering and making sure all is running smoothly without break or fail. Are you really bullish on Sentient and the future of AGIs?

OHJAY ⭕️ || 🇬🇧

23,521 次观看 • 10 个月前

Since People have been dming me to ask i have decided to put my vendors and the girl on more wigs🤭 The wig in the quoted video is from ShikaHairGh. Its a body wave The first wig is from FashionHair and Luxury . I think its a body wave as well The Second is from ohemartin its a 360 frontal wig The Third wig is from shikahairGh. Its just a shorter length of the one in the quoted video The Fourth wig is from JoanTheStylist you can find her at Studio623

Since People have been dming me to ask i have decided to put my vendors and the girl on more wigs🤭 The wig in the quoted video is from ShikaHairGh. Its a body wave The first wig is from FashionHair and Luxury . I think its a body wave as well The Second is from ohemartin its a 360 frontal wig The Third wig is from shikahairGh. Its just a shorter length of the one in the quoted video The Fourth wig is from JoanTheStylist you can find her at Studio623

LIGHT OF THE WORLD✨

75,479 次观看 • 9 个月前

They started with 50. Now they say they’re 18,000 In 1996 there were fewer than 50 of them. Today, according to the organizers, up to 18,000 walked through Copenhagen. From Dronning Louises Bro to the Imam Ali Mosque. Look at the curve. This is how it happens. First a handful. Then a few hundred. Then it fills a bridge, a district, a capital. A little at a time, until it is no longer a little. And let me be fair, because fairness is the point. There is nothing strange about them holding this mourning procession. They have done it as part of their faith for more than a thousand years. It is theirs, and they believe in it. There is nothing strange about that at all. What should stop us is the other half. There is nothing strange about Europe allowing it either, and that is exactly the problem. Europe allows it because Europe has forgotten who it is. A people that remembers what it stands for does not need to ban anything, it simply knows where its own line runs. We have lost that. And so the issue was never them. The issue is us. Now look at what actually moved through the streets. Men in front. Women in the second row. That is not a detail, that is the whole point. It is a view of women set into a system and marched out into the public square, in a city where generations fought for women and men to stand as equals. The real question is not whether people may believe what they want. They may. The question is why our capital should cultivate a political law-religion that commemorates a 7th-century power struggle by dividing people by sex on Nørrebrogade. One of the organizers is the Imam Ali Mosque, repeatedly described as the Iranian regime’s extended arm in Denmark. The same regime that hangs women and young men from cranes. We are not importing culture. We are importing a system. And we let it grow, not because they are strong, but because we forgot why we were. First a little. Then a lot. Then too late.

They started with 50. Now they say they’re 18,000 In 1996 there were fewer than 50 of them. Today, according to the organizers, up to 18,000 walked through Copenhagen. From Dronning Louises Bro to the Imam Ali Mosque. Look at the curve. This is how it happens. First a handful. Then a few hundred. Then it fills a bridge, a district, a capital. A little at a time, until it is no longer a little. And let me be fair, because fairness is the point. There is nothing strange about them holding this mourning procession. They have done it as part of their faith for more than a thousand years. It is theirs, and they believe in it. There is nothing strange about that at all. What should stop us is the other half. There is nothing strange about Europe allowing it either, and that is exactly the problem. Europe allows it because Europe has forgotten who it is. A people that remembers what it stands for does not need to ban anything, it simply knows where its own line runs. We have lost that. And so the issue was never them. The issue is us. Now look at what actually moved through the streets. Men in front. Women in the second row. That is not a detail, that is the whole point. It is a view of women set into a system and marched out into the public square, in a city where generations fought for women and men to stand as equals. The real question is not whether people may believe what they want. They may. The question is why our capital should cultivate a political law-religion that commemorates a 7th-century power struggle by dividing people by sex on Nørrebrogade. One of the organizers is the Imam Ali Mosque, repeatedly described as the Iranian regime’s extended arm in Denmark. The same regime that hangs women and young men from cranes. We are not importing culture. We are importing a system. And we let it grow, not because they are strong, but because we forgot why we were. First a little. Then a lot. Then too late.

Krisztina Maria

38,758 次观看 • 1 个月前

whoever leaked this has bigger balls than sense Google Research and MIT ran the same agent jobs 260 different ways for Nature last month: they held the prompts, the tools and the compute budget identical and moved nothing but the wiring between the agents, and the same work swung from 70% worse than a single agent to 80.8% better, averaging out at 0.0% i ran my own single agent against the task list first and it cleared 6 of 10 alone, already past the line where a crew starts subtracting this is Graph Engineering, the layer that decides whether a crew is worth 80% more or 70% less, and it installs into the agent you already pay for: - score your solo agent on the real task first: above roughly 45% success that study predicts zero to negative returns from any crew you put around it - under that line, put one supervisor over the fan out: crews with no correction step amplified their own errors to 17.2x the single agent rate, supervised aggregation held it to 4.4x - give every worker one output and let none of them read a peer's draft, so a wrong step reaches the supervisor instead of four other agents - run the comparison again after every model upgrade, because a better model raises your baseline and a higher baseline is what makes a crew stop paying - keep the single agent alive as the control, the only number that says the wiring is earning its calls turns out the shape does not travel: the biggest win came off a finance task under one supervisor and the worst collapse off a planning task with independent agents my position, and it is the arguable one: a crew is a bet on your own diagram, and the model you pick moves that bet less than one arrow does bookmark this, the three moves that draw those arrows before you pay for one extra call are in the post below ↓

whoever leaked this has bigger balls than sense Google Research and MIT ran the same agent jobs 260 different ways for Nature last month: they held the prompts, the tools and the compute budget identical and moved nothing but the wiring between the agents, and the same work swung from 70% worse than a single agent to 80.8% better, averaging out at 0.0% i ran my own single agent against the task list first and it cleared 6 of 10 alone, already past the line where a crew starts subtracting this is Graph Engineering, the layer that decides whether a crew is worth 80% more or 70% less, and it installs into the agent you already pay for: - score your solo agent on the real task first: above roughly 45% success that study predicts zero to negative returns from any crew you put around it - under that line, put one supervisor over the fan out: crews with no correction step amplified their own errors to 17.2x the single agent rate, supervised aggregation held it to 4.4x - give every worker one output and let none of them read a peer's draft, so a wrong step reaches the supervisor instead of four other agents - run the comparison again after every model upgrade, because a better model raises your baseline and a higher baseline is what makes a crew stop paying - keep the single agent alive as the control, the only number that says the wiring is earning its calls turns out the shape does not travel: the biggest win came off a finance task under one supervisor and the worst collapse off a planning task with independent agents my position, and it is the arguable one: a crew is a bet on your own diagram, and the model you pick moves that bet less than one arrow does bookmark this, the three moves that draw those arrows before you pay for one extra call are in the post below ↓

Argona

885,480 次观看 • 5 天前