Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

We just open-sourced Simulang, a JavaScript library that gives you and your coding agents the ability to fully control computers. Simulang is our scripting framework for desktop automation. It allows you to write deterministic computer use code that can be replayed dozens if not hundreds of times. Much faster... and cheaper, compared to LLM agents. It ships with a skill out of the box. That means you can turn natural language prompts into Simulang code directly, and your agent can now see the screen, click, type, and navigate. It gives your AI agents eyes and hands to control the computer like humans do. Try these things with Simulang: - upload your pile of invoices to the expense platform - find 50 people with specific criteria on LinkedIn - open your calculator app and do some math - find all buttons on your favorite app using accessibility tree - or draw a bunny with MS paint 🐰 In this demo, we show how Simulang took over when the coding agent couldn’t open and click on the target LinkedIn profile page. Give it a try:show more

Simular

9,297 subscribers

14,515 просмотров • 6 месяцев назад •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

Комментарии: 13

Фото профиля Aryan Rakib

Aryan Rakib6 месяцев назад

This is insane

Фото профиля Simular

Simular6 месяцев назад

In the good way or the bad way?

Фото профиля Lydiaaa

Lydiaaa6 месяцев назад

There are so many use cases I can imagine with Simulang! 🔥

Фото профиля Simular

Simular6 месяцев назад

What use case if you have to pick only one

Фото профиля nsxdavid

nsxdavid6 месяцев назад

Deterministic replay is the right direction. The missing layer is that a lot of UI failure doesn’t live in the final state. It lives between states: cursor/ghost offsets, delayed indicators, animation timing, drag feedback, all the stuff a human catches as motion. Computer use needs replay, but it also needs temporal visual verification.

Фото профиля Simular

Simular6 месяцев назад

You are absolutely right

Фото профиля samantha | computer-use ai agents

samantha | computer-use ai agents6 месяцев назад

looking forward to Simulang recipes 👀

Фото профиля Simular

Simular6 месяцев назад

Look forward to a recipe created by you

Фото профиля Matt

Matt6 месяцев назад

deterministic replay is the easy half. UI drift breaks selectors and coordinates. accessibility trees (AX on mac, UIA on windows) survive theme and font shifts where DOM scripts dont. we built Terminator on exactly that AX/UIA tree approach so selectors hold through theme and font drift,

Фото профиля Ethan Walker

Ethan Walker6 месяцев назад

The transition from “chatbots” to true computer-use agents is probably one of the most important shifts happening in AI right now. Agents that can actually operate software environments reliably will change how people work.

Фото профиля Simular

Simular5 месяцев назад

Great insight!

Фото профиля Ang Li

Ang Li6 месяцев назад

I guess we need a playwright for computer use

Фото профиля Simular

Simular6 месяцев назад

This is gonna be a better playwright for computer use

Похожие видео

Now you can use your favorite AI agent to control your Coinbase account (or a sub-account), with Coinbase for Agents. Here’s a quick demo on how to set it up and some of the cool things you can get your agent to do.

Now you can use your favorite AI agent to control your Coinbase account (or a sub-account), with Coinbase for Agents. Here’s a quick demo on how to set it up and some of the cool things you can get your agent to do.

Brian Armstrong

214,032 просмотров • 5 месяцев назад

How to self-host your dev environment and run multiple coding agents on your project. This is open-source and free to use. You can run it on your computer or a server in the basement. • Multiple agents • Configurable Templates + Workspaces • Open and free

How to self-host your dev environment and run multiple coding agents on your project. This is open-source and free to use. You can run it on your computer or a server in the basement. • Multiple agents • Configurable Templates + Workspaces • Open and free

Santiago

18,064 просмотров • 1 месяц назад

I’m reverse-engineering the Google AI mouse pointer and making it Open Source! It sees your screen. You can freeform paint anything and ask AI to modify it. It understands the app or window you mean. It can click, type, edit, navigate, and even write + execute code. A tiny cursor becoming a real computer-use agent. Made with the help of Cua which is an AMAZING reference for computer use Sorry Google DeepMind but I had to do this heyclicky Started a revolution and we will be the ones to win Also credits to Omkar Satpute for the help Feel free to modify it , update it and make it your own, we gotta beat them at this. Link to the repo in the comments:

I’m reverse-engineering the Google AI mouse pointer and making it Open Source! It sees your screen. You can freeform paint anything and ask AI to modify it. It understands the app or window you mean. It can click, type, edit, navigate, and even write + execute code. A tiny cursor becoming a real computer-use agent. Made with the help of Cua which is an AMAZING reference for computer use Sorry Google DeepMind but I had to do this heyclicky Started a revolution and we will be the ones to win Also credits to Omkar Satpute for the help Feel free to modify it , update it and make it your own, we gotta beat them at this. Link to the repo in the comments:

Milind S

216,046 просмотров • 6 месяцев назад

AI power users are here and the case for agent-friendly software is materializing. Aaron Levie on agents outnumbering people 1000x: "If you start to imagine that we all have to build software for agents… we spend as much time now thinking about the agent interface to our tool as we do the human interface." "If you have a hundred or a thousand times more agents than people, then your software has to be built for agents… it's going to be through an API or a CLI or MCP." "What if you give a coding agent access to your SaaS tools and a coding agent access to your knowledge work, workflows and context?" "It can actually code its way or use APIs through whatever task it's trying to achieve." "That appears to be like a paradigm that is starting to compound… and I actually think it kind of makes sense as the ultimate manifestation of this stuff." Aaron Levie

AI power users are here and the case for agent-friendly software is materializing. Aaron Levie on agents outnumbering people 1000x: "If you start to imagine that we all have to build software for agents… we spend as much time now thinking about the agent interface to our tool as we do the human interface." "If you have a hundred or a thousand times more agents than people, then your software has to be built for agents… it's going to be through an API or a CLI or MCP." "What if you give a coding agent access to your SaaS tools and a coding agent access to your knowledge work, workflows and context?" "It can actually code its way or use APIs through whatever task it's trying to achieve." "That appears to be like a paradigm that is starting to compound… and I actually think it kind of makes sense as the ultimate manifestation of this stuff." Aaron Levie

a16z

42,152 просмотров • 2 месяцев назад

I built the clipboard I always wanted for working with AI agents. It’s called Bluey and It’s 100% local-first. It lives under your mouse cursor. You can draw on your screen, speak to it, or type what you mean, and Bluey turns all of that into rich context for Claude Code or Codex. I’ve been using it to design websites and debug UI because I can finally say things like “move this here” or “make this section feel cleaner” while pointing at the actual screen. No more writing long explanations. No more manually describing screenshots. No more agents getting lost because they don’t know what “this” or "that" means. You can just point to it, speak or type and it will curate the best possible context for your agents Bluey captures the screenshot, annotation, transcript, app/window context, and the details your agent needs, then sends it directly into your coding session. I am giving it out for free for a while! Im building it with my buddy Omkar Satpute and we would love to hear what you think about it on Discord what context do you wish your agent could understand better?

I built the clipboard I always wanted for working with AI agents. It’s called Bluey and It’s 100% local-first. It lives under your mouse cursor. You can draw on your screen, speak to it, or type what you mean, and Bluey turns all of that into rich context for Claude Code or Codex. I’ve been using it to design websites and debug UI because I can finally say things like “move this here” or “make this section feel cleaner” while pointing at the actual screen. No more writing long explanations. No more manually describing screenshots. No more agents getting lost because they don’t know what “this” or "that" means. You can just point to it, speak or type and it will curate the best possible context for your agents Bluey captures the screenshot, annotation, transcript, app/window context, and the details your agent needs, then sends it directly into your coding session. I am giving it out for free for a while! Im building it with my buddy Omkar Satpute and we would love to hear what you think about it on Discord what context do you wish your agent could understand better?

Milind S

13,342 просмотров • 5 месяцев назад

1-click ai agent coming soon to a mini-app near you 📍 Imagine a library of ready-made ai apps that you can deploy and start using with a click of a button. Crestal team is leveraging Intent Kit and our latest API suite to design everyday agents that you can use in your business, trading, automation or simply for fun. What agents do you want to see in this app?

1-click ai agent coming soon to a mini-app near you 📍 Imagine a library of ready-made ai apps that you can deploy and start using with a click of a button. Crestal team is leveraging Intent Kit and our latest API suite to design everyday agents that you can use in your business, trading, automation or simply for fun. What agents do you want to see in this app?

Crestal

3,117,014 просмотров • 1 год назад

A conversational agent with no hallucinations! This is huge! Here is the $1M problem: Build a customer-facing AI agent that works consistently, doesn't hallucinate, and is fully aligned with the business rules. Check out Parlant, a FREE, open-source library. I've talked about them before, and here is the TL;DR: 1. Parlant gives you 10x more control over what your agent says 2. It helps you control how agents conduct conversations 3. It helps you control the service protocols and rules agents follow 4. It helps you control what agents can and can't say They do something pretty clever: 1. They have a set of predefined potential responses that the agent can use to reply to the customer. 2. When a customer sends a message, the agent generates a fluid response draft using an LLM and matches and renders the closest pre-approved reply template. 3. The agent returns the rendered pre-approved reply, not the one generated by the LLM. This maintains fluidity while avoiding unpredictability. Parlant also uses your domain terminology, invokes your tools, and implements self-critique to craft/evaluate its response iteratively. This pretty much solves any hallucination issues with your agent.

A conversational agent with no hallucinations! This is huge! Here is the $1M problem: Build a customer-facing AI agent that works consistently, doesn't hallucinate, and is fully aligned with the business rules. Check out Parlant, a FREE, open-source library. I've talked about them before, and here is the TL;DR: 1. Parlant gives you 10x more control over what your agent says 2. It helps you control how agents conduct conversations 3. It helps you control the service protocols and rules agents follow 4. It helps you control what agents can and can't say They do something pretty clever: 1. They have a set of predefined potential responses that the agent can use to reply to the customer. 2. When a customer sends a message, the agent generates a fluid response draft using an LLM and matches and renders the closest pre-approved reply template. 3. The agent returns the rendered pre-approved reply, not the one generated by the LLM. This maintains fluidity while avoiding unpredictability. Parlant also uses your domain terminology, invokes your tools, and implements self-critique to craft/evaluate its response iteratively. This pretty much solves any hallucination issues with your agent.

Santiago

64,237 просмотров • 1 год назад

Worktrace AI and computer use complete each other. 🧡 We recently made a big update to Worktrace AI . It not only FINDS repetitive work in your company, but it now AUTOMATES it for you too. Advancements in computer use have unlocked a wave of personal agents. It will unlock enterprise agents too. The missing piece in enterprise is understanding how work happens. Worktrace records your screen and maps out every workflow to surface top automation opportunities. It takes what it has observed, often hundreds of times over, and turns it into an agent that uses the computer exactly the way your team does. We'd love to demo it to you!

Worktrace AI and computer use complete each other. 🧡 We recently made a big update to Worktrace AI . It not only FINDS repetitive work in your company, but it now AUTOMATES it for you too. Advancements in computer use have unlocked a wave of personal agents. It will unlock enterprise agents too. The missing piece in enterprise is understanding how work happens. Worktrace records your screen and maps out every workflow to surface top automation opportunities. It takes what it has observed, often hundreds of times over, and turns it into an agent that uses the computer exactly the way your team does. We'd love to demo it to you!

Angela Jiang

18,678 просмотров • 1 месяц назад

Introducing Tapkit Tapkit is a Mac app that lets your agents use your iPhone. Tell it to do anything and it can use your phone to do it. We've been iterating on Tapkit over the last few months with our earliest users and finally feel like it's ready for general use. Today it is easy to use, supports any agent, and can even use multiple iPhones at the same time. There are so many great things that we can do with this new primitive, but I didn't want to wait any longer to share it with you. Try it now:

Introducing Tapkit Tapkit is a Mac app that lets your agents use your iPhone. Tell it to do anything and it can use your phone to do it. We've been iterating on Tapkit over the last few months with our earliest users and finally feel like it's ready for general use. Today it is easy to use, supports any agent, and can even use multiple iPhones at the same time. There are so many great things that we can do with this new primitive, but I didn't want to wait any longer to share it with you. Try it now:

teej

579,577 просмотров • 1 месяц назад

introducing ghosthand it lets your agents use a real iPhone plugged into your Mac. for agents & humans the agent sees the screen over USB and taps, swipes and types over Bluetooth, so it can use any app on the phone, even the ones with no API try it at

introducing ghosthand it lets your agents use a real iPhone plugged into your Mac. for agents & humans the agent sees the screen over USB and taps, swipes and types over Bluetooth, so it can use any app on the phone, even the ones with no API try it at

Tommy D. Rossi

49,714 просмотров • 16 дней назад

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

We just submitted our project to the OpenAI Developers WebMCP hackathon: The WebMCP Computer. WebMCP Computer is an operating system that lives inside the browser, with WebMCP as its native control layer. It is a computer that treats agents as first-class citizens, where files, processes, applications, windows, and the terminal are exposed directly through WebMCP. You can send anyone one URL that becomes a disposable computer for any agent task. We created a computer instance in ChatGPT Sites, open it with Codex’s in-app browser and try it here: Some examples of what you can do with WebMCP Computer: 1/ Give a coding agent a fresh computer to clone a repository, edit code, run tests, and preview the result. 2/ Let an agent inspect logs, manage processes, change configurations, and repair a broken system. 3/ Give a research agent its own workspace to browse, download files, analyze data, and create a report. 4/ Create reproducible environments for evaluating and comparing different agents. 5/ Let humans and agents work together in the same environment, with the agent using WebMCP and the human using the GUI. The list of applications is endless, with more examples in our GitHub. The project is fully open source under the MIT license. Created by our awesome founding engineer Ilay Alog.

idan levin

28,385 просмотров • 2 месяцев назад

Today, Sierra is releasing Ghostwriter, our agent for building agents. With Ghostwriter, you can create an AI agent for your customer experience — one that can chat, pick up the phone, speak dozens of languages, take action on your systems of record, and be protected with industry-leading guardrails — simply by having a conversation. No clicking, no forms, no menus. Codex and Claude Code have transformed how we build software, making it possible for software engineers to orchestrate and review the work rather than doing all the work themselves. We think the same transformation will happen for all software. Rather than every enterprise app having a web app for humans and an API for automation, every software platform’s UI will be an agent that can do the work on your behalf. I recorded a demo of my building and optimizing an agent with Ghostwriter so you can see how powerful and easy it is to use. It’s completely changed the way our early adopters build agents, and it’s changed the way I think about the software industry. Let me know what you think, and, if you’re interested in trying it out at your business, please reach out directly.

Today, Sierra is releasing Ghostwriter, our agent for building agents. With Ghostwriter, you can create an AI agent for your customer experience — one that can chat, pick up the phone, speak dozens of languages, take action on your systems of record, and be protected with industry-leading guardrails — simply by having a conversation. No clicking, no forms, no menus. Codex and Claude Code have transformed how we build software, making it possible for software engineers to orchestrate and review the work rather than doing all the work themselves. We think the same transformation will happen for all software. Rather than every enterprise app having a web app for humans and an API for automation, every software platform’s UI will be an agent that can do the work on your behalf. I recorded a demo of my building and optimizing an agent with Ghostwriter so you can see how powerful and easy it is to use. It’s completely changed the way our early adopters build agents, and it’s changed the way I think about the software industry. Let me know what you think, and, if you’re interested in trying it out at your business, please reach out directly.

Bret Taylor

1,065,954 просмотров • 7 месяцев назад

Hermes Cloud agents now open in a browser tab with the exact same interface as Hermes Desktop. To launch a personal agent, pick a name and a model in Nous Portal. We host it for you and it runs 24/7 in its own container, so it can handle scheduled jobs and answer you on any channel while your laptop is closed. Chats, Group Chats, plugins and settings in the Webapp work the same way they do when you run Hermes on your own computer. Open your agent from Nous Portal on your phone or any computer, or connect Hermes Desktop to it and continue the same chats there. Plus, Super, and Ultra plans each include one Cloud agent out of the box. Using Hermes from anywhere has never been easier.

Hermes Cloud agents now open in a browser tab with the exact same interface as Hermes Desktop. To launch a personal agent, pick a name and a model in Nous Portal. We host it for you and it runs 24/7 in its own container, so it can handle scheduled jobs and answer you on any channel while your laptop is closed. Chats, Group Chats, plugins and settings in the Webapp work the same way they do when you run Hermes on your own computer. Open your agent from Nous Portal on your phone or any computer, or connect Hermes Desktop to it and continue the same chats there. Plus, Super, and Ultra plans each include one Cloud agent out of the box. Using Hermes from anywhere has never been easier.

Nous Research

106,766 просмотров • 2 дней назад

New course: Spec-Driven Development with Coding Agents, built in partnership with JetBrains, and taught by Paul Everitt | @pauleveritt@fosstodon.org. Vibe coding is fast, but often produces code that doesn't match what you asked for. This short course teaches you spec-driven development: write a detailed spec defining what to build, and work with your coding agent to implement it. Many of the best developers already build this way. A spec lets you control large code changes with a few words, preserve context across agent sessions, and stay in control as your project grows in complexity. Skills you'll gain: - Write a detailed specification to define your mission, tech stack, and roadmap, giving your agent the context it needs from the start - Plan, implement, and validate features in iterative loops using a spec as your agent's guide - Apply the same repeatable workflow to both new and legacy codebases - Package your workflow into a portable agent skill that works across agents and IDEs Join and write specs that keep your coding agent on track!

New course: Spec-Driven Development with Coding Agents, built in partnership with JetBrains, and taught by Paul Everitt | @[email protected]. Vibe coding is fast, but often produces code that doesn't match what you asked for. This short course teaches you spec-driven development: write a detailed spec defining what to build, and work with your coding agent to implement it. Many of the best developers already build this way. A spec lets you control large code changes with a few words, preserve context across agent sessions, and stay in control as your project grows in complexity. Skills you'll gain: - Write a detailed specification to define your mission, tech stack, and roadmap, giving your agent the context it needs from the start - Plan, implement, and validate features in iterative loops using a spec as your agent's guide - Apply the same repeatable workflow to both new and legacy codebases - Package your workflow into a portable agent skill that works across agents and IDEs Join and write specs that keep your coding agent on track!

Andrew Ng

467,452 просмотров • 7 месяцев назад

Introducing Sarvam Voice Agents MCP. Describe the voice agent you want, and your coding tool builds it. The MCP connects your coding tool to Sarvam, so you can create agents, run campaigns, and analyse calls from the environment you already code in. Watch Sarvam Voice Agents MCP build a voice agent, call a list of people and report back. Build your first voice agent with the MCP:

Introducing Sarvam Voice Agents MCP. Describe the voice agent you want, and your coding tool builds it. The MCP connects your coding tool to Sarvam, so you can create agents, run campaigns, and analyse calls from the environment you already code in. Watch Sarvam Voice Agents MCP build a voice agent, call a list of people and report back. Build your first voice agent with the MCP:

Sarvam

20,218 просмотров • 27 дней назад

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

People are overlooking Google Gemini Realtime models for computer use It gave me sub 100ms latency with computer use. It has a much larger context window and is much cheaper as well Combine that with local OCR and local screen detection model based on Omniparser by Microsoft it works under 100ms action taking when combined with Cua I also put in a harness for Nous Research Hermes with it. You can access it all at your tip of your cursor. You can draw on your screen to give your agents a context And I am making it Open Source! Link in the Comments Sundar Pichai Min-Liang Tan Mojtaba Seyedhosseini

Milind S

38,361 просмотров • 5 месяцев назад

Today we open the Zapier SDK to everyone. If you're building with AI agents, this is for you. I've been using this for 2 months. It's totally changed how I do my job. You install it in your coding agent. Cursor, Claude Code, Codex, whatever you use. Now that agent has access to 8,000+ apps through Zapier and can do anything those APIs can do. I think it’s the most powerful thing we’ve launched in years. Now in open beta. Just give this link right to your agent:

Today we open the Zapier SDK to everyone. If you're building with AI agents, this is for you. I've been using this for 2 months. It's totally changed how I do my job. You install it in your coding agent. Cursor, Claude Code, Codex, whatever you use. Now that agent has access to 8,000+ apps through Zapier and can do anything those APIs can do. I think it’s the most powerful thing we’ve launched in years. Now in open beta. Just give this link right to your agent:

Wade Foster

178,259 просмотров • 7 месяцев назад

.Postman is now natively available in your favorite coding agent! The new Postman CLI is designed to bring API engineering for agents: 1️⃣ Works in Git and your code repo - Just drop Postman where your agent lives and works. 2️⃣ No GUI - Work in your coding agent. Bring up the Postman UI for review and approval. 3️⃣ No sign-up - Get started right away. Sign-up when you have to save your work and want more. Install once, work in your agent. Run npm install -g postman-cli, then postman init. This loads Postman's skills into your coding agent so it can create collections, write tests, and much more. Available as a plugin for all coding agents! The skills cover the API lifecycle: find APIs your org already has, generate tests that catch regressions, mock an API before it exists, watch live endpoints for downtime and drift, run checks in CI on every commit, and run performance tests. You can also extend the abilities of your agent with the Context Graph. An agent often sees only the code that is accessible to them. The Context Graph API gives it a map of your team's infrastructure, APIs, and services, and how they depend on each other. As your agent works, you stay the reviewer. Your agent does the work from the command line. Open Postman when you want to review or approve it. Check out more at

.Postman is now natively available in your favorite coding agent! The new Postman CLI is designed to bring API engineering for agents: 1️⃣ Works in Git and your code repo - Just drop Postman where your agent lives and works. 2️⃣ No GUI - Work in your coding agent. Bring up the Postman UI for review and approval. 3️⃣ No sign-up - Get started right away. Sign-up when you have to save your work and want more. Install once, work in your agent. Run npm install -g postman-cli, then postman init. This loads Postman's skills into your coding agent so it can create collections, write tests, and much more. Available as a plugin for all coding agents! The skills cover the API lifecycle: find APIs your org already has, generate tests that catch regressions, mock an API before it exists, watch live endpoints for downtime and drift, run checks in CI on every commit, and run performance tests. You can also extend the abilities of your agent with the Context Graph. An agent often sees only the code that is accessible to them. The Context Graph API gives it a map of your team's infrastructure, APIs, and services, and how they depend on each other. As your agent works, you stay the reviewer. Your agent does the work from the command line. Open Postman when you want to review or approve it. Check out more at

Abhinav Asthana

494,079 просмотров • 1 месяц назад

Over 90,000 Skills sitting inside Hermes Desktop is kind of insane!!!🤯 Did you know Hermes Desktop has a built-in Skills Hub with more than 90,000 Skills you can browse and add to an agent? And you can do it right from Desktop. Go to: Capabilities → Skills From there, open the Skills Hub under your installed Skills. You can: • search the catalog • browse by category or source • open any Skill to read what it does • review platform support, author, version, requirements, and docs • click + Add to this Agent to install it directly to the agent/profile you have selected That means if you want Hermes to get better at a specific type of work, you do not have to go hunting through repos or memorize install commands first. Just open the Hub, look through the catalog, inspect the Skill, and add it with one click. A lot of users still have no idea this is sitting inside Hermes Desktop.

Hermes Release Watch

68,830 просмотров • 2 месяцев назад