Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We're launching Computer Use in smolagents! 🥳 -> As vision models become more capable, they become able to power complex agentic workflows. Especially Qwen-VL models, that support built-in grounding, i.e. ability to locate any element in an image by its coordinates, thus to click any item on a screenshot.... show more

m_ric

7,670 subscribers

43,850 views • 1 year ago •via X (Twitter)

Science & Technology News & Politics Education

Anya Rossi• Live Now

Private livecam show

11 Comments

m_ric1 year ago

Try the demo here 👉

MightyBot1 year ago

🧠 Unified Search. Smarter Meetings. Effortless CRM. MightyBot is your AI agent platform for seamless workflows—record meetings, automate CRM updates, and find answers across apps in seconds. 🌟 Focus on what matters. We'll handle the grind.

Dhruv Diddi1 year ago

@Thom_Wolf Multimodal models are going to pick up real fast ⚡️

Oli1 year ago

i think ui tars 1.5 7b would work better here its much better at this

Piyush1 year ago

Awesome! Gonna try this out! 😁

ぽいロード@音技術者1 year ago

Hey Computer use AI ,please use 3d modeling soft and create this

Van Tuan Bui1 year ago

Congrats on the launch! 🎉

Sam Woods1 year ago

Smolagents with computer use powered by vision models like Qwen-VL sounds really powerful

蕭夫SioFU1 year ago

@huggingface cool

Trelis Research1 year ago

nice, what vlms did you test out that work best?

gladosb51 year ago

hmmmmmm

Related Videos

in Singapore malls they literally use computer vision models and OCR every license plate so that you can look up your car and get personalized directions to your car from anywhere in the mall

in Singapore malls they literally use computer vision models and OCR every license plate so that you can look up your car and get personalized directions to your car from anywhere in the mall

swyx 🔜 @aiDotEngineer

107,878 views • 6 months ago

I can't tell you the last time I was so excited to see a new AI capability in action. We plugged in Claude computer use in Replit ⠕ Agent as a human feedback replacement. And… it just works! I feel it won't take long until our agent will become fully autonomous.

I can't tell you the last time I was so excited to see a new AI capability in action. We plugged in Claude computer use in Replit ⠕ Agent as a human feedback replacement. And… it just works! I feel it won't take long until our agent will become fully autonomous.

Michele Catasta

185,920 views • 1 year ago

Hermes won. They just dropped their desktop app and it's excellent It's now the best way to use AI agents on your computer In this video I cover how to set it up, how to use it, and go through EVERY feature in the app Bye bye Telegram

Hermes won. They just dropped their desktop app and it's excellent It's now the best way to use AI agents on your computer In this video I cover how to set it up, how to use it, and go through EVERY feature in the app Bye bye Telegram

Alex Finn

176,035 views • 1 month ago

President Trump on NATO : 🇺🇸 We're very disappointed with NATO, very, very disappointed that they didn't come. Now they want to come and they want to help with the Strait. And it won't take long to clean it out. So we're going to clean out the Strait and they'll be able to use the Strait in not too long a distance.

President Trump on NATO : 🇺🇸 We're very disappointed with NATO, very, very disappointed that they didn't come. Now they want to come and they want to help with the Strait. And it won't take long to clean it out. So we're going to clean out the Strait and they'll be able to use the Strait in not too long a distance.

Donald J Trump Posts TruthSocial

53,216 views • 2 months ago

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Rowan Cheung

1,024,377 views • 1 year ago

Turns out, that viral video from The Oversight Project of immigrants in Norcross, GA claiming they were registered to vote — and some had already voted — was all a lie. They admitted they were not able to find any of them in the voter rolls and investigators didn't find any of them had registed to vote.

Turns out, that viral video from The Oversight Project of immigrants in Norcross, GA claiming they were registered to vote — and some had already voted — was all a lie. They admitted they were not able to find any of them in the voter rolls and investigators didn't find any of them had registed to vote.

Blue Georgia

106,939 views • 1 year ago

After coding is solved, the next frontier is computer use. Today, we are launching Use Computer, the infra for evaluating and training models to use all kinds of computers 👇

After coding is solved, the next frontier is computer use. Today, we are launching Use Computer, the infra for evaluating and training models to use all kinds of computers 👇

Marco Mascorro

44,641 views • 28 days ago

Today we’re announcing a finding that breaks a core assumption in AI: that bigger models are harder to understand. We show the opposite. When interpretability is built into training, models become MORE understandable as they become more capable.

Today we’re announcing a finding that breaks a core assumption in AI: that bigger models are harder to understand. We show the opposite. When interpretability is built into training, models become MORE understandable as they become more capable.

Guide Labs

14,861 views • 28 days ago

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Andrew Ng

170,335 views • 1 year ago

Trump is babbling complete nonsense “Hopefully they'll be able to get these expensive computers going. You know, the reason you use computers is to make time so that it's like fast. You don't use them so that you have to come up with an answer three days later. And that's a little scary when they say what, what are they doing? You use a computer because it calculates quickly and you use paper because you save cost, but the paper turns out to be much quicker than the computer”

Trump is babbling complete nonsense “Hopefully they'll be able to get these expensive computers going. You know, the reason you use computers is to make time so that it's like fast. You don't use them so that you have to come up with an answer three days later. And that's a little scary when they say what, what are they doing? You use a computer because it calculates quickly and you use paper because you save cost, but the paper turns out to be much quicker than the computer”

Headquarters

1,599,470 views • 1 year ago

Epic duo! computer use drives cursor autonomously and builds and entire Tower defense game while maintaining project understanding 🥳 system message in comment You just instruct computer use and it continually instructs Cursor agent to build the project. You can download the script from my patron. link in comment How it works: - Computer is limited to certain coordinates and certain actions this way it should only work with cursor and do nothing else - cursorrules file instructs cursor to not run any commands to run files, since those require user permission and computer use may make mistakes there - you select the coordinates of the top and bottom position of the text input boxes in composer side panel. there is a script that helps you with that. we enter these into comuter_use script - then run the script with your prompt let me know what you think 🙂 ------- If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

Epic duo! computer use drives cursor autonomously and builds and entire Tower defense game while maintaining project understanding 🥳 system message in comment You just instruct computer use and it continually instructs Cursor agent to build the project. You can download the script from my patron. link in comment How it works: - Computer is limited to certain coordinates and certain actions this way it should only work with cursor and do nothing else - cursorrules file instructs cursor to not run any commands to run files, since those require user permission and computer use may make mistakes there - you select the coordinates of the top and bottom position of the text input boxes in composer side panel. there is a script that helps you with that. we enter these into comuter_use script - then run the script with your prompt let me know what you think 🙂 ------- If you are interested in learning how to use Cursor more in depth, I have a 28 chapter 1000x Cursor Course building full apps from scratch with 19 hours of content. link is in my bio if you are interested

echo.hive

126,688 views • 1 year ago

Zelensky to Sky News: They don't use social media in the center of their capital; we can use it on the battlefield. Nobody stops our soldiers. They can do it when they have pauses.

Zelensky to Sky News: They don't use social media in the center of their capital; we can use it on the battlefield. Nobody stops our soldiers. They can do it when they have pauses.

Clash Report

93,857 views • 1 month ago

Pi-GUI now has computer use and a build in integrated terminal! v0.1.0-beta.26 To showcase the power of Mario Zechner 's Pi and OSS, I saw a pi computer-use extension on X, asked my agent to audit the extension for security, then asked it to patch pi-gui to make it compatible. Crazy how easy it was to find an oss extension for a feature I wanted, then make pi-gui work with the feature I want. Links below

Pi-GUI now has computer use and a build in integrated terminal! v0.1.0-beta.26 To showcase the power of Mario Zechner 's Pi and OSS, I saw a pi computer-use extension on X, asked my agent to audit the extension for security, then asked it to patch pi-gui to make it compatible. Crazy how easy it was to find an oss extension for a feature I wanted, then make pi-gui work with the feature I want. Links below

Matthew Lam

65,535 views • 2 months ago

Flashback to when I found a homeless family who got stranded in Bend on their way to Alaska where the husband had a job. They were unsuccessful in getting any help from social services. They had their autistic daughter with them. I did a fundraiser and same day

Flashback to when I found a homeless family who got stranded in Bend on their way to Alaska where the husband had a job. They were unsuccessful in getting any help from social services. They had their autistic daughter with them. I did a fundraiser and same day

Kevin Dahlgren 🥾 🥾

80,144 views • 1 year ago

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Philip Vollet

41,450 views • 2 years ago

#WATCH | Cyprus: On PM Modi's visit to Cyprus, Manish, High Commissioner of India to the Republic of Cyprus, says, "I am trying to push the economic agenda between the India-Cyprus Economic Corridor. We have a policy to become a ‘developed India’ by 2047, and they have ‘Vision 2035’, so there is a synergy there, so we have to see how we can use each other’s strengths and work together to achieve them. I think Cyprus’ advantage is that it can serve as a gateway to the European Union as well as the larger region around it. Cyprus also understands that it has to work with India and it is quite evident that they are reaching out to India to build better relations, to focus more on economic issues, because they know that India is the fastest growing economy in the world. And similarly, if you see, Cyprus is the fastest growing economy in the European Union and they have their own advantages. We have our own strengths and advantages, so we have to work on every aspect of economic engagement…"

#WATCH | Cyprus: On PM Modi's visit to Cyprus, Manish, High Commissioner of India to the Republic of Cyprus, says, "I am trying to push the economic agenda between the India-Cyprus Economic Corridor. We have a policy to become a ‘developed India’ by 2047, and they have ‘Vision 2035’, so there is a synergy there, so we have to see how we can use each other’s strengths and work together to achieve them. I think Cyprus’ advantage is that it can serve as a gateway to the European Union as well as the larger region around it. Cyprus also understands that it has to work with India and it is quite evident that they are reaching out to India to build better relations, to focus more on economic issues, because they know that India is the fastest growing economy in the world. And similarly, if you see, Cyprus is the fastest growing economy in the European Union and they have their own advantages. We have our own strengths and advantages, so we have to work on every aspect of economic engagement…"

ANI

46,372 views • 1 year ago

Introducing Virtual Computer We're giving you a persistent computer with built-in agents that use the web, develop software, and get smarter over time. Create and deploy a website. Generate and edit videos. Automatically fix bugs as they get reported. Your AI team works 24/7 to achieve your goal. Reply with what you'd build - gifting $50 in credits to the most interesting answers

Introducing Virtual Computer We're giving you a persistent computer with built-in agents that use the web, develop software, and get smarter over time. Create and deploy a website. Generate and edit videos. Automatically fix bugs as they get reported. Your AI team works 24/7 to achieve your goal. Reply with what you'd build - gifting $50 in credits to the most interesting answers

Nebula

19,128 views • 3 months ago

Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.

Today, we’re releasing a feature that allows Claude to control your computer: Mouse, keyboard, and screen, giving it the ability to use any app. I believe this is especially useful if used with Dispatch, which allows you to remotely control Claude on your computer while you’re away.

Felix Rieseberg

4,786,731 views • 3 months ago

Open Computer Agent - LLMs completing tasks using a VM. It's playground to test how well current LLM agents use a computer to solve everyday tasks. And this is just the start - very soon models will be 10x faster and 10x better at it! ❤️ built with e2b x qwen2.5-vl x smolagent

Open Computer Agent - LLMs completing tasks using a VM. It's playground to test how well current LLM agents use a computer to solve everyday tasks. And this is just the start - very soon models will be 10x faster and 10x better at it! ❤️ built with e2b x qwen2.5-vl x smolagent

Leandro von Werra

17,126 views • 1 year ago