Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

The Dawn of GUI Agent A Preliminary Case Study with Claude 3.5 Computer Use Game (Honkai: Star Rail) Claude 3.5 Computer Use can help complete Honkai: Star Rail daily tasks, accurately locating and interacting with in-game elements.

AK

508,358 subscribers

81,471 views • 1 year ago •via X (Twitter)

Science & Technology

Anya Rossi• Live Now

Private livecam show

10 Comments

AK1 year ago

discuss:

AK1 year ago

github:

OIiver1 year ago

For the low low price of 50 dollars each run

Paul Calcraft1 year ago

Claude 3.5 computer use may be good for point and click, but still no VLM can play non-awful moves in tic tac toe. I hope none of your tasks require thinking one or two moves ahead!

Pseudonym 🦅1 year ago

When the AI has better luck at character pulls 😭

Alejandro Furman1 year ago

@LernerBenjamin

Cris Can1 year ago

Awesome!

Ayush Thakur1 year ago

cc: @soumikRakshit96 context: We have been trying to play Divine: Original Sin 2 with LLMs.

Thomas Capelle1 year ago

wow!

Crystalwizard1 year ago

why does it have to be a stereotypical anime female?

Related Videos

. Honkai: Star Rail 2025 HoYoFair "Chimerric Park" is now streaming exclusively on Crunchyroll! ✨ Complete missions to earn Crunchyroll and Honkai Star Rail in-game rewards! 👉:

. Honkai: Star Rail 2025 HoYoFair "Chimerric Park" is now streaming exclusively on Crunchyroll! ✨ Complete missions to earn Crunchyroll and Honkai Star Rail in-game rewards! 👉:

Crunchyroll

204,685 views • 9 months ago

Me and lily🪷 made a new song in collaboration with Honkai: Star Rail ◡̈ Link in comments!

Me and lily🪷 made a new song in collaboration with Honkai: Star Rail ◡̈ Link in comments!

TJ BROWN

79,390 views • 1 year ago

Honkai: Star Rail Concert | "My Friend, How About a Little Game?" The Aventurine of Stratagems sets up a scheme, and the light sticks are at stake! "Bust? Or maybe I'll take it all!" Star Rail LIVE Honkai: Star Rail Concert 2025 will start streaming on May 3rd, 2025, 19:05 (UTC+8). Join us for this special moment of universal resonance! Get notified: #StarRailLIVE2025 #HonkaiStarRail #StarRail

Honkai: Star Rail Concert | "My Friend, How About a Little Game?" The Aventurine of Stratagems sets up a scheme, and the light sticks are at stake! "Bust? Or maybe I'll take it all!" Star Rail LIVE Honkai: Star Rail Concert 2025 will start streaming on May 3rd, 2025, 19:05 (UTC+8). Join us for this special moment of universal resonance! Get notified: #StarRailLIVE2025 #HonkaiStarRail #StarRail

Honkai: Star Rail

618,838 views • 1 year ago

Claude computer use is too expensive check out ShowUI Local Run! Now, you can use a lightweight model to power a local GUI Agent. It's compatible with "GPT-4o + ShowUI" and "Qwen2-VL + ShowUI" Each task costs just a few cents—30x cheaper than Claude!

Claude computer use is too expensive check out ShowUI Local Run! Now, you can use a lightweight model to power a local GUI Agent. It's compatible with "GPT-4o + ShowUI" and "Qwen2-VL + ShowUI" Each task costs just a few cents—30x cheaper than Claude!

AK

30,967 views • 1 year ago

You can now enable Claude to use your computer to complete tasks. It opens your apps, navigates your browser, fills in spreadsheets—anything you'd do sitting at your desk. Research preview in Claude Cowork and Claude Code, macOS only.

You can now enable Claude to use your computer to complete tasks. It opens your apps, navigates your browser, fills in spreadsheets—anything you'd do sitting at your desk. Research preview in Claude Cowork and Claude Code, macOS only.

Claude

78,300,646 views • 5 months ago

Honkai: Star Rail World Tour 2023 Recap Hello, Trailblazers! The Honkai: Star Rail World Tour 2023 has finally concluded. Thank you for everyone's support and companionship! Let's take a look at some of our favorite highlights of this year~ The bells of 2024 are poised to ring, heralding the start of a fresh new journey! Stay tuned for our next Star Rail World Tour in 2024! #HonkaiStarRail #SRWorldTour2023

Honkai: Star Rail World Tour 2023 Recap Hello, Trailblazers! The Honkai: Star Rail World Tour 2023 has finally concluded. Thank you for everyone's support and companionship! Let's take a look at some of our favorite highlights of this year~ The bells of 2024 are poised to ring, heralding the start of a fresh new journey! Stay tuned for our next Star Rail World Tour in 2024! #HonkaiStarRail #SRWorldTour2023

Honkai: Star Rail

345,818 views • 2 years ago

A look at computer use and the built-in browser in the new ChatGPT app 👀 dominik kundel walks through how ChatGPT can work with apps on your computer and browse the web to research, navigate websites, and complete tasks with you.

A look at computer use and the built-in browser in the new ChatGPT app 👀 dominik kundel walks through how ChatGPT can work with apps on your computer and browse the web to research, navigate websites, and complete tasks with you.

ChatGPT

125,422 views • 1 month ago

You can now use the Claude Code and Codex harnesses directly whenever you start a cloud agent in Warp. Here's engineer Lili Wilson to show how to implement a real feature in the cloud, with computer use and automated code review to validate work:

You can now use the Claude Code and Codex harnesses directly whenever you start a cloud agent in Warp. Here's engineer Lili Wilson to show how to implement a real feature in the cloud, with computer use and automated code review to validate work:

Warp

14,055 views • 3 months ago

Thanks to Honkai: Star Rail for giving me this opportunity of making the cover of "If I Can Stop One Heart From Breaking". Now download the game for Version 2.2!!: #HonkaiStarRail #崩壊スターレイル #ad

Thanks to Honkai: Star Rail for giving me this opportunity of making the cover of "If I Can Stop One Heart From Breaking". Now download the game for Version 2.2!!: #HonkaiStarRail #崩壊スターレイル #ad

Unnämed

441,583 views • 2 years ago

Claude Sonnet 3.5 Artifacts is now available to use with GPT-4o, Gemini, Llama-3 and other LLMs for just $10 a month. Build interactive experiences, search the web, generate images and audio with GPT-4o and Claude Sonnet 3.5 in just one AI playground.

Claude Sonnet 3.5 Artifacts is now available to use with GPT-4o, Gemini, Llama-3 and other LLMs for just $10 a month. Build interactive experiences, search the web, generate images and audio with GPT-4o and Claude Sonnet 3.5 in just one AI playground.

Shubham Saboo

27,452 views • 2 years ago

We're trying something fundamentally new. Instead of making specific tools to help Claude complete individual tasks, we're teaching it general computer skills—allowing it to use a wide range of standard tools and software programs designed for people.

We're trying something fundamentally new. Instead of making specific tools to help Claude complete individual tasks, we're teaching it general computer skills—allowing it to use a wide range of standard tools and software programs designed for people.

Anthropic

535,849 views • 1 year ago

Stop paying for Claude Code! Use it for FREE with ollama I just set up Claude Code running locally on my computer. Zero API costs, complete privacy, and it actually works for real coding tasks. Full step-by-step guide in my latest video:

Stop paying for Claude Code! Use it for FREE with ollama I just set up Claude Code running locally on my computer. Zero API costs, complete privacy, and it actually works for real coding tasks. Full step-by-step guide in my latest video:

Mervin Praison

57,732 views • 6 months ago

After testing Kimi.ai K2 for a few hours, My overall take: - Performance between Claude 3.5 & Claude 4 (Just my vibe eval!) - The UI generation seems great - But the cost is only 20% of Claude 3.5 - So good enough for most coding agent with a lot more manageable cost Easiest way to use Kimi K2 in Claude Code: - export ANTHROPIC_AUTH_TOKEN=YOUR_MOONSHOT_API - export ANTHROPIC_BASE_URL= - claude ------ Here are my tests & how to integrate to Claude Code: 1. Whole UI component library 2. Make online IDE UI 3. Build Functional mario game ...

After testing Kimi.ai K2 for a few hours, My overall take: - Performance between Claude 3.5 & Claude 4 (Just my vibe eval!) - The UI generation seems great - But the cost is only 20% of Claude 3.5 - So good enough for most coding agent with a lot more manageable cost Easiest way to use Kimi K2 in Claude Code: - export ANTHROPIC_AUTH_TOKEN=YOUR_MOONSHOT_API - export ANTHROPIC_BASE_URL= - claude ------ Here are my tests & how to integrate to Claude Code: 1. Whole UI component library 2. Make online IDE UI 3. Build Functional mario game ...

Jason Zhou

80,871 views • 1 year ago

3/ Robot use - 🥇1st place Combining Claude computer use with a robot arm and have the LLM perform real world actions Amazon robot controlled just by uploading the instruction manual to Claude

3/ Robot use - 🥇1st place Combining Claude computer use with a robot arm and have the LLM perform real world actions Amazon robot controlled just by uploading the instruction manual to Claude

Alex Reibman 🖇️

60,741 views • 1 year ago

Introducing Scrapybara Agent API Here is a Claude Computer Use agent scraping YC W25 companies, finding best contact methods, and drafting personalized messages in a word doc 3 function calls, <50 lines of code computer dot new

Introducing Scrapybara Agent API Here is a Claude Computer Use agent scraping YC W25 companies, finding best contact methods, and drafting personalized messages in a word doc 3 function calls, <50 lines of code computer dot new

justin

80,779 views • 1 year ago

Honestly not every game gets the treatment Honkai Star Rail gets from Hoyoverse Just look at the sheer quality and you even got Milli On a FAN WORK?!?! What the hell Hoyo lol #HonkaiStarRail

Honestly not every game gets the treatment Honkai Star Rail gets from Hoyoverse Just look at the sheer quality and you even got Milli On a FAN WORK?!?! What the hell Hoyo lol #HonkaiStarRail

Genshin Animation Info

59,954 views • 5 months ago

Happy 3rd Anniversary, Young Ones! ❤️ It has been such a joy to portray Himeko in Honkai: Star Rail. Thank you for sharing your love of her and the game with me! May this journey lead us starward! 🥂🚂 #voiceactor #honkaistarrail #hsr #himeko #voxgoddess

Happy 3rd Anniversary, Young Ones! ❤️ It has been such a joy to portray Himeko in Honkai: Star Rail. Thank you for sharing your love of her and the game with me! May this journey lead us starward! 🥂🚂 #voiceactor #honkaistarrail #hsr #himeko #voxgoddess

Cia Court 🎧 Voice Actor

231,904 views • 3 months ago

Keeping up with Star Rail — Moze: Far-Flung Chase Without Pause | Honkai: Star Rail Watch Now: Good evening, Moze. Your destination tonight is the set of the show Keeping Up With Star Rail. Your target is the host of this show, the star host of Interastral Peace Entertainment — Owlbert. You need to charm the audience with your charisma and flair. This is a challenging task, but I know you love a challenge. I won't disturb you while you prepare. Thank you for tuning in to this special Interastral Peace Entertainment program, Keeping Up With Star Rail — Moze: Far-Flung Chase Without Pause English Voice: Owlbert — Bill Butts #HonkaiStarRail #StarRail

Keeping up with Star Rail — Moze: Far-Flung Chase Without Pause | Honkai: Star Rail Watch Now: Good evening, Moze. Your destination tonight is the set of the show Keeping Up With Star Rail. Your target is the host of this show, the star host of Interastral Peace Entertainment — Owlbert. You need to charm the audience with your charisma and flair. This is a challenging task, but I know you love a challenge. I won't disturb you while you prepare. Thank you for tuning in to this special Interastral Peace Entertainment program, Keeping Up With Star Rail — Moze: Far-Flung Chase Without Pause English Voice: Owlbert — Bill Butts #HonkaiStarRail #StarRail

Honkai: Star Rail

213,439 views • 2 years ago