Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Muse Spark 1.1 also excels in perception and multimodal reasoning, inspecting visual and audio inputs, preserving details across long workflows, and acting on them in real execution environments. It shows particular strengths in visual-to-code generation, rich image/video captioning, and agentic computer use. In this demo, using video shot from... show more

AI at Meta

823,341 subscribers

69,239 views • 6 days ago •via X (Twitter)

Science & Technology Education Arts

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

Muse Spark 1.1 is here! Available today via API and on Of all the agentic use cases, I’m most excited about this one: going from a single video shot straight to task completion. 🚀 We’ve also polished and launched Computer Use and Video-Audio Perception for Muse Spark 1.1. Read more:

Muse Spark 1.1 is here! Available today via API and on Of all the agentic use cases, I’m most excited about this one: going from a single video shot straight to task completion. 🚀 We’ve also polished and launched Computer Use and Video-Audio Perception for Muse Spark 1.1. Read more:

Jiahui Yu

109,470 views • 6 days ago

Introducing Meta Perception Encoder: a vision encoder setting new standards in image & video tasks. It excels in zero-shot classification & retrieval, surpassing existing models. Learn more about Meta Perception Encoder, read the research paper, and download the code and dataset

Introducing Meta Perception Encoder: a vision encoder setting new standards in image & video tasks. It excels in zero-shot classification & retrieval, surpassing existing models. Learn more about Meta Perception Encoder, read the research paper, and download the code and dataset

AI at Meta

74,531 views • 1 year ago

META 🔥: Muse Image, the first image-gen model from MSL, is now available on Meta AI. > It uses advanced reasoning to understand complex prompts, seamlessly blending multiple photos into high-quality creations you can download and share anywhere. Users can also access Presets, mention images in the prompt, and edit images directly. Muse Image is also available on WhatsApp and Instagram and is planned to arrive on Messenger and Facebook later. Testing time 👀

META 🔥: Muse Image, the first image-gen model from MSL, is now available on Meta AI. > It uses advanced reasoning to understand complex prompts, seamlessly blending multiple photos into high-quality creations you can download and share anywhere. Users can also access Presets, mention images in the prompt, and edit images directly. Muse Image is also available on WhatsApp and Instagram and is planned to arrive on Messenger and Facebook later. Testing time 👀

🚨 AI News | TestingCatalog

40,989 views • 7 days ago

Code Arena can handle image inputs for agentic web dev tasks, reasoning through multi-step problems and using tools along the way. Watch how it works with Aryan Vichare in this clip. Find a link to the full walkthrough on how Code Arena creates sites and apps from images in thread.

Code Arena can handle image inputs for agentic web dev tasks, reasoning through multi-step problems and using tools along the way. Watch how it works with Aryan Vichare in this clip. Find a link to the full walkthrough on how Code Arena creates sites and apps from images in thread.

Arena.ai

19,933 views • 3 months ago

muse spark 1.1 in AdaL Engineer beats Opus4.8 in Claude Code with 20% of the cost loop engineering, when done right, is beyond just running longer, its delivering better results when contexts are managed well and when workers are better prompted to stay honest. how GANs had changed image generation is how loop engineering is changing agentic engineering. muse sparking is truly musing the world, bringing us to the next more autonomous era.

muse spark 1.1 in AdaL Engineer beats Opus4.8 in Claude Code with 20% of the cost loop engineering, when done right, is beyond just running longer, its delivering better results when contexts are managed well and when workers are better prompted to stay honest. how GANs had changed image generation is how loop engineering is changing agentic engineering. muse sparking is truly musing the world, bringing us to the next more autonomous era.

Li Yin

37,943 views • 5 days ago

Most AI demos stop at text prompts. I wanted to see what Qwen3.7-Plus could do beyond chat, so I tested it through the API using an Airbnb interface screenshot. The model analyzed the UI, identified key components, explained the visual hierarchy, highlighted usability strengths, suggested UX improvements, and even generated a React + Tailwind structure to recreate the interface. A few things that stood out: 👁️ Multimodal understanding from a single screenshot 🧠 Strong reasoning around UX and product design decisions 💻 Able to bridge visual inputs and code generation ⚡ OpenAI-compatible API that was easy to integrate 💰 Impressive capability-to-cost ratio Qwen3.7-Plus isn't just a chatbot. It's a multimodal agent model designed for workflows that combine vision, reasoning, coding, and automation. Try Qwen3.7-Plus with limited time 20% off:

Most AI demos stop at text prompts. I wanted to see what Qwen3.7-Plus could do beyond chat, so I tested it through the API using an Airbnb interface screenshot. The model analyzed the UI, identified key components, explained the visual hierarchy, highlighted usability strengths, suggested UX improvements, and even generated a React + Tailwind structure to recreate the interface. A few things that stood out: 👁️ Multimodal understanding from a single screenshot 🧠 Strong reasoning around UX and product design decisions 💻 Able to bridge visual inputs and code generation ⚡ OpenAI-compatible API that was easy to integrate 💰 Impressive capability-to-cost ratio Qwen3.7-Plus isn't just a chatbot. It's a multimodal agent model designed for workflows that combine vision, reasoning, coding, and automation. Try Qwen3.7-Plus with limited time 20% off:

Kuria | AI

17,044 views • 1 month ago

In this demo, you’ll see Gemini 3 Flash’s frontier-level reasoning and multimodal capabilities on display. The model is able to simultaneously conduct complex geometric calculations while processing complex inputs (video and image). You can play around with the slingshot in Google AI Studio here and share your favorite examples below:

In this demo, you’ll see Gemini 3 Flash’s frontier-level reasoning and multimodal capabilities on display. The model is able to simultaneously conduct complex geometric calculations while processing complex inputs (video and image). You can play around with the slingshot in Google AI Studio here and share your favorite examples below:

Google AI

55,411 views • 6 months ago

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Tencent Hy

122,539 views • 10 months ago

Transformer Explainer Really cool interactive tool to learn about the inner workings of a Transformer model. Apparently, it runs a GPT-2 instance locally in the user's browser and allows you to experiment with your own inputs. This is a nice tool to learn more about the different components inside the Transformer and the transformations that occur. Tool:

Transformer Explainer Really cool interactive tool to learn about the inner workings of a Transformer model. Apparently, it runs a GPT-2 instance locally in the user's browser and allows you to experiment with your own inputs. This is a nice tool to learn more about the different components inside the Transformer and the transformations that occur. Tool:

elvis

121,921 views • 1 year ago

1/2 Meet Wan2.7-Video — The Comprehensive Model for Controllable Video Storytelling! From single clips to full-scale narrative direction, we’ve built more than just a generator. We’ve built a director’s suite: • Multimodal control over performance and style via text, image, audio, and video. • Character customization with up to 5 reference inputs and voice profiles. • Video editing with simple, intuitive instructions. • Full-stack creative toolkit: generation, editing, cloning, restyling, continuation, and more. • Sustained improvements in visual fidelity, motion stability, and prompt adherence.

1/2 Meet Wan2.7-Video — The Comprehensive Model for Controllable Video Storytelling! From single clips to full-scale narrative direction, we’ve built more than just a generator. We’ve built a director’s suite: • Multimodal control over performance and style via text, image, audio, and video. • Character customization with up to 5 reference inputs and voice profiles. • Video editing with simple, intuitive instructions. • Full-stack creative toolkit: generation, editing, cloning, restyling, continuation, and more. • Sustained improvements in visual fidelity, motion stability, and prompt adherence.

Wan

25,571,455 views • 3 months ago

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

🚀 New Paper: Pixel Reasoner 🧠🖼️ How can Vision-Language Models (VLMs) perform chain-of-thought reasoning within the image itself? We introduce Pixel Reasoner, the first open-source framework that enables VLMs to “think in pixel space” through curiosity-driven reinforcement learning. Current VLMs reason only in text — even when grounded in rich images or videos, their logical steps are verbalized in natural language. This restricts their ability to interrogate visual evidence and demonstrate how conclusions are drawn. 🔍 So we ask: What if we could make VLMs "show their work" by reasoning directly in the pixel space? Inspired by GPT-o3’s "think-in-image" ability, we propose a framework where VLMs use interactive visual operations — zoom, select-frame, highlight — to reason through complex visual inputs. To do this, we design a two-stage training process: Instruction tuning with synthesized visual reasoning traces. Reinforcement learning with curiosity-driven reward to balance exploration between pixel and text reasoning ✨ With this, Pixel Reasoner achieves near-SoTA performance on many information-rich multimodal benchmarks: 📊 84% on InfographicsVQA 🧠 84% on V* benchmark 🧩 74% on TallyQA-Complex It also achieves strong accuracy of 68% on MVBench (a video benchmark). Website: Paper: Code: Demo: (coming soon)

Wenhu Chen

82,829 views • 1 year ago

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Got the vibe but not the words? Starting today, you can simply show or tell AI Mode about what you're looking for and get rich, visual results to explore. ✨ This experience works by bringing together Search’s world-class visual understanding from Lens and Image search with Gemini 2.5’s advanced multimodal capabilities. When you show AI Mode what you’re searching for it uses our new “visual search fan-out” technique to get a deeper understanding of what’s in an image (including subtle details and secondary objects), and runs multiple queries in the background. This helps it understand the visual context and nuance of your request to deliver relevant visual responses. You can also tell AI Mode what vibe you’re going for (no image needed). For example, ask AI Mode for bedroom design inspo and go back and forth to refine your vision. We’re rolling out this experience in English in the U.S. starting this week. Try it out and let us know what you think!

Google AI

68,285 views • 9 months ago

meta muse spark 1.1 vs gpt 5.6 sol vs fable 5 vs grok 4.5 meta recently dropped muse spark 1.1 – a multimodal reasoning model from meta superintelligence labs built for agentic tasks. key facts: • 1m token context with active self-management – the model compacts its own history and keeps only the steps needed for later work • trained to orchestrate multi-agent systems: as main agent it plans and delegates to parallel subagents, as subagent it sticks to its job and knows when to escalate back • computer use trained to pick between scripting and clicking – writes automation when it's faster, clicks when it's simpler, batches actions per step • first public api from meta: the meta model api is now in preview • benchmarks: sweeps the agent column – mcp atlas 88.1 (opus 4.8: 82.2), jobbench 54.7 (opus: 48.4), humanity's last exam 62.1 (1st). loses coding – deepswe 1.1 53.3 vs gpt 5.5's 67.0, swe bench pro 61.5 vs opus's 69.2 our test – 3 prompts, single-file html, three.js, fully procedural, no assets: 1. norwegian house cantilevered over a fjord in a snowstorm – transmissive glass wall, fully modelled interior 2. beijing siheyuan courtyard house in dawn fog – instanced roof tiles, dougong brackets, glowing paper windows 3. new mexico adobe pueblo in an approaching dust storm – deep window reveals, windward grit accumulation we ran the test on AI/ML API platform results: - cost #1 muse spark 1.1 – $0.20 #2 grok 4.5 – $0.51 #3 gpt 5.6 sol – $1.93 #4 fable 5 – ~$5.20 - output tokens #1 muse spark 1.1 – 41,868 #2 gpt 5.6 sol – 49,139 #3 grok 4.5 – 64,954 #4 fable 5 – 81,849 - lines of code #1 muse spark 1.1 – 1,799 #2 gpt 5.6 sol – 2,377 #3 fable 5 – 3,088 #4 grok 4.5 – 4,216 observations: • muse spark is the cheapest of the four by a wide margin – 2.5x under grok, ~26x under fable per run. output quality tracks the price • only 7.4% of its output tokens are reasoning (3,104 of 41,868) – the model barely thinks before writing. economic, not pedantic: it commits to the first plan and ships it • the low loc is not compression, it's omission – all three prompts demanded instancing, muse spark delivered it in one muse spark's code quality – reviewed by fable 5: upsides: 1. all three files run 2. the adobe grit effect is legit – shader injection via onbeforecompile, windward faces detect storm direction through a normal-dot-wind term and darken procedurally 3. the fjord glass is real meshphysicalmaterial with transmission and ior, not a transparent quad 4. the siheyuan properly instances barrel tiles, dougong blocks and courtyard pavers downsides: 1. in the fjord file the strafe vector is negated – press a, you move right; press d, you move left. exactly the key mix-up we kept hitting with this model 2. all three files ship the model's self-doubt as comments: "// actually yaw orientation: need correct" sits above a direction vector that gets computed, abandoned and recomputed – dead vectors allocated every frame, 60 times a second 3. the siheyuan registers two separate keydown listeners, one containing an empty if-block 4. snow "accumulation" on the norway roof is a sine wobble on a scale value, not accumulation 5. "instanced snow" became 3,500 plain points. zero dispose calls anywhere pattern: minimal reasoning, minimal code, minimal price. it nails the flashy requirements – shaders, transmissive glass – and quietly drops the boring ones: instancing, controls, cleanup. you get a demo that mostly runs and a control scheme you can't trust follow thehype. for 24/7 ai news, analysis and breakdowns

meta muse spark 1.1 vs gpt 5.6 sol vs fable 5 vs grok 4.5 meta recently dropped muse spark 1.1 – a multimodal reasoning model from meta superintelligence labs built for agentic tasks. key facts: • 1m token context with active self-management – the model compacts its own history and keeps only the steps needed for later work • trained to orchestrate multi-agent systems: as main agent it plans and delegates to parallel subagents, as subagent it sticks to its job and knows when to escalate back • computer use trained to pick between scripting and clicking – writes automation when it's faster, clicks when it's simpler, batches actions per step • first public api from meta: the meta model api is now in preview • benchmarks: sweeps the agent column – mcp atlas 88.1 (opus 4.8: 82.2), jobbench 54.7 (opus: 48.4), humanity's last exam 62.1 (1st). loses coding – deepswe 1.1 53.3 vs gpt 5.5's 67.0, swe bench pro 61.5 vs opus's 69.2 our test – 3 prompts, single-file html, three.js, fully procedural, no assets: 1. norwegian house cantilevered over a fjord in a snowstorm – transmissive glass wall, fully modelled interior 2. beijing siheyuan courtyard house in dawn fog – instanced roof tiles, dougong brackets, glowing paper windows 3. new mexico adobe pueblo in an approaching dust storm – deep window reveals, windward grit accumulation we ran the test on AI/ML API platform results: - cost #1 muse spark 1.1 – $0.20 #2 grok 4.5 – $0.51 #3 gpt 5.6 sol – $1.93 #4 fable 5 – ~$5.20 - output tokens #1 muse spark 1.1 – 41,868 #2 gpt 5.6 sol – 49,139 #3 grok 4.5 – 64,954 #4 fable 5 – 81,849 - lines of code #1 muse spark 1.1 – 1,799 #2 gpt 5.6 sol – 2,377 #3 fable 5 – 3,088 #4 grok 4.5 – 4,216 observations: • muse spark is the cheapest of the four by a wide margin – 2.5x under grok, ~26x under fable per run. output quality tracks the price • only 7.4% of its output tokens are reasoning (3,104 of 41,868) – the model barely thinks before writing. economic, not pedantic: it commits to the first plan and ships it • the low loc is not compression, it's omission – all three prompts demanded instancing, muse spark delivered it in one muse spark's code quality – reviewed by fable 5: upsides: 1. all three files run 2. the adobe grit effect is legit – shader injection via onbeforecompile, windward faces detect storm direction through a normal-dot-wind term and darken procedurally 3. the fjord glass is real meshphysicalmaterial with transmission and ior, not a transparent quad 4. the siheyuan properly instances barrel tiles, dougong blocks and courtyard pavers downsides: 1. in the fjord file the strafe vector is negated – press a, you move right; press d, you move left. exactly the key mix-up we kept hitting with this model 2. all three files ship the model's self-doubt as comments: "// actually yaw orientation: need correct" sits above a direction vector that gets computed, abandoned and recomputed – dead vectors allocated every frame, 60 times a second 3. the siheyuan registers two separate keydown listeners, one containing an empty if-block 4. snow "accumulation" on the norway roof is a sine wobble on a scale value, not accumulation 5. "instanced snow" became 3,500 plain points. zero dispose calls anywhere pattern: minimal reasoning, minimal code, minimal price. it nails the flashy requirements – shaders, transmissive glass – and quietly drops the boring ones: instancing, controls, cleanup. you get a demo that mostly runs and a control scheme you can't trust follow thehype. for 24/7 ai news, analysis and breakdowns

thehype.

130,517 views • 4 days ago

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from video - across massive libraries and live streams alike. We're calling this video intelligence, and we're actively looking to back founders building here. We're most excited about companies pushing on the core capabilities: - Video-native models - multimodal embeddings, temporal reasoning, and retrieval built specifically for video rather than adapted from image or text - Real-time and large-scale pipelines - infrastructure for processing, indexing, and querying video at the speed and scale enterprises actually need - Agentic and reasoning layers - systems that don't just retrieve clips but answer questions, surface anomalies, and take action on what they see The models and infrastructure to make this real are appearing to be crossing a capability threshold right now. Multimodal foundation models are maturing, storage costs have collapsed, and enterprises are sitting on years of unstructured video with no way to use it. That infrastructure unlocks a wide range of applications including media and sports workflows, security and physical operations, enterprise knowledge management, advertising analytics, robotics, and consumer products, where video has historically been dark data. If you're building in video intelligence at the model layer, the platform layer, or in a vertical application, we'd love to talk!

AI has transformed how video is created. We think the next wave is about understanding it. Over the past few years, we've seen remarkable advances in video generation, editing, avatars, and creative tooling. An increasingly important problem is teaching machines to search, analyze, reason over, and extract insight from video - across massive libraries and live streams alike. We're calling this video intelligence, and we're actively looking to back founders building here. We're most excited about companies pushing on the core capabilities: - Video-native models - multimodal embeddings, temporal reasoning, and retrieval built specifically for video rather than adapted from image or text - Real-time and large-scale pipelines - infrastructure for processing, indexing, and querying video at the speed and scale enterprises actually need - Agentic and reasoning layers - systems that don't just retrieve clips but answer questions, surface anomalies, and take action on what they see The models and infrastructure to make this real are appearing to be crossing a capability threshold right now. Multimodal foundation models are maturing, storage costs have collapsed, and enterprises are sitting on years of unstructured video with no way to use it. That infrastructure unlocks a wide range of applications including media and sports workflows, security and physical operations, enterprise knowledge management, advertising analytics, robotics, and consumer products, where video has historically been dark data. If you're building in video intelligence at the model layer, the platform layer, or in a vertical application, we'd love to talk!

Jason Cui

36,205 views • 2 months ago

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Anthropic just announced Computer Use It allows Claude to control your computer screen based on a prompt and take actions on your behalf The use cases in agentic coding with automated debugging, customer support, and education are going to be INSANE

Rowan Cheung

1,024,377 views • 1 year ago

Kling O1 is GOATED! No Cap! Kling AI -Replace characters with a single reference image or element -Change styles with a video reference and prompt -Add visual FX with ease -Change environments -Quality coherency and outputs from both text and image to video. -So much more! Try it now on OpenArt (Link in the comments! 👇)

Kling O1 is GOATED! No Cap! Kling AI -Replace characters with a single reference image or element -Change styles with a video reference and prompt -Add visual FX with ease -Change environments -Quality coherency and outputs from both text and image to video. -So much more! Try it now on OpenArt (Link in the comments! 👇)

OpenArt

31,914 views • 7 months ago

Grok 4.5 and Muse Spark 1.1 are now available on the AI/ML platform for testing. What a week; all 3 models in the comparison dropped just recently. There will be a lot to experiment with. AI/ML has tested the new models on prompts for playable mini-games, including a Fruit Ninja-style slicer and a Crossy Road clone. All three are live in the Playground and via API for side-by-side runs on output and cost.

Grok 4.5 and Muse Spark 1.1 are now available on the AI/ML platform for testing. What a week; all 3 models in the comparison dropped just recently. There will be a lot to experiment with. AI/ML has tested the new models on prompts for playable mini-games, including a Fruit Ninja-style slicer and a Crossy Road clone. All three are live in the Playground and via API for side-by-side runs on output and cost.

🚨 AI News | TestingCatalog

35,532 views • 5 days ago

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Our first short course with Anthropic! Building Towards Computer Use with Anthropic. This teaches you to build an LLM-based agent that uses a computer interface by generating mouse clicks and keystrokes. Computer Use is an important, emerging capability for LLMs that will let AI agents do many more tasks than were possible before, since it lets them interact with interfaces designed for humans to use, rather than only tools that provide explicit API access. I hope you will enjoy learning about it! This course is taught by Anthropic's Head of Curriculum, Colt_Steele. You'll learn to apply image reasoning and tool use to "use" a computer as follows: a model processes an image of the screen, analyzes it to understand what's going on, and navigates the computer via mouse clicks and keystrokes. This course goes through the key building blocks, and culminates in a demo of an AI assistant that uses a web browser to search for a research paper, downloads the PDF, and finally summarizes the paper for you. In detail, you’ll: - Learn about Anthropic's family of models, when to use which one, and make API requests to Claude - Use multi-modal prompts that combine text and image content blocks, and also work with streaming responses - Improve your prompting by using prompt templates, using XML to structure prompts, and providing examples - Implement prompt caching to reduce cost and latency - Apply tool-use to build a chatbot that can call different tools to respond to queries - See all these building blocks come together in Computer Use demo Please sign up here:

Andrew Ng

170,366 views • 1 year ago

Execution is where AI becomes most useful. With OptimAI Claw, tasks don’t stop at answers; they carry through into real actions across the web. 🦞 Another showcase: “Hey OptimAI, open YouTube, find a trending Korean music video, analyze and if it meets high engagement criteria, like the video and leave a relevant comment on my behalf.” OptimAI Claw navigates, evaluates, and executes the action locally on your device. This isn’t about automation scripts. It’s about agents that can interpret intent and carry it through across real environments. As these capabilities expand, routine digital interactions start to disappear; and execution becomes ambient.

Execution is where AI becomes most useful. With OptimAI Claw, tasks don’t stop at answers; they carry through into real actions across the web. 🦞 Another showcase: “Hey OptimAI, open YouTube, find a trending Korean music video, analyze and if it meets high engagement criteria, like the video and leave a relevant comment on my behalf.” OptimAI Claw navigates, evaluates, and executes the action locally on your device. This isn’t about automation scripts. It’s about agents that can interpret intent and carry it through across real environments. As these capabilities expand, routine digital interactions start to disappear; and execution becomes ambient.

OptimAI Network

14,713 views • 3 months ago

I've made a Kubernetes that runs in the browser, and today we're open sourcing it. The hope here is to make super interactive and visual web content about Kubernetes much easier to write, and we have a whole bunch planned this year. Links below.

I've made a Kubernetes that runs in the browser, and today we're open sourcing it. The hope here is to make super interactive and visual web content about Kubernetes much easier to write, and we have a whole bunch planned this year. Links below.

Sam Rose

51,626 views • 26 days ago