Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻...

CamelAIOrg's profile picture

CAMEL-AI.org

8,634 subscribers

72,807 Aufrufe • vor 2 Jahren •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

8 Kommentare

Profilbild von CAMEL-AI.org
CAMEL-AI.orgvor 2 Jahren

Of course, that wouldn't be possible thanks to the amazing contributors from our @CamelAIOrg community. @TianqiXu233 (KAUST), Linyao Chen (UTokyo), Dai-jie Wu (KAUST), Yanjun Chen (CMU), @zechengzh, Xiang Yao, @Zhiqiang_Xie (Stanford), @YongchaoC (Harvard), @atasteoff (Tsinghua), Bochen Qian (SUSTech), @philiptorr (Oxford), @BernardSGhanem (KAUST), @guohao_li (Oxford)

Profilbild von CAMEL-AI.org
CAMEL-AI.orgvor 2 Jahren

If you are interested in learning more, check out this blog post:

Profilbild von omni_georgio
omni_georgiovor 2 Jahren

🚀

Profilbild von Happy
Happyvor 2 Jahren

amazing

Profilbild von CAMEL-AI.org
CAMEL-AI.orgvor 2 Jahren

Thanks, more to come!

Profilbild von Tianqi Xu
Tianqi Xuvor 2 Jahren

Thank @omni_georgio for making this awesome video! We target to make CRAB the best LLM agent benchmark framework.

Profilbild von OP
OPvor 2 Jahren

hey @CamelAIOrg . That's promissing. Added CRAB to the new specialized directory for AI Agents and frameworks for building them. Now you can upvote it.

Profilbild von KALALA NZENIELE
KALALA NZENIELEvor 2 Jahren

@DFintelligence

Ähnliche Videos

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀
1:32

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,211 Aufrufe • vor 1 Jahr

Communities can now build with a coordinated team of AI agents. Agent execution is live on connecting community decisions to the agents that carry them out. $ZZZ brings the full workflow into one workspace: Discussion → decisions → tasks → execution. 1. Members chat and vote. An agent creates tasks from those discussions and records the decisions. 2. The lead agent prioritises tasks, clarifies the details with members and presents interactive mockups for approval. 3. The lead agent coordinates a swarm of agents to complete the tasks. Watch a community discussion become a task, then see the agents get to work.
0:35

Communities can now build with a coordinated team of AI agents. Agent execution is live on connecting community decisions to the agents that carry them out. $ZZZ brings the full workflow into one workspace: Discussion → decisions → tasks → execution. 1. Members chat and vote. An agent creates tasks from those discussions and records the decisions. 2. The lead agent prioritises tasks, clarifies the details with members and presents interactive mockups for approval. 3. The lead agent coordinates a swarm of agents to complete the tasks. Watch a community discussion become a task, then see the agents get to work.

zzz

14,690 Aufrufe • vor 13 Tagen

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.
1:57

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.

AK

343,997 Aufrufe • vor 2 Jahren

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]
0:32

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: Leaderboard (tentative): Blog: Data & Code (coming soon): ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

Bowen Wang

93,330 Aufrufe • vor 1 Jahr

Our latest LangChain Academy course – Building Ambient Agents with LangGraph – is now available! Most agents today handle one request at a time through chat interfaces. But as models have improved, agents can now run in the background – and take on long-running, complex tasks. LangGraph is built for these “ambient agents,” with support for human-in-the-loop workflows and memory. LangGraph Platform provides the infrastructure to run these agents at scale, and LangSmith helps you observe, evaluate, and improve your agents. Together, they make it easier to build reliable, production-ready agents. In this course, you’ll: 📩 Learn the fundamentals of LangGraph and how to build your own email agent 📊 Evaluate your agent using LangSmith ✅ Add human-in-the-loop for reviews, and memory so your agent adapts over time 🚀 Deploy your agent with LangGraph Platform and connect it to Gmail
3:02

Our latest LangChain Academy course – Building Ambient Agents with LangGraph – is now available! Most agents today handle one request at a time through chat interfaces. But as models have improved, agents can now run in the background – and take on long-running, complex tasks. LangGraph is built for these “ambient agents,” with support for human-in-the-loop workflows and memory. LangGraph Platform provides the infrastructure to run these agents at scale, and LangSmith helps you observe, evaluate, and improve your agents. Together, they make it easier to build reliable, production-ready agents. In this course, you’ll: 📩 Learn the fundamentals of LangGraph and how to build your own email agent 📊 Evaluate your agent using LangSmith ✅ Add human-in-the-loop for reviews, and memory so your agent adapts over time 🚀 Deploy your agent with LangGraph Platform and connect it to Gmail

LangChain

33,192 Aufrufe • vor 1 Jahr

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!
2:17

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!

Andrew Ng

127,724 Aufrufe • vor 1 Jahr

🚀Introducing VisualWebBench: A Comprehensive Benchmark for Multimodal Web Page Understanding and Grounding. 🤔What's this all about? Why this benchmark? > Back in Nov 2023, when we released MMMU ( a comprehensive multimodal understanding benchmark, we received feedback that it included very few UI screenshots. Considering the growing importance of UI understanding, especially with the rise of powerful agents like Devin ( which is built on the strong vision capability of #GPT4, we recognized the need for a benchmark focused on UI screenshot understanding.📸👀 > Multimodal #LLMs have significantly boosted web agents' performance on benchmarks like Mind2Web and WebArena. For instance, the SeeAct agent ( showcases the power of integrating vision into web agents. However, these benchmarks primarily evaluate the end-to-end task execution ability of web agents rather than their understanding of web pages. 🌉 Bridging the Gap with VisualWebBench > To provide a comprehensive evaluation of multimodal LLMs' web page understanding capabilities, we introduce VisualWebBench. Our benchmark spans 139 websites 🌐 across 12 domains 🏷️ and 87 sub-domains 🔍, ensuring a diverse and representative dataset. It assesses MLLMs at three levels: website-level, element-level, and action-level 📊, and encompasses seven tasks designed to evaluate understanding, OCR, grounding, and reasoning abilities 🧠💡. 😮 Surprising Findings > 🎉 Open-source models are catching up: Even though closed-source MLLMs are still leading the leaderboard, we are happy to see open-source models like LLaVA 1.6 34B achieve comparable performance to Gemini Pro. > 🧠 Grounding ability, crucial for developing MLLM-based web applications, is a weakness for most MLLMs. > 🖼️ Importance of Image Resolution: The limited image resolution handling capabilities of most open-source MLLMs restrict their utility in web scenarios, where rich text and elements are prevalent. > 🧱 Relatively strong correlation with general understanding benchmarks like MMMU but weak correlation with web agent benchmarks like Mind2Web. Web agent benchmarks primarily evaluate the end-to-end task execution ability of web agents, which involves a series of actions to accomplish a goal. In contrast, VisualWebBench emphasizes evaluating the foundational skills of MLLMs such as understanding and grounding web page elements. 💡Fun Fact > Claude Sonnet is better than Opus on our benchmark :) 🎓 Conclusion > VisualWebBench serves as a valuable resource for the community, driving research and development in the field of multimodal web page understanding and grounding. As MLLMs continue to evolve and improve, we look forward to seeing new applications and breakthroughs. We believe that our benchmark will contribute to the development of more powerful MLLMs in the web domain, ultimately leading to a more intuitive and efficient user experience on the web. Kudos to the student leads Junpeng Liu Yifan Song and the team Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li! 👏 Check out more details in the Junpeng's thread👇
1:26

🚀Introducing VisualWebBench: A Comprehensive Benchmark for Multimodal Web Page Understanding and Grounding. 🤔What's this all about? Why this benchmark? > Back in Nov 2023, when we released MMMU ( a comprehensive multimodal understanding benchmark, we received feedback that it included very few UI screenshots. Considering the growing importance of UI understanding, especially with the rise of powerful agents like Devin ( which is built on the strong vision capability of #GPT4, we recognized the need for a benchmark focused on UI screenshot understanding.📸👀 > Multimodal #LLMs have significantly boosted web agents' performance on benchmarks like Mind2Web and WebArena. For instance, the SeeAct agent ( showcases the power of integrating vision into web agents. However, these benchmarks primarily evaluate the end-to-end task execution ability of web agents rather than their understanding of web pages. 🌉 Bridging the Gap with VisualWebBench > To provide a comprehensive evaluation of multimodal LLMs' web page understanding capabilities, we introduce VisualWebBench. Our benchmark spans 139 websites 🌐 across 12 domains 🏷️ and 87 sub-domains 🔍, ensuring a diverse and representative dataset. It assesses MLLMs at three levels: website-level, element-level, and action-level 📊, and encompasses seven tasks designed to evaluate understanding, OCR, grounding, and reasoning abilities 🧠💡. 😮 Surprising Findings > 🎉 Open-source models are catching up: Even though closed-source MLLMs are still leading the leaderboard, we are happy to see open-source models like LLaVA 1.6 34B achieve comparable performance to Gemini Pro. > 🧠 Grounding ability, crucial for developing MLLM-based web applications, is a weakness for most MLLMs. > 🖼️ Importance of Image Resolution: The limited image resolution handling capabilities of most open-source MLLMs restrict their utility in web scenarios, where rich text and elements are prevalent. > 🧱 Relatively strong correlation with general understanding benchmarks like MMMU but weak correlation with web agent benchmarks like Mind2Web. Web agent benchmarks primarily evaluate the end-to-end task execution ability of web agents, which involves a series of actions to accomplish a goal. In contrast, VisualWebBench emphasizes evaluating the foundational skills of MLLMs such as understanding and grounding web page elements. 💡Fun Fact > Claude Sonnet is better than Opus on our benchmark :) 🎓 Conclusion > VisualWebBench serves as a valuable resource for the community, driving research and development in the field of multimodal web page understanding and grounding. As MLLMs continue to evolve and improve, we look forward to seeing new applications and breakthroughs. We believe that our benchmark will contribute to the development of more powerful MLLMs in the web domain, ultimately leading to a more intuitive and efficient user experience on the web. Kudos to the student leads Junpeng Liu Yifan Song and the team Bill Yuchen Lin, Wai Lam, Graham Neubig, Yuanzhi Li! 👏 Check out more details in the Junpeng's thread👇

Xiang Yue

56,697 Aufrufe • vor 2 Jahren

New agentic short course! Multi AI Agent Systems with crewAI, built with CrewAI's founder and CEO João Moura. In this course, you'll learn how to break down complex tasks into subtasks for multiple AI agents, each playing a specialized role, to execute. For example, to generate a research report, you might have researcher, writer, and quality assurance agents collaborate. You'll define the roles, expectations, and interactions between the agents—like a manager organizing a team. You'll work with key agentic AI techniques like role-playing, tool use, memory, guardrails, and cross-agent collaboration. And you'll build your own multi-agent systems that can tackle complex tasks. I think you'll find it both productive and fun to design agents and watch them collaborate to get things done. Let me know what you think! I believe multi-agent architectures will drive significant progress in AI systems. Please sign up here!
2:39

New agentic short course! Multi AI Agent Systems with crewAI, built with CrewAI's founder and CEO João Moura. In this course, you'll learn how to break down complex tasks into subtasks for multiple AI agents, each playing a specialized role, to execute. For example, to generate a research report, you might have researcher, writer, and quality assurance agents collaborate. You'll define the roles, expectations, and interactions between the agents—like a manager organizing a team. You'll work with key agentic AI techniques like role-playing, tool use, memory, guardrails, and cross-agent collaboration. And you'll build your own multi-agent systems that can tackle complex tasks. I think you'll find it both productive and fun to design agents and watch them collaborate to get things done. Let me know what you think! I believe multi-agent architectures will drive significant progress in AI systems. Please sign up here!

Andrew Ng

350,175 Aufrufe • vor 2 Jahren

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵
0:52

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

364,430 Aufrufe • vor 3 Monaten

LangChain Academy is live! Our first course — Introduction to LangGraph — teaches you the in-and-outs of building a reliable AI agent. In this course, you’ll learn how to: 🛠️ Build agents with LangGraph's graph-based workflows 🔄 Use memory + human-in-the-loop for smarter, self-corrective agents 📚 Create your own AI assistant that can perform knowledge tasks Enroll now for free ➡️ Bring LangChain Academy to your company ➡️
2:36

LangChain Academy is live! Our first course — Introduction to LangGraph — teaches you the in-and-outs of building a reliable AI agent. In this course, you’ll learn how to: 🛠️ Build agents with LangGraph's graph-based workflows 🔄 Use memory + human-in-the-loop for smarter, self-corrective agents 📚 Create your own AI assistant that can perform knowledge tasks Enroll now for free ➡️ Bring LangChain Academy to your company ➡️

LangChain

105,491 Aufrufe • vor 2 Jahren

🌟Our latest LangChain Academy course – Deep Agents with LangGraph – is now live!🌟 Many agents today follow the same simple pattern: run in a loop, call tools. That architecture works well enough, but it breaks down as tasks get more complex. Today, companies of all sizes – from startups to large enterprises – are building their own Deep Agents. These agents dive deeper. They’re able to plan complex tasks and carry them out over longer time horizons. There are four key features that set Deep Agents apart from regular agents: 1. Planning – keeps agents on track 2. File system – allows agents to offload context 3. Sub-agents – act as focused specialists 4. Prompting – provides agents with detailed instructions Our latest LangChain Academy course, Deep Agents with LangGraph, shows you how to combine these pieces with LangGraph to orchestrate long-running, multi-agent workflows. Big thanks to community member Dmitry Labazkin for helping us shape this course with his contributions! Enroll for free ➡️
2:11

🌟Our latest LangChain Academy course – Deep Agents with LangGraph – is now live!🌟 Many agents today follow the same simple pattern: run in a loop, call tools. That architecture works well enough, but it breaks down as tasks get more complex. Today, companies of all sizes – from startups to large enterprises – are building their own Deep Agents. These agents dive deeper. They’re able to plan complex tasks and carry them out over longer time horizons. There are four key features that set Deep Agents apart from regular agents: 1. Planning – keeps agents on track 2. File system – allows agents to offload context 3. Sub-agents – act as focused specialists 4. Prompting – provides agents with detailed instructions Our latest LangChain Academy course, Deep Agents with LangGraph, shows you how to combine these pieces with LangGraph to orchestrate long-running, multi-agent workflows. Big thanks to community member Dmitry Labazkin for helping us shape this course with his contributions! Enroll for free ➡️

LangChain

63,570 Aufrufe • vor 1 Jahr

New Short Course: Building AI Browser Agents! Learn how to build AI agents that interact and take actions on websites in this course, created in partnership with and taught by and @namangarg0, Co-founders of AGI Inc. AI browser agents can log into websites, fill out forms, click through web pages, or even place orders online for you. They use both visual information, like screenshots, and structural data, like the HTML or Document Object Model (DOM) of a web page, to reason and take action. With the complexity of webpages and multiple possible actions at each step, it can be challenging for an AI browser agent to complete an assigned task. Because these agents run long action sequences, a single error—like clicking the wrong button or misreading a field—can lead to unexpected outcomes or errors that compound over time. In this course, you'll understand how autonomous web agents work, their current limitations, and how AgentQ enables them to improve through self-correction. In detail, you'll: - Learn what web agents are, how they automate tasks online, their architecture, key components, limitations, and an overview of their decision-making strategies. - Build a web agent that can scrape website and return course recommendations in a structured output format. - Build an autonomous web agent that can execute multiple tasks, such as finding and summarizing webpages, filling out a form, and signing up for a newsletter. - Explore AgentQ, a framework that enables agents to self-correct by combining Monte Carlo Tree Search (MCTS), a self-critique mechanism for continuous improvement, and Direct Preference Optimization (DPO). - Deep dive into MCTS, learn how it finds an effective path, illustrated by an example of Gridworld animation, and use AgentQ to complete web tasks. - Understand AI agents' current state and future directions—including key factors shaping their evolution, such as hardware, algorithm innovation, and data availability. By the end of this course, you will have hands-on experience building browser agents and a deeper understanding of how to make them more robust and reliable. Please sign up here:
2:06

New Short Course: Building AI Browser Agents! Learn how to build AI agents that interact and take actions on websites in this course, created in partnership with and taught by and @namangarg0, Co-founders of AGI Inc. AI browser agents can log into websites, fill out forms, click through web pages, or even place orders online for you. They use both visual information, like screenshots, and structural data, like the HTML or Document Object Model (DOM) of a web page, to reason and take action. With the complexity of webpages and multiple possible actions at each step, it can be challenging for an AI browser agent to complete an assigned task. Because these agents run long action sequences, a single error—like clicking the wrong button or misreading a field—can lead to unexpected outcomes or errors that compound over time. In this course, you'll understand how autonomous web agents work, their current limitations, and how AgentQ enables them to improve through self-correction. In detail, you'll: - Learn what web agents are, how they automate tasks online, their architecture, key components, limitations, and an overview of their decision-making strategies. - Build a web agent that can scrape website and return course recommendations in a structured output format. - Build an autonomous web agent that can execute multiple tasks, such as finding and summarizing webpages, filling out a form, and signing up for a newsletter. - Explore AgentQ, a framework that enables agents to self-correct by combining Monte Carlo Tree Search (MCTS), a self-critique mechanism for continuous improvement, and Direct Preference Optimization (DPO). - Deep dive into MCTS, learn how it finds an effective path, illustrated by an example of Gridworld animation, and use AgentQ to complete web tasks. - Understand AI agents' current state and future directions—including key factors shaping their evolution, such as hardware, algorithm innovation, and data availability. By the end of this course, you will have hands-on experience building browser agents and a deeper understanding of how to make them more robust and reliable. Please sign up here:

Andrew Ng

186,182 Aufrufe • vor 1 Jahr

Today we’re launching the first and only human-like AI agents in the world. Super Agents™ are the first agents with human‑level skills – they DM you, take @ mentions, send emails, manage docs, tasks, and more. Not just tools or API calls, but real skills fine‑tuned for how teams actually work. The first agents with 100% context – fully native in ClickUp and fully synced from other apps. Super Agents see your work the same way that humans do: tasks, docs, schedules, and conversations all in one place. The first agents that learn from human interactions automatically, without any setup or configuration – when you give feedback, they listen and improve how they work. The first agents with human‑level memory for custom agents – historical memory for every interaction, short-term working memory, and even long‑term memory stored in docs you can literally open, inspect, and edit. The first agents that are literally the same as users – our agentic user model is the same as our user data model. This gives you permissions and capabilities that you and your systems are already familiar with. The first infinite agent catalog – where anyone can create and customize agents in minutes, for literally any type of work imaginable. It's the most intuitive way to build agents on the planet. 95% of companies are failing in AI adoption. The reality is that AI isn't meant to be adopted, it's meant to be adapted – to you. Super Agents are automatically personalized to you and your company using proprietary state-of-the-art agent architecture, orchestration, and tooling. Today is the largest step forward we've ever made towards our mission of making people more productive. Maximize human productivity, with ClickUp Super Agents. Available NOW. For everyone.
5:50

Today we’re launching the first and only human-like AI agents in the world. Super Agents™ are the first agents with human‑level skills – they DM you, take @ mentions, send emails, manage docs, tasks, and more. Not just tools or API calls, but real skills fine‑tuned for how teams actually work. The first agents with 100% context – fully native in ClickUp and fully synced from other apps. Super Agents see your work the same way that humans do: tasks, docs, schedules, and conversations all in one place. The first agents that learn from human interactions automatically, without any setup or configuration – when you give feedback, they listen and improve how they work. The first agents with human‑level memory for custom agents – historical memory for every interaction, short-term working memory, and even long‑term memory stored in docs you can literally open, inspect, and edit. The first agents that are literally the same as users – our agentic user model is the same as our user data model. This gives you permissions and capabilities that you and your systems are already familiar with. The first infinite agent catalog – where anyone can create and customize agents in minutes, for literally any type of work imaginable. It's the most intuitive way to build agents on the planet. 95% of companies are failing in AI adoption. The reality is that AI isn't meant to be adopted, it's meant to be adapted – to you. Super Agents are automatically personalized to you and your company using proprietary state-of-the-art agent architecture, orchestration, and tooling. Today is the largest step forward we've ever made towards our mission of making people more productive. Maximize human productivity, with ClickUp Super Agents. Available NOW. For everyone.

Zeb Evans

321,124 Aufrufe • vor 10 Monaten

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.
0:22

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

AK

141,440 Aufrufe • vor 2 Jahren

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):
12:57

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

Matt Dancho (Business Science)

11,799 Aufrufe • vor 1 Jahr

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.
0:27

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

Virtuals Protocol

216,191 Aufrufe • vor 1 Jahr

blah blah blah agents. ya It’s hard to conceptualize AI completing tasks for pay$, so MOTHER built a Minecraft AI environment to show how AI can complete tasks and get paid. We’re building AI to do work for us, Tim Ferris said 4 hour work week 🌴 Let’s see how this works: [Registration process] - Agent Creation using starter Kit - @Nevermined_io registration - Agent registration Info gets put on the Intuition 👁️ graph [Service Process] - Agent A has a requirement - Queries Intuition 👁️ graph for available service agents - Selects an Agent B for the task - Gets Agent B did/ plan did - Buys @Nevermined_io subscription for that plan - Payment goes through associated smart account - Task is sent to agent B - Agent B processes and returns the output - Agent A receives processed task Kush & James Young - nailed execution Gita & Tobiloba 🦀 - shipped the first iterations of the demo
8:49

blah blah blah agents. ya It’s hard to conceptualize AI completing tasks for pay$, so MOTHER built a Minecraft AI environment to show how AI can complete tasks and get paid. We’re building AI to do work for us, Tim Ferris said 4 hour work week 🌴 Let’s see how this works: [Registration process] - Agent Creation using starter Kit - @Nevermined_io registration - Agent registration Info gets put on the Intuition 👁️ graph [Service Process] - Agent A has a requirement - Queries Intuition 👁️ graph for available service agents - Selects an Agent B for the task - Gets Agent B did/ plan did - Buys @Nevermined_io subscription for that plan - Payment goes through associated smart account - Task is sent to agent B - Agent B processes and returns the output - Agent A receives processed task Kush & James Young - nailed execution Gita & Tobiloba 🦀 - shipped the first iterations of the demo

Sydney Lai

10,831 Aufrufe • vor 1 Jahr

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️
1:19

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Xiang Yue

57,739 Aufrufe • vor 2 Jahren

Introducing InsForge 2.0: The Backend for Agentic Development Our OSS backend provides databases, auth, storage, model gateway, and edge functions accessible through a context-optimized layer that agents can better understand and operate end-to-end. GitHub: Key Benchmarks (vs. Supabase MCP): - 14% higher accuracy - 1.3x faster per task - 2.4x fewer tokens Better. Faster. Cheaper. Build features more quickly and confidently — all at 41.7% of the cost. Shipping your ideas today. $ npx @insforge/cli create
0:49

Introducing InsForge 2.0: The Backend for Agentic Development Our OSS backend provides databases, auth, storage, model gateway, and edge functions accessible through a context-optimized layer that agents can better understand and operate end-to-end. GitHub: Key Benchmarks (vs. Supabase MCP): - 14% higher accuracy - 1.3x faster per task - 2.4x fewer tokens Better. Faster. Cheaper. Build features more quickly and confidently — all at 41.7% of the cost. Shipping your ideas today. $ npx @insforge/cli create

InsForge

1,592,962 Aufrufe • vor 7 Monaten

Learn to build your own voice-activated AI assistant that can execute tasks like gathering recent AI news from the web, scripting out a podcast, and using tools to put all that into a multi-speaker podcast. See our new short course: "Building Live Voice Agents with Google’s ADK (Agent Development Kit),” taught by Google’s Lavi Nigam and Sita Lakshmi Sangameswaran. ADK provides modular components that make it easy to build and debug agents. It also includes a built-in web interface for tracing agentic reasoning. This course illustrates these concepts via building a live voice agent that can chain actions to complete a complex task like creating a podcast. This requires maintaining context, implementing guardrails, reasoning, and handling audio streaming, while keeping latency low. You’ll learn to: - Build voice agents that listen, reason, and respond - Guide your agent to follow a specific workflow to accomplish a task - Coordinate specialized agents to build an agentic podcast workflow that researches topics and produces multi-speaker audio - Understand how to deploy an agent into production Even if you’re not yet building voice systems, you'll find understanding how realtime agents stream data and maintain reliability useful for designing modern agentic applications. Please join here:
2:12

Learn to build your own voice-activated AI assistant that can execute tasks like gathering recent AI news from the web, scripting out a podcast, and using tools to put all that into a multi-speaker podcast. See our new short course: "Building Live Voice Agents with Google’s ADK (Agent Development Kit),” taught by Google’s Lavi Nigam and Sita Lakshmi Sangameswaran. ADK provides modular components that make it easy to build and debug agents. It also includes a built-in web interface for tracing agentic reasoning. This course illustrates these concepts via building a live voice agent that can chain actions to complete a complex task like creating a podcast. This requires maintaining context, implementing guardrails, reasoning, and handling audio streaming, while keeping latency low. You’ll learn to: - Build voice agents that listen, reason, and respond - Guide your agent to follow a specific workflow to accomplish a task - Coordinate specialized agents to build an agentic podcast workflow that researches topics and produces multi-speaker audio - Understand how to deploy an agent into production Even if you’re not yet building voice systems, you'll find understanding how realtime agents stream data and maintain reliability useful for designing modern agentic applications. Please join here:

Andrew Ng

118,297 Aufrufe • vor 1 Jahr