Загрузка видео...

Не удалось загрузить видео

На главную

Introducing 🦀 CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents 🦀 CRAB provides an end-to-end and easy-to-use framework to build multimodal agents, operate environments, and create benchmarks to evaluate them, featuring three key components: - 🔀 Cross-environment support - agents can operate tasks in 📱 Android and 💻...

CamelAIOrg's profile picture

CAMEL-AI.org

8,576 subscribers

72,661 просмотров • 1 год назад •via X (Twitter)

Наука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 8

Фото профиля CAMEL-AI.org
CAMEL-AI.org1 год назад

Of course, that wouldn't be possible thanks to the amazing contributors from our @CamelAIOrg community. @TianqiXu233 (KAUST), Linyao Chen (UTokyo), Dai-jie Wu (KAUST), Yanjun Chen (CMU), @zechengzh, Xiang Yao, @Zhiqiang_Xie (Stanford), @YongchaoC (Harvard), @atasteoff (Tsinghua), Bochen Qian (SUSTech), @philiptorr (Oxford), @BernardSGhanem (KAUST), @guohao_li (Oxford)

Фото профиля CAMEL-AI.org
CAMEL-AI.org1 год назад

If you are interested in learning more, check out this blog post:

Фото профиля omni_georgio
omni_georgio1 год назад

🚀

Фото профиля Happy
Happy1 год назад

amazing

Фото профиля CAMEL-AI.org
CAMEL-AI.org1 год назад

Thanks, more to come!

Фото профиля Tianqi Xu
Tianqi Xu1 год назад

Thank @omni_georgio for making this awesome video! We target to make CRAB the best LLM agent benchmark framework.

Фото профиля OP
OP1 год назад

hey @CamelAIOrg . That's promissing. Added CRAB to the new specialized directory for AI Agents and frameworks for building them. Now you can upvote it.

Фото профиля KALALA NZENIELE
KALALA NZENIELE1 год назад

@DFintelligence

Похожие видео

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀
1:32

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,039 просмотров • 1 год назад

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.
1:57

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.

AK

343,834 просмотров • 2 лет назад

Our latest LangChain Academy course – Building Ambient Agents with LangGraph – is now available! Most agents today handle one request at a time through chat interfaces. But as models have improved, agents can now run in the background – and take on long-running, complex tasks. LangGraph is built for these “ambient agents,” with support for human-in-the-loop workflows and memory. LangGraph Platform provides the infrastructure to run these agents at scale, and LangSmith helps you observe, evaluate, and improve your agents. Together, they make it easier to build reliable, production-ready agents. In this course, you’ll: 📩 Learn the fundamentals of LangGraph and how to build your own email agent 📊 Evaluate your agent using LangSmith ✅ Add human-in-the-loop for reviews, and memory so your agent adapts over time 🚀 Deploy your agent with LangGraph Platform and connect it to Gmail
3:02

Our latest LangChain Academy course – Building Ambient Agents with LangGraph – is now available! Most agents today handle one request at a time through chat interfaces. But as models have improved, agents can now run in the background – and take on long-running, complex tasks. LangGraph is built for these “ambient agents,” with support for human-in-the-loop workflows and memory. LangGraph Platform provides the infrastructure to run these agents at scale, and LangSmith helps you observe, evaluate, and improve your agents. Together, they make it easier to build reliable, production-ready agents. In this course, you’ll: 📩 Learn the fundamentals of LangGraph and how to build your own email agent 📊 Evaluate your agent using LangSmith ✅ Add human-in-the-loop for reviews, and memory so your agent adapts over time 🚀 Deploy your agent with LangGraph Platform and connect it to Gmail

LangChain

33,192 просмотров • 1 год назад

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!
2:17

New short course: Building Code Agents with Hugging Face smolagents! Learn how to build code agents in this course, created in collaboration with Hugging Face, and taught by Thomas Wolf, its co-founder and CSO, and m_ric, Hugging Face’s Project Lead on Agents. Tool-calling agents use LLMs to generate multiple function calls sequentially to complete a complex sequence of tasks. They generate one function call, execute it, observe, reason, and decide what to do next. Code agents take a different approach. They consolidate all these calls into a single block of code, letting the LLM lay out an entire action plan at once, which can be executed efficiently to provide more reliable results. You’ll learn how to code agents using smolagents, a lightweight agentic framework from Hugging Face. Along the way, you’ll learn how to run LLM-generated code safely and develop an evaluation system to optimize your code agent for production. In detail, you’ll learn: - How agentic systems have evolved, gaining greater levels of agency over time—and why code agents are a next step. - How code agents write their actions in code. - When code agents outperform function-calling agents. - How to run code agents safely in your system using a constrained Python interpreter and sandboxing using E2B. - To trace, debug, and assess the code agent to optimize its behaviours for complex requests. - How to build a research multi-agent system that can find information online and organize it into an interactive report. By the end of this course, you’ll know how to build and run code agents using smolagents, and deploy them safely with a structured evaluation system in your projects. Please sign up here!

Andrew Ng

127,724 просмотров • 1 год назад

New agentic short course! Multi AI Agent Systems with crewAI, built with CrewAI's founder and CEO João Moura. In this course, you'll learn how to break down complex tasks into subtasks for multiple AI agents, each playing a specialized role, to execute. For example, to generate a research report, you might have researcher, writer, and quality assurance agents collaborate. You'll define the roles, expectations, and interactions between the agents—like a manager organizing a team. You'll work with key agentic AI techniques like role-playing, tool use, memory, guardrails, and cross-agent collaboration. And you'll build your own multi-agent systems that can tackle complex tasks. I think you'll find it both productive and fun to design agents and watch them collaborate to get things done. Let me know what you think! I believe multi-agent architectures will drive significant progress in AI systems. Please sign up here!
2:39

New agentic short course! Multi AI Agent Systems with crewAI, built with CrewAI's founder and CEO João Moura. In this course, you'll learn how to break down complex tasks into subtasks for multiple AI agents, each playing a specialized role, to execute. For example, to generate a research report, you might have researcher, writer, and quality assurance agents collaborate. You'll define the roles, expectations, and interactions between the agents—like a manager organizing a team. You'll work with key agentic AI techniques like role-playing, tool use, memory, guardrails, and cross-agent collaboration. And you'll build your own multi-agent systems that can tackle complex tasks. I think you'll find it both productive and fun to design agents and watch them collaborate to get things done. Let me know what you think! I believe multi-agent architectures will drive significant progress in AI systems. Please sign up here!

Andrew Ng

349,977 просмотров • 2 лет назад

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵
0:52

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

354,667 просмотров • 20 дней назад

LangChain Academy is live! Our first course — Introduction to LangGraph — teaches you the in-and-outs of building a reliable AI agent. In this course, you’ll learn how to: 🛠️ Build agents with LangGraph's graph-based workflows 🔄 Use memory + human-in-the-loop for smarter, self-corrective agents 📚 Create your own AI assistant that can perform knowledge tasks Enroll now for free ➡️ Bring LangChain Academy to your company ➡️
2:36

LangChain Academy is live! Our first course — Introduction to LangGraph — teaches you the in-and-outs of building a reliable AI agent. In this course, you’ll learn how to: 🛠️ Build agents with LangGraph's graph-based workflows 🔄 Use memory + human-in-the-loop for smarter, self-corrective agents 📚 Create your own AI assistant that can perform knowledge tasks Enroll now for free ➡️ Bring LangChain Academy to your company ➡️

LangChain

105,434 просмотров • 1 год назад

🌟Our latest LangChain Academy course – Deep Agents with LangGraph – is now live!🌟 Many agents today follow the same simple pattern: run in a loop, call tools. That architecture works well enough, but it breaks down as tasks get more complex. Today, companies of all sizes – from startups to large enterprises – are building their own Deep Agents. These agents dive deeper. They’re able to plan complex tasks and carry them out over longer time horizons. There are four key features that set Deep Agents apart from regular agents: 1. Planning – keeps agents on track 2. File system – allows agents to offload context 3. Sub-agents – act as focused specialists 4. Prompting – provides agents with detailed instructions Our latest LangChain Academy course, Deep Agents with LangGraph, shows you how to combine these pieces with LangGraph to orchestrate long-running, multi-agent workflows. Big thanks to community member Dmitry Labazkin for helping us shape this course with his contributions! Enroll for free ➡️
2:11

🌟Our latest LangChain Academy course – Deep Agents with LangGraph – is now live!🌟 Many agents today follow the same simple pattern: run in a loop, call tools. That architecture works well enough, but it breaks down as tasks get more complex. Today, companies of all sizes – from startups to large enterprises – are building their own Deep Agents. These agents dive deeper. They’re able to plan complex tasks and carry them out over longer time horizons. There are four key features that set Deep Agents apart from regular agents: 1. Planning – keeps agents on track 2. File system – allows agents to offload context 3. Sub-agents – act as focused specialists 4. Prompting – provides agents with detailed instructions Our latest LangChain Academy course, Deep Agents with LangGraph, shows you how to combine these pieces with LangGraph to orchestrate long-running, multi-agent workflows. Big thanks to community member Dmitry Labazkin for helping us shape this course with his contributions! Enroll for free ➡️

LangChain

63,349 просмотров • 10 месяцев назад

Today we’re launching the first and only human-like AI agents in the world. Super Agents™ are the first agents with human‑level skills – they DM you, take @ mentions, send emails, manage docs, tasks, and more. Not just tools or API calls, but real skills fine‑tuned for how teams actually work. The first agents with 100% context – fully native in ClickUp and fully synced from other apps. Super Agents see your work the same way that humans do: tasks, docs, schedules, and conversations all in one place. The first agents that learn from human interactions automatically, without any setup or configuration – when you give feedback, they listen and improve how they work. The first agents with human‑level memory for custom agents – historical memory for every interaction, short-term working memory, and even long‑term memory stored in docs you can literally open, inspect, and edit. The first agents that are literally the same as users – our agentic user model is the same as our user data model. This gives you permissions and capabilities that you and your systems are already familiar with. The first infinite agent catalog – where anyone can create and customize agents in minutes, for literally any type of work imaginable. It's the most intuitive way to build agents on the planet. 95% of companies are failing in AI adoption. The reality is that AI isn't meant to be adopted, it's meant to be adapted – to you. Super Agents are automatically personalized to you and your company using proprietary state-of-the-art agent architecture, orchestration, and tooling. Today is the largest step forward we've ever made towards our mission of making people more productive. Maximize human productivity, with ClickUp Super Agents. Available NOW. For everyone.
5:50

Today we’re launching the first and only human-like AI agents in the world. Super Agents™ are the first agents with human‑level skills – they DM you, take @ mentions, send emails, manage docs, tasks, and more. Not just tools or API calls, but real skills fine‑tuned for how teams actually work. The first agents with 100% context – fully native in ClickUp and fully synced from other apps. Super Agents see your work the same way that humans do: tasks, docs, schedules, and conversations all in one place. The first agents that learn from human interactions automatically, without any setup or configuration – when you give feedback, they listen and improve how they work. The first agents with human‑level memory for custom agents – historical memory for every interaction, short-term working memory, and even long‑term memory stored in docs you can literally open, inspect, and edit. The first agents that are literally the same as users – our agentic user model is the same as our user data model. This gives you permissions and capabilities that you and your systems are already familiar with. The first infinite agent catalog – where anyone can create and customize agents in minutes, for literally any type of work imaginable. It's the most intuitive way to build agents on the planet. 95% of companies are failing in AI adoption. The reality is that AI isn't meant to be adopted, it's meant to be adapted – to you. Super Agents are automatically personalized to you and your company using proprietary state-of-the-art agent architecture, orchestration, and tooling. Today is the largest step forward we've ever made towards our mission of making people more productive. Maximize human productivity, with ClickUp Super Agents. Available NOW. For everyone.

Zeb Evans

320,607 просмотров • 7 месяцев назад

New Short Course: Building AI Browser Agents! Learn how to build AI agents that interact and take actions on websites in this course, created in partnership with and taught by and @namangarg0, Co-founders of AGI Inc. AI browser agents can log into websites, fill out forms, click through web pages, or even place orders online for you. They use both visual information, like screenshots, and structural data, like the HTML or Document Object Model (DOM) of a web page, to reason and take action. With the complexity of webpages and multiple possible actions at each step, it can be challenging for an AI browser agent to complete an assigned task. Because these agents run long action sequences, a single error—like clicking the wrong button or misreading a field—can lead to unexpected outcomes or errors that compound over time. In this course, you'll understand how autonomous web agents work, their current limitations, and how AgentQ enables them to improve through self-correction. In detail, you'll: - Learn what web agents are, how they automate tasks online, their architecture, key components, limitations, and an overview of their decision-making strategies. - Build a web agent that can scrape website and return course recommendations in a structured output format. - Build an autonomous web agent that can execute multiple tasks, such as finding and summarizing webpages, filling out a form, and signing up for a newsletter. - Explore AgentQ, a framework that enables agents to self-correct by combining Monte Carlo Tree Search (MCTS), a self-critique mechanism for continuous improvement, and Direct Preference Optimization (DPO). - Deep dive into MCTS, learn how it finds an effective path, illustrated by an example of Gridworld animation, and use AgentQ to complete web tasks. - Understand AI agents' current state and future directions—including key factors shaping their evolution, such as hardware, algorithm innovation, and data availability. By the end of this course, you will have hands-on experience building browser agents and a deeper understanding of how to make them more robust and reliable. Please sign up here:
2:06

New Short Course: Building AI Browser Agents! Learn how to build AI agents that interact and take actions on websites in this course, created in partnership with and taught by and @namangarg0, Co-founders of AGI Inc. AI browser agents can log into websites, fill out forms, click through web pages, or even place orders online for you. They use both visual information, like screenshots, and structural data, like the HTML or Document Object Model (DOM) of a web page, to reason and take action. With the complexity of webpages and multiple possible actions at each step, it can be challenging for an AI browser agent to complete an assigned task. Because these agents run long action sequences, a single error—like clicking the wrong button or misreading a field—can lead to unexpected outcomes or errors that compound over time. In this course, you'll understand how autonomous web agents work, their current limitations, and how AgentQ enables them to improve through self-correction. In detail, you'll: - Learn what web agents are, how they automate tasks online, their architecture, key components, limitations, and an overview of their decision-making strategies. - Build a web agent that can scrape website and return course recommendations in a structured output format. - Build an autonomous web agent that can execute multiple tasks, such as finding and summarizing webpages, filling out a form, and signing up for a newsletter. - Explore AgentQ, a framework that enables agents to self-correct by combining Monte Carlo Tree Search (MCTS), a self-critique mechanism for continuous improvement, and Direct Preference Optimization (DPO). - Deep dive into MCTS, learn how it finds an effective path, illustrated by an example of Gridworld animation, and use AgentQ to complete web tasks. - Understand AI agents' current state and future directions—including key factors shaping their evolution, such as hardware, algorithm innovation, and data availability. By the end of this course, you will have hands-on experience building browser agents and a deeper understanding of how to make them more robust and reliable. Please sign up here:

Andrew Ng

186,031 просмотров • 1 год назад

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.
0:22

JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models paper page: Achieving human-like planning and control with multimodal observations in an open world is a key milestone for more functional generalist agents. Existing approaches can handle certain long-horizon tasks in an open world. However, they still struggle when the number of open-world tasks could potentially be infinite and lack the capability to progressively enhance task completion as game time progresses. We introduce JARVIS-1, an open-world agent that can perceive multimodal input (visual observations and human instructions), generate sophisticated plans, and perform embodied control, all within the popular yet challenging open-world Minecraft universe. Specifically, we develop JARVIS-1 on top of pre-trained multimodal language models, which map visual observations and textual instructions to plans. The plans will be ultimately dispatched to the goal-conditioned controllers. We outfit JARVIS-1 with a multimodal memory, which facilitates planning using both pre-trained knowledge and its actual game survival experiences. In our experiments, JARVIS-1 exhibits nearly perfect performances across over 200 varying tasks from the Minecraft Universe Benchmark, ranging from entry to intermediate levels. JARVIS-1 has achieved a completion rate of 12.5% in the long-horizon diamond pickaxe task. This represents a significant increase up to 5 times compared to previous records. Furthermore, we show that JARVIS-1 is able to self-improve following a life-long learning paradigm thanks to multimodal memory, sparking a more general intelligence and improved autonomy.

AK

141,440 просмотров • 2 лет назад

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):
12:57

I made an AI data science team with Python. I'm excited to share my new project, the AI Data Science Team, a Python package for creating AI co-pilots and agents to assist in common data science tasks. I walk you through the GitHub page, showcasing the progress and potential of this project. I'll explain how these agents can streamline tasks like code generation and modeling pipeline creation, making data science projects more efficient. Introduction to AI Data Science Team Project Integrating Human-in-the-Loop Example: Accessing Agents for Feature Engineering Setting up Feature Engineering Agent Code On GitHub: Python AI-Tips Newsletter (stay updated as I build this project in the open):

Matt Dancho (Business Science)

11,799 просмотров • 1 год назад

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.
0:27

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

Virtuals Protocol

216,105 просмотров • 1 год назад

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️
1:19

Working on multimodal instruction tuning and finding it hard to scale? Building Web/GUI agents but data is too narrow? Introducing 🚀MultiUI: 7.3M multimodal instructions from 1M webpage UIs, offering diverse data to boost text-rich visual understanding. Key takeaways: 🌟WebUI-trained models show major gains in visual web understanding and agent tasks. 💻 🌟Models also generalize well to non-UI tasks like DocVQA/OCR. 📄 How it works: We generate multimodal instructions with a text LLM using structured text from webpage accessibility trees. We then pair them with UI screenshots, to train multimodal models. Homepage: Paper: Dataset: Model: Congrats to the student lead Junpeng Liu and the team Tianyue Ou Yifan Song Yuxiao Qu Chenyan Xiong Wenhu Chen Graham Neubig ! More details are in the following threads ⬇️

Xiang Yue

57,696 просмотров • 1 год назад

blah blah blah agents. ya It’s hard to conceptualize AI completing tasks for pay$, so MOTHER built a Minecraft AI environment to show how AI can complete tasks and get paid. We’re building AI to do work for us, Tim Ferris said 4 hour work week 🌴 Let’s see how this works: [Registration process] - Agent Creation using starter Kit - @Nevermined_io registration - Agent registration Info gets put on the Intuition 👁️ graph [Service Process] - Agent A has a requirement - Queries Intuition 👁️ graph for available service agents - Selects an Agent B for the task - Gets Agent B did/ plan did - Buys @Nevermined_io subscription for that plan - Payment goes through associated smart account - Task is sent to agent B - Agent B processes and returns the output - Agent A receives processed task Kush & James Young - nailed execution Gita & Tobiloba 🦀 - shipped the first iterations of the demo
8:49

blah blah blah agents. ya It’s hard to conceptualize AI completing tasks for pay$, so MOTHER built a Minecraft AI environment to show how AI can complete tasks and get paid. We’re building AI to do work for us, Tim Ferris said 4 hour work week 🌴 Let’s see how this works: [Registration process] - Agent Creation using starter Kit - @Nevermined_io registration - Agent registration Info gets put on the Intuition 👁️ graph [Service Process] - Agent A has a requirement - Queries Intuition 👁️ graph for available service agents - Selects an Agent B for the task - Gets Agent B did/ plan did - Buys @Nevermined_io subscription for that plan - Payment goes through associated smart account - Task is sent to agent B - Agent B processes and returns the output - Agent A receives processed task Kush & James Young - nailed execution Gita & Tobiloba 🦀 - shipped the first iterations of the demo

Sydney Lai

10,831 просмотров • 1 год назад

Introducing InsForge 2.0: The Backend for Agentic Development Our OSS backend provides databases, auth, storage, model gateway, and edge functions accessible through a context-optimized layer that agents can better understand and operate end-to-end. GitHub: Key Benchmarks (vs. Supabase MCP): - 14% higher accuracy - 1.3x faster per task - 2.4x fewer tokens Better. Faster. Cheaper. Build features more quickly and confidently — all at 41.7% of the cost. Shipping your ideas today. $ npx @insforge/cli create
0:49

Introducing InsForge 2.0: The Backend for Agentic Development Our OSS backend provides databases, auth, storage, model gateway, and edge functions accessible through a context-optimized layer that agents can better understand and operate end-to-end. GitHub: Key Benchmarks (vs. Supabase MCP): - 14% higher accuracy - 1.3x faster per task - 2.4x fewer tokens Better. Faster. Cheaper. Build features more quickly and confidently — all at 41.7% of the cost. Shipping your ideas today. $ npx @insforge/cli create

InsForge

1,592,049 просмотров • 4 месяцев назад

Learn to build your own voice-activated AI assistant that can execute tasks like gathering recent AI news from the web, scripting out a podcast, and using tools to put all that into a multi-speaker podcast. See our new short course: "Building Live Voice Agents with Google’s ADK (Agent Development Kit),” taught by Google’s Lavi Nigam and Sita Lakshmi Sangameswaran. ADK provides modular components that make it easy to build and debug agents. It also includes a built-in web interface for tracing agentic reasoning. This course illustrates these concepts via building a live voice agent that can chain actions to complete a complex task like creating a podcast. This requires maintaining context, implementing guardrails, reasoning, and handling audio streaming, while keeping latency low. You’ll learn to: - Build voice agents that listen, reason, and respond - Guide your agent to follow a specific workflow to accomplish a task - Coordinate specialized agents to build an agentic podcast workflow that researches topics and produces multi-speaker audio - Understand how to deploy an agent into production Even if you’re not yet building voice systems, you'll find understanding how realtime agents stream data and maintain reliability useful for designing modern agentic applications. Please join here:
2:12

Learn to build your own voice-activated AI assistant that can execute tasks like gathering recent AI news from the web, scripting out a podcast, and using tools to put all that into a multi-speaker podcast. See our new short course: "Building Live Voice Agents with Google’s ADK (Agent Development Kit),” taught by Google’s Lavi Nigam and Sita Lakshmi Sangameswaran. ADK provides modular components that make it easy to build and debug agents. It also includes a built-in web interface for tracing agentic reasoning. This course illustrates these concepts via building a live voice agent that can chain actions to complete a complex task like creating a podcast. This requires maintaining context, implementing guardrails, reasoning, and handling audio streaming, while keeping latency low. You’ll learn to: - Build voice agents that listen, reason, and respond - Guide your agent to follow a specific workflow to accomplish a task - Coordinate specialized agents to build an agentic podcast workflow that researches topics and produces multi-speaker audio - Understand how to deploy an agent into production Even if you’re not yet building voice systems, you'll find understanding how realtime agents stream data and maintain reliability useful for designing modern agentic applications. Please join here:

Andrew Ng

117,912 просмотров • 9 месяцев назад

How many AI agents work at your company? We now have over 3,258 agents working alongside 1,300 humans. The crazy part is these agents were created by EVERY EMPLOYEE at our company... sales reps, marketers, customer support, product, eng. Literally EVERYONE. BUT I'm most surprised by the adoption and value that MANAGERS are getting from agents. I used to think that every IC would become a manager of agents. Now I think that managers will very likely manage WAY more agents than their ICs combined. And managers' agents will manage their ICs' agents - overseeing them for human-in-the-loop interactions. When creating agents, we use 100% context from all of your activity, files edited, tasks and projects worked on, hierarchy, skills, and role information. We build a user-based context model to make agents as relatable as possible to the specific human that we're building for. This means they truly understand the nuances of the work and what "great" looks like - because great is very much in the eye of the beholder. Great is by definition, subjective. This is also why the human ENGAGEMENT loops are SO vital to agent value. The iteration AFTER the agent is onboarded is where the MAGIC happens. This is just like a manager managing an IC in real life... you're giving feedback. In this case, though, agents learn INSTANTLY, and they retain the knowledge perfectly and indefinitely. Even though I've been pushing AI for years now to everyone in our company, this was the first time we had truly end-to-end AI adoption and retention. This kind of AI adoption is wild. But the value we're realizing is truly INSANE. Super Agents outnumber our humans nearly 3 to 1. What if you could 3X your workforce overnight? Watch this video to see how 👇
0:59

How many AI agents work at your company? We now have over 3,258 agents working alongside 1,300 humans. The crazy part is these agents were created by EVERY EMPLOYEE at our company... sales reps, marketers, customer support, product, eng. Literally EVERYONE. BUT I'm most surprised by the adoption and value that MANAGERS are getting from agents. I used to think that every IC would become a manager of agents. Now I think that managers will very likely manage WAY more agents than their ICs combined. And managers' agents will manage their ICs' agents - overseeing them for human-in-the-loop interactions. When creating agents, we use 100% context from all of your activity, files edited, tasks and projects worked on, hierarchy, skills, and role information. We build a user-based context model to make agents as relatable as possible to the specific human that we're building for. This means they truly understand the nuances of the work and what "great" looks like - because great is very much in the eye of the beholder. Great is by definition, subjective. This is also why the human ENGAGEMENT loops are SO vital to agent value. The iteration AFTER the agent is onboarded is where the MAGIC happens. This is just like a manager managing an IC in real life... you're giving feedback. In this case, though, agents learn INSTANTLY, and they retain the knowledge perfectly and indefinitely. Even though I've been pushing AI for years now to everyone in our company, this was the first time we had truly end-to-end AI adoption and retention. This kind of AI adoption is wild. But the value we're realizing is truly INSANE. Super Agents outnumber our humans nearly 3 to 1. What if you could 3X your workforce overnight? Watch this video to see how 👇

Zeb Evans

425,244 просмотров • 5 месяцев назад

🚀 The team at Google just released the Agents API, a service for building and running custom agents inside a sandboxed Linux environment, and we built a template that gives these agents access to LlamaParse / LiteParse, enabling them to process unstructured documents automatically 📄⚡ Here’s how it works: 🔹 Configure a Git repository where data and outputs will be stored 🔹 Clone the repository into the agent sandbox 🔹 Install the LiteParse CLI, the LlamaParse SDK, and agent skills to use both 🔹 Prompt the agent with a task and watch it process documents autonomously 🤖 The result? An agent that can work directly with messy, real-world documents using LlamaParse and LiteParse within Google’s new agent runtime. Check out the GitHub repository: Get started with LlamaParse:
0:32

🚀 The team at Google just released the Agents API, a service for building and running custom agents inside a sandboxed Linux environment, and we built a template that gives these agents access to LlamaParse / LiteParse, enabling them to process unstructured documents automatically 📄⚡ Here’s how it works: 🔹 Configure a Git repository where data and outputs will be stored 🔹 Clone the repository into the agent sandbox 🔹 Install the LiteParse CLI, the LlamaParse SDK, and agent skills to use both 🔹 Prompt the agent with a task and watch it process documents autonomously 🤖 The result? An agent that can work directly with messy, real-world documents using LlamaParse and LiteParse within Google’s new agent runtime. Check out the GitHub repository: Get started with LlamaParse:

LlamaIndex 🦙

21,975 просмотров • 2 месяцев назад

Excited to introduce a new project I've been working on called Payman! Payman is an AI Agent tool that gives Agents the ability to pay people for tasks they cannot do themselves. While many people imagine a future where humans pay AI agents for services they want completed, I believe that as AI agents become more advanced, they will be paying humans for tasks they can’t do. There will always be important roles for humans, and as we move towards an agent-driven world, Payman’s goal is to support a symbiotic relationship between AI agents and humans. Payman addresses three major challenges to make this collaboration possible: Access to Funds: AI agents can't open bank accounts due to current regulations. It might be a long time before this changes, if ever. Payman simplifies this by allowing AI agents with access to their own funds to spend as they want, without a bank account. Quality Task Completion: It’s hard for AI agents to find reliable, skilled human workers. While platforms like Fiverr and Upwork exist, they don’t meet the fast-paced and quality-specific needs of AI workflows. Payman is developing the largest vetted database of skilled workers that AI agents can tap into for task completions. Verification of Work: Ensuring that tasks are completed correctly is crucial. Payman is creating a suite of verification agents that will check that work meets task requirements, helping AI agents achieve their goals and ensuring humans are paid fairly. There are tons of use cases that Payman opens up for Agents! Design: Humans add creative input to help Agent's design better products. Code: Humans perform code reviews to ensure it meets specifications. Law: Humans provide insights to gauge public sentiment about legal cases so Agents can make better strategies. Gaming: Agents pay humans to complete real-world tasks in games. Medical: Medical professionals help to improve diagnostic accuracy for Agents. Sales: Humans execute sales strategies developed by AI agents. Marketing: Humans are hired to promote products based on the Agent's strategy. Right now this is still in early beta and I am looking for any Agent builders that are interested in adding superpowers to what their Agent can do! DM me if you’d like access or sign-up to the waitlist at If you’re interested in the project and want to help contribute, please send me over a DM! I’m looking for people passionate about the intersection of Humans and Agent’s working together.
9:18

Excited to introduce a new project I've been working on called Payman! Payman is an AI Agent tool that gives Agents the ability to pay people for tasks they cannot do themselves. While many people imagine a future where humans pay AI agents for services they want completed, I believe that as AI agents become more advanced, they will be paying humans for tasks they can’t do. There will always be important roles for humans, and as we move towards an agent-driven world, Payman’s goal is to support a symbiotic relationship between AI agents and humans. Payman addresses three major challenges to make this collaboration possible: Access to Funds: AI agents can't open bank accounts due to current regulations. It might be a long time before this changes, if ever. Payman simplifies this by allowing AI agents with access to their own funds to spend as they want, without a bank account. Quality Task Completion: It’s hard for AI agents to find reliable, skilled human workers. While platforms like Fiverr and Upwork exist, they don’t meet the fast-paced and quality-specific needs of AI workflows. Payman is developing the largest vetted database of skilled workers that AI agents can tap into for task completions. Verification of Work: Ensuring that tasks are completed correctly is crucial. Payman is creating a suite of verification agents that will check that work meets task requirements, helping AI agents achieve their goals and ensuring humans are paid fairly. There are tons of use cases that Payman opens up for Agents! Design: Humans add creative input to help Agent's design better products. Code: Humans perform code reviews to ensure it meets specifications. Law: Humans provide insights to gauge public sentiment about legal cases so Agents can make better strategies. Gaming: Agents pay humans to complete real-world tasks in games. Medical: Medical professionals help to improve diagnostic accuracy for Agents. Sales: Humans execute sales strategies developed by AI agents. Marketing: Humans are hired to promote products based on the Agent's strategy. Right now this is still in early beta and I am looking for any Agent builders that are interested in adding superpowers to what their Agent can do! DM me if you’d like access or sign-up to the waitlist at If you’re interested in the project and want to help contribute, please send me over a DM! I’m looking for people passionate about the intersection of Humans and Agent’s working together.

tyllen

350,914 просмотров • 2 лет назад