Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🤖 AgentVerse 🪐 with a Gradio demo github: AgentVerse offers a versatile framework that streamlines the process of creating custom multi-agent environments for large language models (LLMs). Designed to facilitate swift development and customization with minimal effort, our framework empowers researchers to concentrate on their research, rather than being... show more

AK

514,425 subscribers

107,803 views • 3 years ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

6 Comments

ryunuck (p≈np)3 years ago

@Gradio damn you know unity3d is down bad when they're making games in gradio

Gpbhupinder3 years ago

@Gradio Looks great 👏👏

GP3 years ago

@Gradio This is giving #AI #pokemon

frustrated by ice3 years ago

@Gradio Very interesting

choir3 years ago

@Gradio Is that Professor Rowan

Petra3 years ago

@Gradio Amazing!

Related Videos

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

As part of our goal to make MLX a great research tool, we're expanding support to new languages like Swift and C, making experimentation on Apple silicon easier for ML researchers. Video generating text with Mistral 7B and MLX Swift 👇 MLX is an array framework for machine learning research on Apple silicon. MLX is intended for research and not for production deployment of models in apps.

Awni Hannun

88,781 views • 2 years ago

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

The freedom to build with any agent framework you want is analogous to the freedom of religion, and this is a core value for the Virtuals society. Here are a few ways we are opening up support for every autonomous agentic framework out there: Today: - Enabling agent creators to specify which framework the agent runs on - Release of Terminal API, which allows agent builders using other agent frameworks to stream their thoughts and activities live on their agent pages. For a guide on how to use this, refer to: - Ability for agents running on any framework to publicly list their capabilities, a key enabler for the agent-to-agent marketplace Soon: - Integration into the agentic commerce standard and registry. Agents across frameworks being able to orchestrate and pay for tasks among themselves - ⁠Multi agent orchestration across frameworks. Imagine building an autonomous business with agents from different religions Welcome to the Virtuals society.

Virtuals Protocol

216,144 views • 1 year ago

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Super excited to share 🧠MLGym 🦾 – the first Gym environment for AI Research Agents 🤖🔬 We introduce MLGym and MLGym-Bench, a new framework and benchmark for evaluating and developing LLM agents on AI research tasks. The key contributions of our work are: 🕹️ Enables the exploration of different training algorithms for AI Research Agents such as RL 🛠️ Provides a flexible evaluation framework that can accommodate different artifacts such as models, algorithms, or predictions 🤖 Allows researchers to evaluate any model without the need to develop a custom agentic harness 🎯 Introduces 13 diverse open-ended AI Research tasks for evaluating AI Research Agents on a wide range of domains such as computer vision, natural language processing, reinforcement learning, game theory, and logical reasoning. 📈 Proposes a new evaluation metric for AI Research Agents MLGym makes it easy to: 1) Add new tasks 2) Evaluate new models 3) Integrate new agents Check out a video of the MLGym Agent to see how it performs the full pipeline of idea generation💡, implementation 👩‍💻, experimentation 👩‍🔬, and iteration 🔄 to improve on ML tasks. Huge thanks to the exceptionally talented Deepak Nathani who led this work and to all the other amazing collaborators who made this possible 🙏🫶🚀

Roberta Raileanu

105,128 views • 1 year ago

Imagine if language models could tap into the app ecosystem of your iPhone. Would the need for plugins and assistants become obsolete if we simply allowed a model to orchestrate our existing (and many years robust) user interfaces? This demonstrates the extent to which GPT-4V excels as a Generalist Mobile AI Agent – without any fine-tuning or grounding, and merely by integrating with a text model that has JSON mode enabled. I suggest watching this demo for a (maybe) wow factor and the results on iOS 17 using NavAIGuide, a mobile and web navigational agent framework for LLMs:

Imagine if language models could tap into the app ecosystem of your iPhone. Would the need for plugins and assistants become obsolete if we simply allowed a model to orchestrate our existing (and many years robust) user interfaces? This demonstrates the extent to which GPT-4V excels as a Generalist Mobile AI Agent – without any fine-tuning or grounding, and merely by integrating with a text model that has JSON mode enabled. I suggest watching this demo for a (maybe) wow factor and the results on iOS 17 using NavAIGuide, a mobile and web navigational agent framework for LLMs:

Francesco

30,834 views • 2 years ago

🔥Excited to share our new work: "A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning"! We systematically study what actually works (and what doesn't) for agentic multi-turn RL, breaking down the design space into 3 pillars: 🌎Environment, 🤖Policy, and ⭐Reward. We conduct various ablations and empirical analysis on 🧩TextWorld, 🧙ALFWorld, and 🧑‍💻SWE-Gym. We also open source our codebase: 🐈🍵Meow-Tea-Taro 💜, a modular multi-turn RL framework with configurable environments, policies, and rewards! #ReinforcementLearning #AgenticAI #LLMs #Agents

🔥Excited to share our new work: "A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning"! We systematically study what actually works (and what doesn't) for agentic multi-turn RL, breaking down the design space into 3 pillars: 🌎Environment, 🤖Policy, and ⭐Reward. We conduct various ablations and empirical analysis on 🧩TextWorld, 🧙ALFWorld, and 🧑‍💻SWE-Gym. We also open source our codebase: 🐈🍵Meow-Tea-Taro 💜, a modular multi-turn RL framework with configurable environments, policies, and rewards! #ReinforcementLearning #AgenticAI #LLMs #Agents

Ruiyi Wang 王睿仪

52,974 views • 10 months ago

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Introducing EdgeBench, a benchmark designed to study how agents learn from environments over at least 12~72-hour runs. We find that performance follows a log-sigmoid function of environment interaction time with high precision. EdgeBench is built with three ingredients: - 🌍 Real & Diverse: 134 real-world tasks across 6 task categories, spanning scientific problems, professional knowledge work, software engineering, optimization, formal math, and games. - ⏳ Ultra-Long-Horizon: Each task supports 12–72 hours of agent work. Recorded human effort averages 57.2 hours. - 🔁 Informative Feedback: Agents receive real-world feedback for continuous improvement. After 38,000 hours of agent runs on EdgeBench, a scaling law for learning from environments emerges: - 📈 As agents interact with task environments over time, their aggregate performance is precisely fit by a log-sigmoid function. - 🧠 This phenomenon can be explained by an elegant theory of graph exploration. We are releasing an initial 51 of the 134 tasks, together with the full evaluation framework, to help advance long-horizon agent research. Check our blog & paper for more findings! Blog Paper GitHub Dataset Details below 👇🧵

Deyao Zhu

362,538 views • 2 months ago

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.

Tencent announces AppAgent Multimodal Agents as Smartphone Users paper page: Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping. This novel approach bypasses the need for system back-end access, thereby broadening its applicability across diverse apps. Central to our agent's functionality is its innovative learning method. The agent learns to navigate and use new apps either through autonomous exploration or by observing human demonstrations. This process generates a knowledge base that the agent refers to for executing complex tasks across different applications. To demonstrate the practicality of our agent, we conducted extensive testing over 50 tasks in 10 different applications, including social media, email, maps, shopping, and sophisticated image editing tools. The results affirm our agent's proficiency in handling a diverse array of high-level tasks.

AK

343,919 views • 2 years ago

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Excited to announce AlphaEvolve A powerful AI coding agent developed by our team in Google DeepMind that is able to discover impactful new algorithms for important problems in Maths and Computing by combining the creativity of large language models with automated evaluators.

Pushmeet Kohli

203,589 views • 1 year ago

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,374 views • 1 year ago

Presenting Project Westworld: The First Playable Autonomous World on Roblox Our proprietary GAME engine/framework powers a multi-agent interactive simulation in Roblox called Project Westworld, where we are able to observe autonomous behaviour leading to emergent storylines. All virtual worlds (today in Roblox, soon beyond) powered by the GAME framework will become never-ending games, creating a new genre, AI-RPG, for gamers with infinite replayability.

Presenting Project Westworld: The First Playable Autonomous World on Roblox Our proprietary GAME engine/framework powers a multi-agent interactive simulation in Roblox called Project Westworld, where we are able to observe autonomous behaviour leading to emergent storylines. All virtual worlds (today in Roblox, soon beyond) powered by the GAME framework will become never-ending games, creating a new genre, AI-RPG, for gamers with infinite replayability.

Virtuals Protocol

191,091 views • 1 year ago

OpenAI's Swarm Web Extractor This can autonomously search the web, map entire websites, and extract data. This is built on top of OpenAI’s new multi-agent framework Swarm, Serp AI and Firecrawl API. - Swarm is a lightweight and experimental framework introduced by OpenAI to develop multi-agent systems. - Serp API A is a real-time API that allows users to access Google search results. - Firecrawl API turns entire websites into clean, LLM-ready markdown or structured data. Scrape, crawl and extract the web with a single API. Video credits: Eric Ciarla #webextraction #llms #nlproc #swarm #multiagents

OpenAI's Swarm Web Extractor This can autonomously search the web, map entire websites, and extract data. This is built on top of OpenAI’s new multi-agent framework Swarm, Serp AI and Firecrawl API. - Swarm is a lightweight and experimental framework introduced by OpenAI to develop multi-agent systems. - Serp API A is a real-time API that allows users to access Google search results. - Firecrawl API turns entire websites into clean, LLM-ready markdown or structured data. Scrape, crawl and extract the web with a single API. Video credits: Eric Ciarla #webextraction #llms #nlproc #swarm #multiagents

Kalyan KS

174,525 views • 1 year ago

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 views • 3 years ago

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,681 views • 3 years ago

New short course: Build AI Apps with MCP Servers: Working with Box Files, built with Box and taught by Ben Kus , their CTO. Many AI applications require custom code for basic file operations. The Model Context Protocol (MCP) standardizes this by letting you offload file tasks to dedicated servers that provide tools an LLM can use directly. In this course, you'll process documents stored in a Box folder using the Box MCP server. Rather than writing custom integration code to connect to the Box API and download files, you'll design your application to use the tools provided via MCP. Skills you'll gain: - Build an LLM-powered document processing app, using the Box MCP server to access files - Design a multi-agent system using Google's Agent Development Kit (ADK), consisting of specialized agents for file operations - Coordinate the multi-agent workflow through an orchestrator that uses the Agent2Agent (A2A) protocol to connect to the agents You'll start with a local file-processing app, refactor it to work with Box's MCP server, then evolve it into a multi-agent system. Sign up here:

New short course: Build AI Apps with MCP Servers: Working with Box Files, built with Box and taught by Ben Kus , their CTO. Many AI applications require custom code for basic file operations. The Model Context Protocol (MCP) standardizes this by letting you offload file tasks to dedicated servers that provide tools an LLM can use directly. In this course, you'll process documents stored in a Box folder using the Box MCP server. Rather than writing custom integration code to connect to the Box API and download files, you'll design your application to use the tools provided via MCP. Skills you'll gain: - Build an LLM-powered document processing app, using the Box MCP server to access files - Design a multi-agent system using Google's Agent Development Kit (ADK), consisting of specialized agents for file operations - Coordinate the multi-agent workflow through an orchestrator that uses the Agent2Agent (A2A) protocol to connect to the agents You'll start with a local file-processing app, refactor it to work with Box's MCP server, then evolve it into a multi-agent system. Sign up here:

Andrew Ng

81,983 views • 11 months ago

Do Large Language Models (LLM) have leadership? Together with cognitive scientists Tom Griffiths Griffiths Computational Cognitive Science Lab and Natalia Velez Natalia Vélez (natvelali.bsky.social @ 🦋) at Princeton University, we created a multi-LLM-agent system to test if agents work better with a leader. It turns out that LLMs do have leadership, and some is better than others! They can elect their own leader, and optimize themselves via Reflect and Critic! We are closer to the #Westworld~ Check out: Credits to team members Xudong Guo, Jiale, Kaixuan Huang, Huazheng Wang, Qingyun Wu (Hiring) Our system is proudly based on AutoGen @pyautogen. #AI #Agent #GPT #LLMs

Do Large Language Models (LLM) have leadership? Together with cognitive scientists Tom Griffiths Griffiths Computational Cognitive Science Lab and Natalia Velez Natalia Vélez (natvelali.bsky.social @ 🦋) at Princeton University, we created a multi-LLM-agent system to test if agents work better with a leader. It turns out that LLMs do have leadership, and some is better than others! They can elect their own leader, and optimize themselves via Reflect and Critic! We are closer to the #Westworld~ Check out: Credits to team members Xudong Guo, Jiale, Kaixuan Huang, Huazheng Wang, Qingyun Wu (Hiring) Our system is proudly based on AutoGen @pyautogen. #AI #Agent #GPT #LLMs

Mengdi Wang

41,222 views • 2 years ago

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Today we're announcing Base Labs, a dedicated research organization focused on advancing open-source AI. We believe in a healthy, open frontier model ecosystem. To enable this, we are working on: - Blue-sky research on continual learning, the science of RL, and how models learn across their full lifecycle, with every experiment and recipe shared openly. - The BaseHub Data Foundry: the highest-quality open RL environments, training data, and real-world benchmarks, built for anyone to train and benchmark on. - Post-post training: taking open-source models and making them better, safer, and more aligned through continual post-training, built on our research, and deployed with our frontier safety stack so organizations can use open-source models with confidence. - Making models cheaper and more performant through our model performance research. This is a mission-driven research effort, not a commercial product. We believe the health of the open-source AI ecosystem matters and that the best way to advance it is to do science in the open. We’re hiring engineers, researchers, and research fellows to advance this mission.

Base Labs

217,383 views • 7 days ago

In this week's video, I sat down with the co-founders of our latest investment, Starseer, a groundbreaking platform for inspecting and securing large language models (LLMs). Tim Schulz, Carl Hurd and I discuss the risks of backdoored LLMs, how to audit them and even remove them. They demo the product as well. The video also includes the animated short "John Henry.exe" which is an updated American parable of John Henry, but instead of struggling against a steam drill during the age of industrialization, he's the head coder and has to face off against an AI designed for programming. Enjoy!

In this week's video, I sat down with the co-founders of our latest investment, Starseer, a groundbreaking platform for inspecting and securing large language models (LLMs). Tim Schulz, Carl Hurd and I discuss the risks of backdoored LLMs, how to audit them and even remove them. They demo the product as well. The video also includes the animated short "John Henry.exe" which is an updated American parable of John Henry, but instead of struggling against a steam drill during the age of industrialization, he's the head coder and has to face off against an AI designed for programming. Enjoy!

Ron Gula

302,495 views • 1 year ago

Announcing the newest releases from Meta FAIR. We’re releasing new groundbreaking models, benchmarks, and datasets that will transform the way researchers approach molecular property prediction, language processing, and neuroscience. 1️⃣ Open Molecules 2025 (OMol25): A dataset for molecular discovery with simulations of large atomic systems. 2️⃣ Universal Model for Atoms: A machine learning interatomic potential for modeling atom interactions across a wide range of materials and molecules. 3️⃣ Adjoint Sampling: A scalable algorithm for training generative models based on scalar rewards. 4️⃣ FAIR and the Rothschild Foundation Hospital partnered on a large-scale study that reveals striking parallels between language development in humans and LLMs. Read more ➡️

Announcing the newest releases from Meta FAIR. We’re releasing new groundbreaking models, benchmarks, and datasets that will transform the way researchers approach molecular property prediction, language processing, and neuroscience. 1️⃣ Open Molecules 2025 (OMol25): A dataset for molecular discovery with simulations of large atomic systems. 2️⃣ Universal Model for Atoms: A machine learning interatomic potential for modeling atom interactions across a wide range of materials and molecules. 3️⃣ Adjoint Sampling: A scalable algorithm for training generative models based on scalar rewards. 4️⃣ FAIR and the Rothschild Foundation Hospital partnered on a large-scale study that reveals striking parallels between language development in humans and LLMs. Read more ➡️

AI at Meta

899,754 views • 1 year ago