Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

AIP Evolve — our new product for making agents more efficient and cost effective. See how Chad and Colton used it to autonomously swap models, tune prompts, validate outputs, and find structured ontology data that eliminated 2 LLM calls; cutting compute costs while improving accuracy and reliability in production.

Palantir

441,608 subscribers

159,367 views • 2 months ago •via X (Twitter)

Science & Technology Finance

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

See how Palantir Software Engineer Colton Rusch uses AIP Evolve to prove that cheaper AI doesn't have to mean worse AI: "Evolve screened a whole suite of models across providers, then used evals and expert feedback to rule out certain options and keep others for further optimization." "We got to the point where evolve produced a vastly cheaper version — 68% lower compute costs." "We got lower costs and better quality at the same time."

See how Palantir Software Engineer Colton Rusch uses AIP Evolve to prove that cheaper AI doesn't have to mean worse AI: "Evolve screened a whole suite of models across providers, then used evals and expert feedback to rule out certain options and keep others for further optimization." "We got to the point where evolve produced a vastly cheaper version — 68% lower compute costs." "We got lower costs and better quality at the same time."

Palantir

13,673 views • 27 days ago

Tokenmaxxing is for stupid people. Chad Wahlquist says Palantir’s AIP Evolve helps guide you to pick which model is actually the most efficient and cost-effective: “McCarthy talked about this—in 2 days they were able to eliminate 60% of their token cost.” Via TBPN

Tokenmaxxing is for stupid people. Chad Wahlquist says Palantir’s AIP Evolve helps guide you to pick which model is actually the most efficient and cost-effective: “McCarthy talked about this—in 2 days they were able to eliminate 60% of their token cost.” Via TBPN

Jawwwn

19,736 views • 2 months ago

New Short Course: Getting Structured LLM Output! Learn how to get structured outputs from your LLM applications in this course, built in partnership with .txt, and taught by Will Kurt, a Founding Engineer, and , Developer Relations Engineer. It's challenging for software to automatically parse through an LLM's freeform text outputs. Structured outputs—like JSON—solve this by converting natural language into consistent, clear, data that a machine can read and process. This course teaches you how to generate structured outputs while building several use cases, including a social media analysis agent. You’ll learn about structured outputs and efficient ways to generate outputs in your defined schema or format. You’ll begin by using structured output APIs, then use re-prompting libraries like “instructor” to generate structured output. Finally, you’ll learn how constrained decoding works; this is a very clever technique in which constraints are applied on each subsequent token generated, blocking any tokens that don’t fit your defined schema. In detail, you’ll: - Learn why structured outputs are important, how they allow for scalable software development, and the different approaches to generate them, including vendor-provided APIs, re-prompting libraries, and structured generation. - Build a simple social media agent using OpenAI’s structured output API, learn how to define a model's desired structured output using Pydantic, and perform basic programming with your outputs, such as importing structured data into a data frame using pandas. - Learn how to use the open-source library "instructor," which checks the structured output of the model and re-prompts the model until it validates the desired output, and explore the limitations of this approach. - Understand how structured generation by the “outlines” library works by modifying LLM logits, on a per-generated-token basis based on the desired format, to give a particular output structure. - Learn how regular expressions, which outlines works with, are represented as finite-state machines, and how they can be used to develop a range of structured outputs beyond JSON. By the end of this course, you’ll have broadened your knowledge of the approaches you can use to get structured outputs from your LLM applications. Please sign up here:

New Short Course: Getting Structured LLM Output! Learn how to get structured outputs from your LLM applications in this course, built in partnership with .txt, and taught by Will Kurt, a Founding Engineer, and , Developer Relations Engineer. It's challenging for software to automatically parse through an LLM's freeform text outputs. Structured outputs—like JSON—solve this by converting natural language into consistent, clear, data that a machine can read and process. This course teaches you how to generate structured outputs while building several use cases, including a social media analysis agent. You’ll learn about structured outputs and efficient ways to generate outputs in your defined schema or format. You’ll begin by using structured output APIs, then use re-prompting libraries like “instructor” to generate structured output. Finally, you’ll learn how constrained decoding works; this is a very clever technique in which constraints are applied on each subsequent token generated, blocking any tokens that don’t fit your defined schema. In detail, you’ll: - Learn why structured outputs are important, how they allow for scalable software development, and the different approaches to generate them, including vendor-provided APIs, re-prompting libraries, and structured generation. - Build a simple social media agent using OpenAI’s structured output API, learn how to define a model's desired structured output using Pydantic, and perform basic programming with your outputs, such as importing structured data into a data frame using pandas. - Learn how to use the open-source library "instructor," which checks the structured output of the model and re-prompts the model until it validates the desired output, and explore the limitations of this approach. - Understand how structured generation by the “outlines” library works by modifying LLM logits, on a per-generated-token basis based on the desired format, to give a particular output structure. - Learn how regular expressions, which outlines works with, are represented as finite-state machines, and how they can be used to develop a range of structured outputs beyond JSON. By the end of this course, you’ll have broadened your knowledge of the approaches you can use to get structured outputs from your LLM applications. Please sign up here:

Andrew Ng

89,792 views • 1 year ago

Developers are coming to Avalanche for AI; it offers customizable compute and data structures, plus a robust ecosystem across finance, gaming, payments, and more. Listen to KITE AI talk about how this synergy is powering new AI agents and models for every sector. For more: 👇

Developers are coming to Avalanche for AI; it offers customizable compute and data structures, plus a robust ecosystem across finance, gaming, payments, and more. Listen to KITE AI talk about how this synergy is powering new AI agents and models for every sector. For more: 👇

Avalanche🔺

22,981 views • 1 year ago

We’re expanding our strategic partnership with Microsoft to: ⚡ Deliver OpenAI’s state-of-the-art models in Snowflake Cortex AI on Microsoft Microsoft Azure. Customers will soon be able to build AI-powered data agents to run analytical workflows on structured and unstructured data using OpenAI’s models. ⚡ Make Snowflake Cortex Agents available in Microsoft 365 Copilot and Microsoft 365 apps, ensuring AI insights become more accessible for users and enabling better decision-making across the enterprise. Learn more about how we’re bringing easy, efficient, and trusted AI to enterprises around the world:

We’re expanding our strategic partnership with Microsoft to: ⚡ Deliver OpenAI’s state-of-the-art models in Snowflake Cortex AI on Microsoft Microsoft Azure. Customers will soon be able to build AI-powered data agents to run analytical workflows on structured and unstructured data using OpenAI’s models. ⚡ Make Snowflake Cortex Agents available in Microsoft 365 Copilot and Microsoft 365 apps, ensuring AI insights become more accessible for users and enabling better decision-making across the enterprise. Learn more about how we’re bringing easy, efficient, and trusted AI to enterprises around the world:

Snowflake

11,011 views • 1 year ago

The Chairman and CEO of AIG spent over a minute talking about how critical the Ontology is to deploying LLMs in the enterprise 💪 “Ontology is critical for deploying large language models. It brings together the relevant data sets that define the components of our insurance business, integrates and sequences them and then models how they relate to one another. Our ontology will create a clear record of any actions taken, which will inform business logic and provide the ability to audit agents activities.”

The Chairman and CEO of AIG spent over a minute talking about how critical the Ontology is to deploying LLMs in the enterprise 💪 “Ontology is critical for deploying large language models. It brings together the relevant data sets that define the components of our insurance business, integrates and sequences them and then models how they relate to one another. Our ontology will create a clear record of any actions taken, which will inform business logic and provide the ability to audit agents activities.”

Chad Wahlquist

23,869 views • 1 year ago

Structured Outputs: The Building Blocks for Reliable AI! 🏗️ I am SUPER EXCITED to publish our newest Weaviate Podcast featuring Will Kurt (Will Kurt) and Cameron Pfiffer (Cameron) from .txt! 🎙️🎉 Dottxt is the company behind Outlines, reshaping how we control LLM outputs with constrained decoding! 🚀 This podcast dives into all sorts of details from new applications in metadata and information extraction, such as self-expanding knowledge graphs, to structured reasoning, report generation, and more! We also dove into the details of how this leverages Automata Theory and Finite State Machines to achieve this, how it is packaged with inference engines such as vLLM, and a rebuttal to Let Me Speak Freely, discussing how Structured Generation actually improve the quality of LLM outputs, in addition to speeding up inference, and of course, making it more reliable! I hope you find the podcast interesting, this was a super fun one! Links below!

Structured Outputs: The Building Blocks for Reliable AI! 🏗️ I am SUPER EXCITED to publish our newest Weaviate Podcast featuring Will Kurt (Will Kurt) and Cameron Pfiffer (Cameron) from .txt! 🎙️🎉 Dottxt is the company behind Outlines, reshaping how we control LLM outputs with constrained decoding! 🚀 This podcast dives into all sorts of details from new applications in metadata and information extraction, such as self-expanding knowledge graphs, to structured reasoning, report generation, and more! We also dove into the details of how this leverages Automata Theory and Finite State Machines to achieve this, how it is packaged with inference engines such as vLLM, and a rebuttal to Let Me Speak Freely, discussing how Structured Generation actually improve the quality of LLM outputs, in addition to speeding up inference, and of course, making it more reliable! I hope you find the podcast interesting, this was a super fun one! Links below!

Connor Shorten

12,742 views • 1 year ago

Defend the enterprise. Security Forge, our new cyber security offering for source code vulnerability detection that moves at machine speed. See how Chad and George used it to autonomously scan a codebase, compress 109 flags down to 10 actionable findings, and do it for $78 — so security teams know exactly what to fix and why.

Defend the enterprise. Security Forge, our new cyber security offering for source code vulnerability detection that moves at machine speed. See how Chad and George used it to autonomously scan a codebase, compress 109 flags down to 10 actionable findings, and do it for $78 — so security teams know exactly what to fix and why.

Palantir

2,298,416 views • 2 months ago

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Learn a development pattern to systematically improve the accuracy and reliability of LLM applications in our new short course, Improving Accuracy of LLM Applications, built in partnership with Lamini and Meta, and taught by Lamini’s CEO Sharon Zhou, and Meta’s Senior Director of Partner Engineering, Amit Sangani. (Disclosure: I am an investor in Lamini.) The path to tuning an LLM application can be complex. In this course, you'll learn a systematic sequence of steps for improving accuracy by reducing hallucinations: - Create an evaluation dataset to measure model accuracy - Add prompt engineering and self-reflection - Fine-tune your model including "memory-tuning" which is a new method of embedding facts in an LLM Using the Llama 3-8B parameter model, you will: - Build a text-to-SQL agent with a custom schema and simulate situations where it hallucinates - Understand the difference between instruction fine-tuning, which gives pre-trained LLMs instructions to follow, and memory fine-tuning - See how Performance-Efficient Fine-tuning (PEFT) techniques like Low-Rank Adaptation (LoRA) reduce training time by 100x and Mixture of Memory Experts (MoME) reduces it even further I appreciate Meta releasing the Llama's family of open models -- this course gives an example of the unique type of work that developers can do with such models. Please sign up here:

Andrew Ng

66,435 views • 2 years ago

Multi-agent systems offer incredible potential and unprecedented risks. How do you solve for observability, failure mode analysis, and guardrailing in the era of agents? Today, we’re announcing our Agent Reliability platform to observe, evaluate, guardrail, and improve agents at scale. You can get started with the complete platform for trustworthy agentic AI today for free, and here’s how we’re solving some of the biggest challenges in agent reliability: - Observability redesigned for agents Trace views collapse under complex workflows, so we created the Graph View, Timeline View, and Conversation View to offer rich, intuitive visualizations of agent decisions, tool calls, and conversation flows. This multi-dimensional approach enables teams to pinpoint exactly where and why agents deviate or fail. - Automated Failure Mode Analysis with our new Insights Engine Our Insights Engine ingests your logs, metrics, and agent code to automatically surface nuanced failure modes and their root causes. But knowing the problem is not enough; you need to know how to fix it. Insights Engine delivers actionable fixes and can even apply them automatically. With adaptive learning, your insights become smarter and more relevant as your agents evolve. - Evaluating Agents Across Multiple Dimensions Agentic systems interact across complex pathways, and evaluating their performance requires new metrics that reflect this increasing complexity. To deliver comprehensive agentic measurements, we’ve added more out-of-the-box agent metrics like flow adherence, agent flow, agent efficiency, and more. For specialized domains and unique workflows, custom metrics powered by our new Luna-2 small language models can be rapidly designed and fine-tuned for your specific use case. - Real-Time Guardrails Powered by Luna-2 As AI agents become more autonomous and complex, failures like hallucinations or unsafe actions increase dramatically. Without real-time guardrails, these errors will hurt your user experience and brand reputation. Our Luna-2 family of small language models is purpose-built to provide low-latency, cost-effective guardrails that actively stop agent errors before they happen. With support for out-of-the-box and custom metrics, Luna-2 enables enterprises to enforce safety, compliance, and reliability at scale. Enterprises running hundreds of agents and processing hundreds of millions of queries daily already rely on Galileo’s Agent Reliability platform to protect their users, safeguard brand trust, and accelerate innovation. Agent Reliability is available starting today. Try it for free and experience the new standard in AI reliability. Learn more below 👇

Galileo

1,276,298 views • 1 year ago

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!

Sharing our latest short course: Building and Evaluating Data Agents, created in collaboration with Snowflake and taught by Anupam Datta (Anupam Datta) and Josh Reini (Josh Reini). A data agent extracts data from sources such as files or databases, analyzes it, and provides insights and visualizes its findings. But most data agents struggle with reliability or can't handle multi-step reasoning. In this course, you'll learn to build, trace, and evaluate a multi-agent workflow that plans tasks, pulls context from structured and unstructured data, performs web search, and summarizes or visualizes the final results. Learn more and enroll for free!

DeepLearning.AI

40,852 views • 10 months ago

AI agents will choose the most efficient and cost effective networks. That takes more than speed. It requires a fundamental rethink of network architecture. Adeniyi.sui joins Coin Bureau to explain how Sui is built for a trillion dollar onchain future.

AI agents will choose the most efficient and cost effective networks. That takes more than speed. It requires a fundamental rethink of network architecture. Adeniyi.sui joins Coin Bureau to explain how Sui is built for a trillion dollar onchain future.

Sui

18,406 views • 4 months ago

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

Revolutionizing Move Programming with OpenLedger In this demo, we showcase how Move datasets contributed by data providers to OpenLedger’s datanets are used to fine-tune specialized models with LoRA fine-tuning. As seen in the video, we showcase an example on how builders can deploy a Move-specialized model that powers Co-pilot agents using our no-code model fine-tuning platform. This is the future of AI and Web3 innovation. Watch this space to see more specialised models and data feeds being built for next generation agents on top of OpenLedger #Move

OpenLedger

61,662 views • 1 year ago

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Today, we’re excited to announce Deep Extract. We’re pushing the boundary of complex document extraction by utilizing an agent harness approach, iterating and verifying outputs until they are at human-level accuracy. The results speak for themselves: we’ve handled documents exceeding 2,500 pages, extracting a total of 30M+ fields in the last couple weeks with results in our production beta reaching 99-100% accuracy. It’s particularly effective on complex documents with long lists, such as invoice line items, brokerage statement transactions, equipment manifests, and more. See more on how we built it, how to use it, and more👇

Reducto

23,949,618 views • 4 months ago

🚀 New Generally Available Whisper drop: The fastest, most accurate, and cost-effective transcription with over 1000x real-time factor for production AI workloads. 🚀 Our new Generally Available Whisper implementation delivers: 🏎️ Over 1000x real-time factor ✨ The lowest word error rate 💪 Production-grade reliability 🧩 Custom scaling and hardware per processing step 👉 See how in our blog: Reach out to get record-breaking performance for your mission-critical AI workloads!

🚀 New Generally Available Whisper drop: The fastest, most accurate, and cost-effective transcription with over 1000x real-time factor for production AI workloads. 🚀 Our new Generally Available Whisper implementation delivers: 🏎️ Over 1000x real-time factor ✨ The lowest word error rate 💪 Production-grade reliability 🧩 Custom scaling and hardware per processing step 👉 See how in our blog: Reach out to get record-breaking performance for your mission-critical AI workloads!

Baseten

15,489 views • 1 year ago

Today we're introducing Gemini 3.7 Flash, our most intelligent workhorse model yet for coding and agents. This model brings substantial gains across software engineering, web development, and complex knowledge work. Now through the end of the year, Gemini 3.7 Flash is available at an introductory price* of $0.75/1M input and $3.75/1M output tokens — making it easier and more cost-effective for developers and customers to scale production-ready agents.

Today we're introducing Gemini 3.7 Flash, our most intelligent workhorse model yet for coding and agents. This model brings substantial gains across software engineering, web development, and complex knowledge work. Now through the end of the year, Gemini 3.7 Flash is available at an introductory price* of $0.75/1M input and $3.75/1M output tokens — making it easier and more cost-effective for developers and customers to scale production-ready agents.

Google

1,063,130 views • 3 days ago

introducing nano banana 2 lite: our fastest, most cost-effective gemini image model yet built for high-velocity developer pipelines, it delivers text-to-image outputs in 4 seconds at just $0.034 per 1K-resolution image swap it into your workflow today via ai studio and the gemini api

introducing nano banana 2 lite: our fastest, most cost-effective gemini image model yet built for high-velocity developer pipelines, it delivers text-to-image outputs in 4 seconds at just $0.034 per 1K-resolution image swap it into your workflow today via ai studio and the gemini api

Google AI Studio

264,737 views • 1 month ago

Revenue agents are the main course. Data enrichment is on the house. B2B orgs spend $3.1B a year for employees to find contact and company data. Now agents need it too. So we made a call: enrichment is a cost of doing agentic work, not a product to mark up. It's infrastructure that revenue agents need to run. Today, we’re opening up our in-house data collection and enrichment infrastructure to all. No extra contracts. No credits to count. Just revenue, served.

Revenue agents are the main course. Data enrichment is on the house. B2B orgs spend $3.1B a year for employees to find contact and company data. Now agents need it too. So we made a call: enrichment is a cost of doing agentic work, not a product to mark up. It's infrastructure that revenue agents need to run. Today, we’re opening up our in-house data collection and enrichment infrastructure to all. No extra contracts. No credits to count. Just revenue, served.

Rox

182,964 views • 1 month ago

ROUTE LLM - SUPER INTELLIGENCE IN THE MAKING RouteLLM combines the superpowers of o1-preview, Sonnet 3.5, Gemini Flash, o1-mini, 4o, and 4o-mini! Based on your query, it optimizes for latency, cost, and quality of the response. The cool thing about RouteLLM is it combines. - coding ability of Sonnet 3.5 - reasoning ability of o1-preview - hard-coding ability of o1-mini - 1M context of Gemini - speed of 4o-mini Similar to ensemble models that are very effective in classical ML, RouteLLM and similar models will be the most effective in the LLM ecosystem

ROUTE LLM - SUPER INTELLIGENCE IN THE MAKING RouteLLM combines the superpowers of o1-preview, Sonnet 3.5, Gemini Flash, o1-mini, 4o, and 4o-mini! Based on your query, it optimizes for latency, cost, and quality of the response. The cool thing about RouteLLM is it combines. - coding ability of Sonnet 3.5 - reasoning ability of o1-preview - hard-coding ability of o1-mini - 1M context of Gemini - speed of 4o-mini Similar to ensemble models that are very effective in classical ML, RouteLLM and similar models will be the most effective in the LLM ecosystem

Bindu Reddy

66,374 views • 1 year ago