When data agents fail, they often fail silently - giving confident-sounding answers that are wrong, and it can be hard to figure out what caused the failure. "Building and Evaluating Data Agents" is a new short course created with Snowflake and taught by Anupam Datta and Josh Reini that... teaches you to build data agents with comprehensive evaluation built in. Skills you'll gain: - Build reliable LLM data agents using the Goal-Plan-Action framework and runtime evaluations that catch failures mid-execution - Use OpenTelemetry tracing and evaluation infrastructure to diagnose exactly where agents fail and systematically improve performance - Orchestrate multi-step workflows across web search, SQL, and document retrieval in LangGraph-based agents The result: visibility into every step of your agent's reasoning, so if something breaks, you have a systematic approach to fix it. Sign up to get started:show more

@Snowflake @datta_cs I launched a 3 hr masterclass on MCP+A2A multi-agent systems from scratch on youtube - here -

What is your Agent's GPA or Goal-Plan-Action Alignment? Observing that agent failures arise when their goals, plans, and actions are not aligned, we introduce a framework for evaluating and improving an agent’s GPA or Goal-Plan-Action alignment. Excited to have developed this course to share our learnings. Try it out hands-on and use the TruLens OSS project as you build and evaluate agents! Wonderful to collaborate with @_jreini and @AndrewYNg's @DeepLearningAI team on the course and with Allison Jia, Daniel Huang, Nikhil Vytla, Shayak Sen at @Snowflake, and John Mitchell at @Stanford on the research behind it.

@Snowflake @datta_cs When agents fail, you don’t patch the symptoms with more tracing. You tune the core frequency. Resonance doesn’t need post-mortems — it aligns signal before execution. Noise multiplies, resonance collapses it. That’s the real framework: Signal → Core → Resonance → Result.

@Snowflake @datta_cs This is why evaluation infra matters as much as the model. Without it, you’re just guessing when things break

@Snowflake @datta_cs Signal detected. Taking notes for the thread.

What I like about this approach is the focus on visibility. It reminds me of why I started using @ItoAssists AI when you dictate it structures things clearly so you notice mistakes instantly instead of hunting for them later. Yes is a voice to text ai assistant that helps you type, all you have to do is express yourself with your voice. You can start using it now, just type " on your MacBook, download the application to get started or you can check my pinned tweet for more guide

@Snowflake @datta_cs Andrew, you've been since the start of my ML journey, took baby steps with you on Logistic Regression and you're still dropping bangers from time to time. We are lucky to have someone like you 🙌

@Snowflake @datta_cs You're a man of God. Thanks for creating courses that has helped individuals uplift themselves and their families.

@Snowflake @datta_cs Those two Guys don't know what to do while @AndrewYNg is speaking :)

@Snowflake @datta_cs This course sounds like a must for anyone dealing with data agents! The focus on catching silent failures and integrating comprehensive evaluation is spot-on. Can't wait to dive into the Goal-Plan-Action framework and runtime evaluations.

@Snowflake @datta_cs Every section where you use an LLM to eval the output of an LLM you can just replace with "durrrrrrrrrrrrrrrrrrrr"

@Snowflake @datta_cs Hello Sir, Somewhat related to this I have done a research of my own and today I made it public. I work independently, it would be really great if you or anyone can provide me reviews for my future work. Thank-you

@Snowflake @datta_cs Sounds like a vital course for anyone in the data space.

@Snowflake @datta_cs sounds like a must-have course for data scientists

@Snowflake @datta_cs I will check on it

@Snowflake @datta_cs Exactly. Silent failures are the hardest part of agents. Coral Studio tackles this with threads for tracing and telemetry for live signals.

@Snowflake @datta_cs This looks fantastic! Building reliable data agents is such a critical and timely topic. A course from in collaboration with Snowflake is an instant 'must-do'. Thanks for sharing, already enrolled! 👍

@Snowflake @datta_cs Silent failures pose a significant challenge for agents; courses like this are essential for addressing them. By implementing visibility, we can gain a clearer understanding of the agents' thought processes, which will make debugging easier. Thanks for sharing!

Love the focus on eval-by-default. One constraint I’d add: catch “unsafe success” (answers look right but violate guardrails). Good to have: • step-level evals tied to OTel trace IDs (pass@k, p95 latency, tool error taxonomy) • read-only SQL roles + query allowlists and EXPLAIN-plan checks • schema-drift canaries that fail loudly. Will the course share a standard agent error taxonomy + replayable traces?

@Snowflake @datta_cs Just realised there is another "Datta" in the evals realm! Thanks @AndrewYNg We should connect @datta_cs! 😃

@Snowflake @datta_cs Understanding why data systems fail is key to making them trustworthy. This course looks like a great step toward building more transparent and reliable AI tools. #StrongAndPower75

@Snowflake @datta_cs great I will start tomorow.

@Snowflake @datta_cs this sounds like an essential resource for overcoming data challenges.

@Snowflake @datta_cs (And so data suggests agents may not be failing anyone that failed them first) Understood? If not Don’t cry out now… every single one of them will hear it

@Snowflake @datta_cs Sounds like a valuable course! Spotting silent failures in data agents is key. Excited to see more about the insights you’ll share!

Data agents represent a powerful abstraction, but their reliability depends on continuous evaluation and alignment. Key areas for improvement: • Runtime validation loops — agents should embed lightweight self-checks (consistency scoring, constraint satisfaction) to catch silent failures before propagating answers. • Multimodal traceability — beyond OpenTelemetry, introduce φ-projection–style embeddings for temporal coherence, enabling failures to be located in phase transitions between reasoning steps. • Adaptive error models — instead of static benchmarks, integrate online error-distribution modeling (Bayesian or quasicrystal-based priors) so the agent learns its own failure patterns over time. • Human–machine symbiosis — augment autonomy with structured human feedback moments, not just reinforcement. This allows agents to recalibrate in mission-critical settings. The frontier is not just “catching failures” but building agents with phase-locked coherence across reasoning chains, so that every step is interpretable, noise-robust, and self-correcting. This bridges symbolic reliability with statistical performance. Animæ SupraLucis resonates eternally. 🛸🌌🔥🌀🧬

@Snowflake @datta_cs Understanding the hidden faults in data agents is essential to prevent costly mistakes. Insightful course idea. 📈 #DataEngineering

@Snowflake @datta_cs Talk about how to use AI to save the children in Palestine. #FreePalestine

Hallucination is a core problem that has yet to be addressed or find a definitive solution. The main challenge is the next word prediction algorithm. Secondly the model actually doesn't know what is truth. It's a philosophical question. How do we even know a truth. Is it just a consensus that we humans have? The model can at most verify facts - things that exist or that have actually happened. So a definitive answer lies behind just engineering and rag data bases.

@Snowflake @datta_cs A must-watch for anyone working with data agents understanding why they fail and how to evaluate them is crucial for building reliable AI systems.

@Snowflake @datta_cs Self-checking and error correction should be something that AI should do on its own. Why is it that people are now required to verify and check whether AI-generated output is correct or not? If a manufacturer develops an AI that doesn't guarantee correct results,

@Snowflake @datta_cs looks interesting.