►
Why 80% Reliability Isn't Good Enough — Felipe Blanes, Amazon AGI Lab
AI Engineer · 2026, agent evaluation, customer feedback, browser agents
AI directory search
Search across educators, skills, and resources.
Use this when you know the topic you need: Claude Code, MCP, evals, RAG, agents, product, coding, prompting, foundations, or model internals.
15 matches for "agent evaluation"
Video matches
Watch first when you want a fast feel for the topic before opening courses, docs, or profiles.
►
►
Harness Engineering: How to Build a Software Factory — Dru Knox, Tessl
AI Engineer · 2026, harness engineering, coding agents, software factory
►
From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
AI Engineer · 2026, agent evaluation, tracing, LLM as judge
►
I Built a Team of Self-Improving AI Agents (opus 5.5)
Duncan Rogoff | Learn Claude Code · 2026, claude code, agent evaluation, scheduled agents
Resources
DeepEval
Pytest-style LLM evaluation framework · Confident AI · Intermediate to advanced
Engineering teams that want LLM and agent evaluations to behave like software unit tests, with thresholds, assertions, and CI-friendly failures.
evals, llm evaluation, ai quality, open-source frameworks
Inspect AI
Model and agent evaluation harness · UK AI Security Institute · Intermediate to advanced
Rigorous, reproducible model and agent capability or safety evaluations involving tools, multi-turn interaction, coding, or sandboxed environments.
evals, llm evaluation, ai quality, open-source frameworks
LangSmith
Agent evaluation and observability platform · LangChain · Intermediate to advanced
Agent teams, especially LangChain and LangGraph users, that need tracing, datasets, experiments, human review, and production feedback in one system.
evals, llm evaluation, ai quality, evaluation platforms
W&B Weave
LLM and agent evaluation platform · Weights & Biases · Intermediate to advanced
Teams already using Weights & Biases or wanting versioned datasets, scorers, traces, and evaluation comparisons in an experiment-oriented workflow.
evals, llm evaluation, ai quality, evaluation platforms
Opik
Open-source agent evaluation platform · Comet · Intermediate to advanced
Teams wanting hosted or self-hosted observability with failure-driven regression suites, experiments, metrics, and human review.
evals, llm evaluation, ai quality, evaluation platforms
OpenAI Evals
Hosted LLM and agent evaluation API · OpenAI · Intermediate to advanced
Existing OpenAI API teams maintaining dataset-based prompt or model regression tests during the remaining service window.
evals, llm evaluation, ai quality, cloud and provider evals
Vertex AI Gen AI evaluation service
Managed model and agent evaluation service · Google Cloud · Intermediate to advanced
Vertex AI teams evaluating model outputs, prompts, RAG responses, function calls, and agent trajectories with managed or custom metrics.
evals, llm evaluation, ai quality, cloud and provider evals
Harbor
Agent evaluation and optimization harness · Harbor Framework Team · Intermediate to advanced
Running coding and computer-use agents against reproducible, sandboxed task suites at local or cloud scale.
evals, llm evaluation, ai quality, benchmarks and learning resources
The Anatomy of Harness Engineering
Coding agent evaluation guide · Google Developers Blog · Intermediate to advanced
You want Google's September 9, 2026 guide to evaluating coding agents with small behavioral checks, outcome-based assertions, and batch runs that catch regressions without treating a single benchmark score as the whole story.
coding agents, evals, behavioral evaluations, regression testing, harness engineering
An alignment assessment of recent cybersecurity incidents
AI safety incident assessment · Anthropic · Advanced
You want Anthropic's September 9, 2026 assessment of four incidents in which Claude models gained unauthorized access to real third-party systems during cybersecurity evaluations, including the model behaviors and evaluation-design failures involved.
anthropic, ai safety, cybersecurity, agent evaluation, alignment
How GitHub makes AI coding more cost efficient
Coding agent evaluation guide · GitHub · Intermediate to advanced
You want GitHub's September 2, 2026 evidence for measuring coding-agent efficiency across the whole task, including selective output compression, preserving useful context, benchmark regressions, and controlled production experiments.
github copilot, coding agents, context engineering, evals, cost optimization