►
From Vibes to Production: Evaluating and Shipping AI Agents That Work 101 — Laurie Voss, Arize AI
AI Engineer · 2026, agent evaluation, tracing, LLM as judge
AI directory search
Search across educators, skills, and resources.
Use this when you know the topic you need: Claude Code, MCP, evals, RAG, agents, product, coding, prompting, foundations, or model internals.
16 matches for "datasets"
Video matches
Watch first when you want a fast feel for the topic before opening courses, docs, or profiles.
►
Providers and platforms
Hugging Face
Hugging Face Learn · Beginner to advanced
One of the best free ecosystems for learning open-source AI by building with models, datasets, spaces, agents, context engineering, and MCP workflows.
Topics
Agents, Context engineering, MCP, Transformers, Post-training, Open models
Langfuse
Langfuse Docs · Intermediate
Good operational material for tracing, scoring, and improving production LLM apps.
Topics
Observability, Prompt management, Evals, Tracing
Humanloop
Humanloop Blog and Docs · Intermediate
Useful for teams building repeatable AI product processes around prompts, datasets, and evaluations.
Topics
Prompt management, Evals, LLM workflows
Resources
Arize Phoenix
AI observability and evaluation platform · Arize AI · Intermediate to advanced
Teams that want evaluation connected to OpenTelemetry traces, datasets, experiments, prompt iterations, and production troubleshooting.
evals, llm evaluation, ai quality, evaluation platforms
Braintrust
AI evaluation and observability platform · Braintrust Data · Intermediate to advanced
Teams that want a feedback loop from production traces and failures into datasets, regression experiments, CI gates, and continuous scoring.
evals, llm evaluation, ai quality, evaluation platforms
LangSmith
Agent evaluation and observability platform · LangChain · Intermediate to advanced
Agent teams, especially LangChain and LangGraph users, that need tracing, datasets, experiments, human review, and production feedback in one system.
evals, llm evaluation, ai quality, evaluation platforms
Langfuse
Open-source evals and observability platform · ClickHouse · Intermediate to advanced
Teams prioritizing open-source data control and one workflow across tracing, prompts, datasets, experiments, annotation, and evaluation.
evals, llm evaluation, ai quality, evaluation platforms
W&B Weave
LLM and agent evaluation platform · Weights & Biases · Intermediate to advanced
Teams already using Weights & Biases or wanting versioned datasets, scorers, traces, and evaluation comparisons in an experiment-oriented workflow.
evals, llm evaluation, ai quality, evaluation platforms
Anthropic evaluation guide
Evaluation methodology guide · Anthropic · Intermediate to advanced
Teams designing task-specific evaluation suites for Claude applications that need criteria, datasets, edge cases, grading patterns, and code examples.
evals, llm evaluation, ai quality, cloud and provider evals
Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers
Training guide · Hugging Face · Advanced
You need an end-to-end August 26, 2026 recipe for adapting a late-interaction retriever to your own domain, from datasets and losses through training, evaluation, and index optimization.
hugging face, sentence transformers, rag, retrieval, fine-tuning
OpenAI eval design guide
Guide · OpenAI · Intermediate
You need practical guidance for designing representative eval datasets, choosing graders, and turning model testing into an engineering loop instead of ad hoc spot checks, especially while OpenAI's older Evals platform is winding down toward read-only status on October 31, 2026.
openai, evals, quality, datasets, regression testing
OpenAI evals quickstart and datasets
Guide · OpenAI · Intermediate
You want OpenAI's current quickstart for turning examples into dataset-backed evals and improvement loops instead of relying on a deprecated docs path.
openai, datasets, evals, fine-tuning, quality
Cohere Embed Jobs API
API docs · Cohere · Intermediate
You need Cohere's bulk embedding workflow for larger corpora instead of issuing one embedding request per document at ingestion time.
cohere, embed jobs, embeddings, batch processing, retrieval
Hugging Face model hub
Model catalog · Hugging Face · Beginner to advanced
You need to discover, compare, and run open model checkpoints, datasets, and demos.
hugging face, model hub, open models, llama, qwen
Hugging Face MCP Server
Integration guide · Hugging Face · Intermediate
You want the official Hugging Face MCP setup guide for connecting Codex, Claude Desktop, Cursor, VS Code, or other MCP clients directly to Hub tools and search.
hugging face, mcp, hub, tools, coding agents