►
Local AI Is Only 4 Months Behind. I Did the Math.
Ash · 2026, local AI, open-weight models, Qwen3.8
AI directory search
Search across educators, skills, and resources.
Use this when you know the topic you need: Claude Code, MCP, evals, RAG, agents, product, coding, prompting, foundations, or model internals.
20 matches for "benchmarks"
Video matches
Watch first when you want a fast feel for the topic before opening courses, docs, or profiles.
►
►
CMU AI Agents 2026: 10. Deep Research
Graham Neubig · 2026, deep research, search agents, retrieval
►
Why 80% Reliability Isn't Good Enough — Felipe Blanes, Amazon AGI Lab
AI Engineer · 2026, agent evaluation, customer feedback, browser agents
►
Don't Use Claude, Use This 320B Local Coding AI Instead
The Stack · 2026, iquest-q1, local models, coding agents
Providers and platforms
OpenRouter
OpenRouter docs · Beginner to intermediate
Useful for learning model comparison, latest-family aliases, routing, fallback behavior, agent construction, project-specific evals, and API-compatible experimentation across proprietary and open model families.
Topics
Model routing, Model comparison, Auto Router, Agent SDK, Coding-agent harnesses, GPT models, Claude models, Gemini, Llama, Mistral, DeepSeek, Qwen, API examples, Evaluation
Fireworks AI
Fireworks AI model catalog · Intermediate to advanced
Official material for comparing serverless and dedicated model serving, training specialized models, and measuring quality, token use, cost, and task duration on production-shaped evaluations.
Topics
Ember-1, Kimi K3, Serverless inference, Dedicated deployment, Model training, Model evaluation, Reasoning efficiency
Resources
Harbor
Agent evaluation and optimization harness · Harbor Framework Team · Intermediate to advanced
Running coding and computer-use agents against reproducible, sandboxed task suites at local or cloud scale.
evals, llm evaluation, ai quality, benchmarks and learning resources
Language Model Evaluation Harness
Language-model benchmark runner · EleutherAI · Intermediate to advanced
Reproducible few-shot and zero-shot evaluation of base or instruction-tuned language models on established academic benchmarks.
evals, llm evaluation, ai quality, benchmarks and learning resources
Lighteval
Multi-backend LLM evaluation toolkit · Hugging Face · Intermediate to advanced
Teams evaluating local, distributed, or API-served models across a large multilingual task catalog while retaining sample-level results.
evals, llm evaluation, ai quality, benchmarks and learning resources
SWE-bench
Software-engineering agent benchmark · SWE-bench team · Intermediate to advanced
Measuring whether coding agents can resolve real GitHub issues by producing repository patches that pass executable tests.
evals, llm evaluation, ai quality, benchmarks and learning resources
Terminal-Bench
Terminal-agent benchmark · Harbor Framework and Laude Institute · Intermediate to advanced
Comparing agents on difficult, verifiable coding, systems, security, and scientific work inside sandboxed terminals.
evals, llm evaluation, ai quality, benchmarks and learning resources
Demystifying evals for AI agents
Agent-evaluation learning resource · Anthropic · Intermediate to advanced
Product and engineering teams designing practical evaluations for multi-turn, tool-using agents.
evals, llm evaluation, ai quality, benchmarks and learning resources
LLM Evals: Everything You Need to Know
Application-evaluation learning resource · Hamel Husain and Shreya Shankar · Intermediate to advanced
Practitioners building product-specific evals from traces, domain-expert judgments, and observed failures instead of relying on generic benchmarks.
evals, llm evaluation, ai quality, benchmarks and learning resources
Designing the hf CLI for agents
Agent tooling and evaluation guide · Hugging Face · Intermediate
You want a measured guide to agent-friendly CLI design, including structured output, retry-safe commands, independent grading, and benchmarks showing where higher-level tools reduce calls and token use.
hugging face, coding agents, hf cli, evals, codex
Simon Willison on GPT-6 Astra
Model analysis · Simon Willison · Intermediate to advanced
You want a concise independent read of GPT-6 Astra's launch claims, benchmark caveats, long-context results, pricing, and early comparison with Claude Fable 5.1 and GPT-5.6 Sol.
gpt-6 astra, model selection, benchmarks, coding agents, long context
How to Choose the Best AI Model (Live, in Your Editor)
Model selection guide · OpenRouter · Intermediate
You want a practical six-step workflow for shortlisting models from live data, testing them on your own prompts, measuring cost per completed task, and choosing or routing from inside a coding assistant.
openrouter, model selection, evals, benchmarks, cost
Perplexity Search Evals
GitHub repo · Perplexity · Intermediate
You want Perplexity's open evaluation suites for benchmarking grounded search quality and comparing search API behavior against other retrieval stacks.
perplexity, search evals, benchmarks, grounded answers, evaluation
OpenRouter benchmarks API
API reference · OpenRouter · Intermediate
You want machine-readable benchmark data for coding, intelligence, or agentic tasks before choosing or routing across model families.
openrouter, benchmarks, model selection, coding, agentic tasks
OpenRouter rankings data API
Data API · OpenRouter · Intermediate
You want current usage and rankings data that reflects what developers are actually using, not only benchmark scores or launch-day marketing.
openrouter, rankings, usage data, benchmarks, model selection
OpenRouter Rankings
Live model rankings · OpenRouter · Beginner to intermediate
You want a current usage-and-benchmark signal for routed models before deciding which providers and families deserve a real evaluation run.
openrouter, rankings, model selection, market signals, benchmarks