⇩ Markdown

considering dark software factories→dark software factory -- key media→blog post - OpenAI harness engineering 2026-02→review - Open AI harness blog post - 2026-02-22→blog post - the three pillars of AI observability - 2025-11→evals should be easy and cheap to run

evals should be easy and cheap to run

evals should be easy and cheap to run

Easy so your evals persona -- domain expert can do it. Cheap so it happens often

Backlinks

blog post - the three pillars of AI observability - 2025-11
If you're building AI that real customers rely on, you need to:
1. Trace everything. you can't improve what you can't measure, and you can't measure what you can't see.
2. Run evals constantly. Both online (to catch regressions) and offline (to test improvements) x evals should be easy and cheap to run
3. Build annotation into your workflow. The best AI systems improve over time by learning from expert feedback.
see in context