If you're building AI that real customers rely on, you need to:
- Trace everything. you can't improve what you can't measure, and you can't measure what you can't see.
- Run evals constantly. Both online (to catch regressions) and offline (to test improvements) x evals should be easy and cheap to run
- Build annotation into your workflow. The best AI systems improve over time by learning from expert feedback.