dim - model performance

Backlinks

blog post - the three pillars of AI observability - 2025-11

evals: Quantify performance, both in production (online) and in dev and CI (offline), to understand how well your application works and systematically improve it
...
dim - model performance (but not really model performance... more like compound system performance)

see in context

In traditional observability, when you notice something is wrong, the next action to take is almost always to update code and try again. However, in AI, incorrect behavior often requires input from an expert (product manager, subject-matter expert, or even a user) who can clarify the behavior. The best workflows for annotation involve curating interesting examples that would benefit from annotation, flagging them for review, and then utilizing the annotated data in evals to improve performance.
...
annotation evals persona -- domain expert curating interesting examples
...
utilizing annotations in evals to improve dim - model performance

see in context
dim - model performance -- high

dim - model performance

see in context
dim - model performance -- smart enough

^ dim - model performance -- smart enough

see in context