⇩ Markdown

considering dark software factories→dark software factory -- key media→blog post - software factories and the agentic moment - 2026-02→validation naming is hard→why give evals a different name than just tests?→the LLM model in LLM as a judge is often smaller than the task model

the LLM model in LLM as a judge is often smaller than the task model

often you will use a link not tracked to check the output of the bigger one
...
LLM as a judge link not tracked

Backlinks

tweet - llm as a judge can be crazy expensive

LLM as a judge dim - costs -- high link not tracked so the LLM model in LLM as a judge is often smaller than the task model

see in context
why give evals a different name than just tests?

If you do use LLM as a judge you need to do fitness to task optimization for the judge as well. For example the LLM model in LLM as a judge is often smaller than the task model because tweet - llm as a judge can be crazy expensive, otherwise. So, you need to experiment to see just how small that evaluation model can be and still get evals you can trust. In traditional testing, you primarily test the code.

see in context