⇩ Markdown

considering dark software factories→dark software factory -- key media→blog post - software factories and the agentic moment - 2026-02→validation naming is hard→why give evals a different name than just tests?→dim - amount of parallelism -- very high

dim - amount of parallelism -- very high

link not tracked

Backlinks

why give evals a different name than just tests?

In AI, running prompts can be extremely slow, so you typically want to do it with great parallelism so your eval suite will conclude in a reasonable amount of time.
...
dim - amount of parallelism -- very high

see in context

Interesting constraints... primarily constraint - external call time and maybe constraint - cost, which are quite different from the standard constraint - compute time. This means that you can actually run dim - amount of parallelism -- very high potentially within a single thread if using non-blocking IO.

see in context

Although slow, and costly, the work is done by machines on external infrastructure, so you can hit them with extreme parallelism without needing to be concerned about your own system capacity.
...
dim - amount of parallelism -- very high

see in context