This video explores one solution to that problem, a product called grit, and goes into the technical details about how it uses massively concurrent forking agentic workflows to solve the problem. I found the section at 14:00 about the use of firecracker to do fast, exploratory hypothesis testing (which feels a little like beam search) to be particularly intriguing.