video - Code Generation and Maintenance at Scale with Agentic Workflows - 2024-10
https://youtu.be/Ve-akpov78Q?si=HDg6UPXTqBDGafC2
Summary
Large enterprises' codebases are extremely hard to manage because they are huge, interdependent, and often highly varied. They already grow like weeds, but in the near future, they will grow even faster, thanks to AI-enabled development. New tools need to be developed to allow the enterprises' staff who work on cross-cutting concerns, to manage that more abstract complexity better.
At a narrow scope, tools like link not tracked have shown how AI can aid developers in quickly adding features, but what about when you need something more like a bulldozer, for huge tasks like changing your logging platform, enterprise-wide, across a thousand repos? For link not tracked in link not tracked
This video explores one solution to that problem, a product called link not tracked, and goes into the technical details about how it uses massively concurrent forking agentic workflows to solve the problem. I found the section at 14:00 about the use of link not tracked to do fast, exploratory link not tracked (which feels a little like link not tracked) to be particularly intriguing.
Details
agent link not tracked codified agentic workflow
At 1:57 link not tracked the product, link not tracked, is aimed at link not tracked instead of link not tracked. Basically, it is targeted at top developers to be a very large-scale code bulldozer (vs. helping those who are less familiar with coding be able to make things they would not have been able to). X Kinda funny because link not tracked but that's essentially what you'll get with these super-high-leverage system-wide tools.
...
link not tracked
At 3:20 once more developers start using AI (and the AI becomes more powerful, too) there's gonna be way more code, and we're gonna need tools for managing just tons of code. Rather than single lines, you're gonna be making huge changes across many files. X link not tracked link not tracked link not tracked
At 8:00 when searching to replace the logging for a whole enterprise across thousands of repos, they use their own special SQL that does both static analysis in concert with using LLMs . X Roughly makes me think of link not tracked techniques more generally.
At 10:50 the idea that IDEs are already giving us superhuman tools with link not tracked, link not tracked, etc, and that we need to ensure that we give our LLMs the same tools X link not tracked
At 12:50 the iteration time for checking a potential solution is comprised on coming up with a code change in the LLM, then checking the compilation. In big enterprises, the build time for link not tracked can be multi-minute, which actually dwarfs the LLM time and kills the iteration time. You need to make sure that you are using the same language server tools that you would as a person using the IDE... where they do incremental change detection and you get feedback almost immediately instead of relying on your CLI tools that are part of CI (since they are so slow).
...
link not tracked
...
What you want is link not tracked for your link not tracked
At 14:00 he talks about the problem of compounding errors and how you can get in a bad state that you can't recover from. The solve is to use link not tracked to be able to snapshot the memory of a process at a known good state and then try 10 hypothetical changes, then you rely on things like link not trackeds, etc. to determine which ones are the best and you establish a link not tracked to decide what should become the new base to fork from x link not tracked link not tracked
...
This also reminds be of link not tracked approaches to reasoning using the link not tracked methods. Basically link not tracked link not tracked then winnowing things down by link not tracked. The difference here is that these seem to be short-term hypotheses and link not tracked with the aim of getting to some new link not tracked link not tracked, then link not tracked off of that to go further. That feels a little like link not tracked. This is all to avoid the compounding error (AKA error accumulation), which is particularly important because of dim - reliability -- low and a lack of revisiting link not tracked. LLMs doing link not tracked link not tracked tend to have link not tracked and link not tracked, which causes the errors to get worse over time.
At 15:50 he talks about how it can get really expensive to do so many forks to your code and edit the whole file in each of the forks because link not tracked are more expensive than input tokens. Because it's auto regressive. Also the output limits are not growing as fast as the context windows, which have become very large.
At 17:00 he describes the challenges with applying changes. Full file works well, but is very expensive, LLMs are not very good at generating link not trackeds with good line numbers, etc. so they invented a loose search and replace. X link not tracked