AI-Friendly Code: The Missing Link for Speed with Quality • Adam Tornhill • GOTO Copenhagen 2026
Coding agents dramatically increase developer speed but degrade code health and skyrocket defects unless quality rules and MCP safeguards are enforced.
Highlights
5 clips · 15 min of 45 min-
- Controlled studies show a 41% increase in software defects after adopting AI coding tools.
- Open source projects experienced an initial 200% velocity boost that was fully canceled out within two months due to rising code complexity.
- AI frequently operates in a self-harm mode, generating code that it cannot safely modify later.
-
- Testing coding agents on competitive programming datasets reveals a steep drop in reliability as code health decreases.
- While a code health score of 9 is considered acceptable for humans, an AI's defect risk jumps by nearly 60% at that level.
- AI agents require a code health of 9.5 to 10 to keep defect risks properly in check.
-
- Packaging code health metrics into an MCP server allows Claude to review and score its own generated code automatically.
- When health issues are detected, the MCP server provides specific feedback, triggering an automated self-correcting refactoring loop.
- Developers only review the final, healthy code output, bypassing intermediate messy iterations.
-
- Uplifting Street Fighter 3 (300,000 lines of 25-year-old C code) to a perfect 10.0 code health took less than a week using Claude Opus.
- Manual refactoring of this scale would have taken 12 to 18 months with a specialized senior team.
- Functional correctness during refactoring was successfully maintained using visual trace replay harnesses and play tests.
-
- During large-scale refactoring, the AI automatically compiled a playbook of narrow, codebase-specific patterns and constraints.
- These narrow receipts successfully bridge local codebase problems to general LLM design and refactoring knowledge.
- With specific receipts, agentic refactoring becomes a predictable and mechanical process at scale.
That’s all the highlights.
Questions this video answers
11 · every answer cites the moment it was said01 Why do initial AI velocity gains disappear after a couple of months? 2 sources · 5:31
Initial velocity spikes can vanish because coding agents frequently increase code complexity and operate in a self-harm mode. This creates code that the agent cannot safely modify later, resulting in projects ultimately slowing down.
02 How does code health impact task completion speed and defect risk? 2 sources · 9:30
Working in healthy code enables developers to complete tasks up to 10x faster than working in unhealthy code. Additionally, healthy code drastically reduces defect risks by up to 15x, making changes much safer and easier.
03 Are coding agents more sensitive to code quality than human developers? 2 sources · 12:02
Yes, research shows that AI coding agents have a much lower tolerance for poor code health than humans. While a code health of 9 is generally acceptable for humans, an AI's defect risk increases by nearly 60% at that level, requiring a score closer to 10.
04 What kinds of quality problems do coding agents typically introduce? 2 sources · 14:18
Coding agents frequently write code lacking proper modularity by packing too many responsibilities into a single function. They also introduce deep nested logic and poor error handling, such as catch-all clauses that silently swallow exceptions and cause resource leaks.
05 How do MCP servers safeguard AI-generated code? 1 source · 19:30
Packaging the code health metric into an MCP server allows the agent to review its generated code automatically. The AI enters a self-correcting refactoring loop based on MCP feedback, resolving code smells before a human ever reviews it.
06 What is the typical code health score of an average codebase? 1 source · 22:22
While optimal AI-friendly code requires a score of at least 9.5 or 10, the average legacy codebase sits at a meager 5.15. This vast gap means letting unstructured agents loose will inevitably cause spikes in bugs and incidents.
07 How does adding MCP data improve agentic refactoring? 2 sources · 24:34
Without guidance, AI models become extremely conservative and only fix about 20% of code health issues, mostly doing minor renames. When provided with MCP code health metrics, the AI can form a concrete plan to address specific issues and successfully clean up technical debt.
08 How long does it take to uplift a large legacy codebase using agents? 2 sources · 29:43
After spending about two weeks setting up processes and automated test harnesses, the actual agentic uplift of a 300,000-line C codebase took less than a week. This process raised every single file from legacy status to a perfect 10.0 code health score.
09 How do codebase-specific refactoring receipts bridge local problems to general design patterns? 2 sources · 31:29
As the AI refactors, it documents successful transformations along with specific constraints into a playbook. These narrow, codebase-specific receipts adapt to local problems while connecting them to broader design patterns encoded in the LLM.
10 What is the financial impact of code health on token consumption? 1 source · 36:17
Working with healthy code significantly reduces API token costs when building features with AI. On average, implementing features in healthy code requires 40% to 45% fewer tokens compared to unhealthy code.
11 Why do architectural god classes lead to agentic merge conflicts? 2 sources · 42:06
God classes compulsively collect responsibilities and span thousands of lines, forcing agents working on distinct features to modify the exact same files. This accidental complexity acts as a traffic jam and directly causes enterprise-scale merge conflicts.
Section-by-section summary
9 parts
AI coding speeds up work but risks quality
While AI coding provides a 2x to 3x speedup and helps unblock simple tasks like GitHub actions, it also introduces a significant rise in defects and accidental code complexity that can stall projects within months.
AI code quality and the illusion of instant software
Studies on open-source projects show that initial velocity gains from AI are often canceled out over time due to accumulating complexity. Building the first prototype is easy, but 95% of software costs occur during maintenance where code quality matters most.
Code health is an objective metric for quality
Code health measures readability and structure on a 10-point scale, heavily correlating with both human task completion speed and defect risk. Healthy code allows developers to work up to 10x faster and reduces bugs drastically.
Coding agents are extremely sensitive to code health
Research on competitive programming datasets demonstrates that AI agents break tests much more frequently on unhealthy code. While a score of 9 satisfies humans, agents require a code health of 9.5 or higher to operate reliably.
Safeguarding AI generation with MCP feedback loops
Developers can integrate code health rules into an MCP server that evaluates AI-generated code instantly. The agent automatically enters a self-correcting refactoring loop until the code meets high quality standards before human review.
Legacy codebases are a bottleneck for enterprise AI adoption
With the average codebase scoring around 5.15 on code health, legacy technical debt prevents coding agents from operating safely. Enterprises must uplift their codebases to fully leverage agentic coding capabilities.
Uplifting large legacy codebases agentically in days
Using Claude Opus guided by code health MCP data, a team successfully refactored a 300,000-line C codebase from legacy status to a perfect 10.0 code health in just a few days. Functional correctness was preserved via screenshot trace hashes and play testing.
Codebase-specific refactoring playbooks and token savings
During large-scale refactoring, AI generates specific patterns and receipts that map local problems to general design knowledge. Furthermore, healthy code reduces token consumption by 40 to 45%, making agentic workflows significantly cheaper.
Architectural god classes cause agentic merge conflicts
High author density and recurring merge conflicts in AI workflows are socio-technical symptoms of monolithic god classes. Modularizing and refactoring at the architectural level is the next frontier for scaling agentic engineering.
n next · p previous clip · t highlights/full
< > slower/faster · / search what was said
-
Behavioral code analysis2× · first 1:03
An analysis technique used to track stable software fundamentals and identify coordination clashes.
-
Code health16× · first 6:30
An objective numeric metric measuring code complexity and quality on a scale from 1 to 10.
-
MCP server5× · first 19:32
A server implementation packaging code health rules to provide real-time feedback loops for coding agents.
-
Illusion of instant software2× · first 16:39
The false impression that building a prototype easily is the hardest part, ignoring long-term maintenance costs.
-
AI whiplash2× · first 18:14
A reporting trend showing that increased coding speed often leads to higher bug counts and production incidents.
-
Refactoring playbook2× · first 28:16
A document built up by an AI during refactoring that details successful receipts and constraints.
-
Action parameter4× · first 31:47
An AI-innovated refactoring pattern used to deduplicate similar coordinate transformations using function pointers.
-
God class2× · first 42:01
A massive module that compulsively collects responsibilities and forces multiple developers or agents into merge conflicts.
7,361 words · about 29 min to read
AI coding speeds up work but risks quality
Thanks for showing up to this session on AI friendly code. And there's a reason why I would like to give this presentation and that reason is because software development is now at a point where it seems to change more or less at a weekly basis. Right? At least that seems to be the common sentiment. My claim is slightly different because yes, some aspects of our work do change frequently. It might even be on a weekly basis, but the fundamentals they remain surprisingly stable. And those fundamentals of software, they remain stable not only for weeks or months, they remain stable over cloud releases too, right? They might remain stay with us for decades.
And this talk is an attempt at capturing those stable aspects. And for that I'm going to return to something I call behavioral code analysis. It's a concept I've written a lot about in my book. And um I'm going to see how behavioral code analysis can help us make better agentic coding decisions. But before we get there, let me return to the present. How is coding different with AI?
You see the timing is quite good because next year I'm celebrating on anniversary. Next year I've been a programmer for 40 years. But last year, September 2025, I wrote my last line of code by hand. And I've been doing everything aentic ever since. And I probably produced more software over the past year than I did in the decades prior. And one thing I learned that I'm going to talk about today is that developing software with agents is harder. It's significantly harder than writing code manually.
And of course, a statement like that begs the question, if something is so much harder, why bother at all, right? Why should we do something the hard way? what's the return on investment with going aentic and it happens that for years my team we have been tracking our task completion times and I did look into that and what I noticed after just a few months of going Aentic is that we had a noticeable speed up right we were able to complete task two to 3x faster now I think that speed alone kind of misses the real benefits, right?
It's of course a big win for the business to be able to do things two or three times faster, but that's not where the real value lies for me. Rather, the value is in what that speed enables. Because with increased speed, we can do things. We can take on tasks that just weren't possible before. We can do some absolutely crazy things. And I'm going to show you later in this presentation what crazy means to me. But there are also smaller wins. One thing I like in particular with AI is that it helps us unblock ourselves, right? It reduces coordination needs. And to give you one example, despite being a programmer for almost 40 years, there are some things I just never learned properly. One of those things it's it's almost embarrassingly to admit admit, right? But I'm amongst friends, I think. So one thing I never learned to do properly is GitHub actions. Right? For some reason, my brain just refused to grasp that
concept. And uh that always meant that the moment I had to do something with GitHub action, I had to interrupt one of my colleagues. They had to context switch. They had to go and help me out. Now with agents, I can just unblock myself as long as it's some fairly simple things. My agent can do whatever I need to do in a GitHub action, right? And I don't have to disturb my colleagues and I can remain in the flow. And that's something I really really like.
That said, there are also some challenges with agents and one of the most obvious challenges
AI code quality and the illusion of instant software
relates to quality. And I'm talking about external quality now, right? What our customers and users experience. And I'm going to show you a bunch of uh research here. And this is the type of research that you can no longer do. You could do it in 2025 before all developers shifted to AI. These days, everyone is already using
1 Highlight · AI coding tools increase defects and complexity
AI. you can no longer do these type of controlled experiments. But this is a a finding from late 2025 where they measured what's the impact when developers get access to AI coding tools, right? And this is measured on real production data. And what you see here is that there is an increase with in defects, right? 41% more defects after adopting AI. That's probably not good, is it? No, it's a disaster, of course. But what about speed, right? That's one of the benefits.
This is a study I really, really like. This is a really cool study. [snorts] So, this is a study done on more than 1,000 open source projects where they were measuring what's the impact once they adopt AI. And what they found in that research was that during the first few weeks, there was a massive velocity boost, right? So, velocity was up 200%. Which pretty much matches my own numbers.
The interesting thing is that fast forward a couple of weeks and two months later all those initial promising velocity gains were fully canceled out and then some. So these projects were all slower after two months with the eye than they were preai. And the reason for that the research is very clear on is a massive increase in code complexity.
So this is why I have this claim that AI to me seems to operate in some kind of self harm mode very often writing code that it just cannot safely modify itself. So a big big part of succeeding with
Code health is an objective metric for quality
aentic coding lies in managing this accidental code complexity and that's what I want to talk about next. And for doing so, we need to have some kind of measure of code complexity, some kind of measure, objective metric for code quality. And the metric that I'm going to run with is something called code health. And I'm not talking about the concept. I'm talking about the specific metric. And you will get some additional information on this soon. But instead of going through a bunch of rules on what uh code health is, I wanted to try to show you by example.
So, please have a look at this code here. Hands up if you can read the code. Ah, a lot of you. And those in in the in the back who cannot see it, don't worry. You're not missing out on anything. I think we can all agree that this is a horrible mess. I've been programming for so many years and I can can't even start to guess what this code does. I cannot even guess at the domain, right? It's just so complex.
and you see all these uh nested ifs and all this excess logic. It clearly seems to lack structure and intent. However, the nice thing with code, what I really like about code is that there's an infinite number of ways that we can express the same problem. And this is good for us because it means we can take code like that and we can refactor it to make it more readable. And this is an exercise I've been doing. So, first I would say that this is unhealthy code. I think we can all agree on that, right?
But after refactoring, if you look at the code to your right, you see that the domain just pops out. Right? This is an agentic loop. Right? So this is code that I like to claim is so much easier to understand. And since it's so much easier to understand, it's also easier to navigate, not only for us as humans, but also for an agent. Because pretend that we're prompting agent that we want to change how feedback is evaluated in this code. If you look to the right, you see there's one obvious place to look for that, right? Feedback evaluate, right? And that helps an agent find the right place to modify. And it's also going to contain the blast radius of that change, making it both safer and easier to validate.
So this is what healthy code looks like. So please keep this contrast in the back of your head as I go through what this means in practice. Now the codoff metric is interesting in the sense that it predates generative AI and we have been doing a bunch of research on it and studying how is code health impacting us as developers and this is from a paper called code red the business impact of code quality and what you see here in the graph is the three different categories of code health right so you can have a healthy code green code that's where you want to be with the majority of your code and then increasing levels of complex code technical that now on the y-axis you see the average task completion time and this is a study that kind of control for task complexity control for codebase size and all of that and what we see here is quite dramatical we see that if you have healthy code you can be up to 10x faster than someone that works on
similar problems in unhealthy code right so there's a big big benefit in terms of speed if you have healthy code. What's also increasingly relevant in the AI space is that healthy code reduces the defect risk with up to 15x. It's simply safer and easier to change. Now, as I said, this data predates the
Coding agents are extremely sensitive to code health
gentic age. So, how is code health relevant to an AI? Does an AI care about code quality? I kind of alluded to that by showing those studies, right? But we wanted to figure out in our research team, how important is code health to a coding agent. And the kind of idea we started out with, our hypothesis was that machines will get confused by the same patterns as we humans, right? Because after all, an AI is trained on largely humanly written code.
So our paper is called code for machines. And the way we did this study was that we used an open data set which I think is really really important in the this day and age because that means anyone can reproduce our findings right and they can try different versions and metrics. This data set was from competitive programming and the reason we choose that was because in competitive programming you have a very strong suit of automated tests which meant we can give an agent a task and then we run the tests. If the test pass, the agent did its job. If the test fails, the agent has broken the program. They have introduced a bug.
So, let's jump to the conclusions. Is code quality important for an AI? And a
2 Highlight · AI agents have much higher defect rates on unhealthy code
quick spoiler, yes. Otherwise, this would be a very short presentation. What surprised me though was how strong the relationship is. So, please have a look at the graph here. On the x-axis you see the code health scale and you see that under the hood it's not just three categories it's much more granular right it's a numeric scale going from 10.0 zero all the way down to one. On the y-axis, you have what we call break rate. So that is how frequently did an AI break the automated tests, right?
Introduced a bug. And as you see here, what I find so fascinating is that the code health scale when we originally built it and calibrated it, we wanted a code health of nine to indicate humanly readable code, right? Code that we humans can, you know, comfortably work with. But if we look at the break rate, we see that at a code of nine, which is a bar that's good enough for humans, we notice that an AI has already its defect risk has already increased by almost 60%.
Right? So what that means is for an AI, we need to push the quality bar even further to the right. We need to aim for a code health of at least 9.5, maybe even a perfect 10, right? To keep the defect risk in check. And that is something that surprised me that an AI seems to be even more sensitive and picky about code quality than we humans now. I will get back to what that means for us in practice. But before we do that, I think I need to address this silicon silicon elephant in the room.
Right? Because I mean what I'm telling is that really relevant and AI will write perfect code for sure, right? So I can just point my cloud instance to my legacy codebase and tell cloud to rewrite it, right? Wouldn't that work? Well, let's have a look at that. So please have a look at the following beauty.
Again, text might be small, but you can all probably see the shape of it. This is a piece of C++ code, and the details aren't important. Please focus on the overall structure here. So what I did now was that I took an existing legacy codebase and I asked my agent to make an extension to it, add a new feature. I told it that I wanted it to respond to each keyboard press and report the keyboard presses as statistics to Amplitude. And for those who haven't used it, Amplitude is just a product analytics tool in the cloud. Now the AI solved the problem. The code even seemed to work when I did my first test drive. But what I want to do now is to comment on this and critique it from a code perspective. So that way we we see what kind of problems on AI tend to introduce and we can also get a feeling for what's actually included in this code scoring. So the first thing that really really stands out to me is that there's not a lot of code here. Despite that the AI has squeezed a lot of responsibility into a single function,
right? So this is code that clearly lacks modularity that does way too many things and those responsibilities aren't clearly captured in the structure of the code. What's potentially even worse is this shape right? You see all that deep nested logic and that's something that really worries me because there's some very good research indicating that deep nested logic is responsible for roughly 20% of all programming mistakes. So they're very likely to impact on AI negatively too.
But perhaps the worst problem is the error handling or should I say the lack thereof because C++ is a language with exceptions. The moment anything here goes wrong and the code throws an exception, this code is going to leak resources all over the place. And adding insult to aentic injury, the AI was also friendly enough to introduce a catchall clause, right, that silently swallows any exception. So where does that leaves us? Well, when something goes wrong, it will go horribly wrong and we won't know why. Not exactly a prize-winning design.
And of course, if this was a C++ workshop, I would have gone and I would nitpick about magic numbers and excess uh null pointer references and whatnot. But I'm not going to go there. I'm going to leave the AI code as is. But my verdict is that an AI frequently generates unhealthy code. And it's the type of code of that quality that the AI itself won't be able to safely iterate on later. So it's a self- enforcing def loop.
Now some of you might be surprised to hear this because if your LinkedIn feeds, if your X feeds are anything like mine, they are filled with stories of people building amazing things with AI, right? So, you've all been there on a Monday morning, picked up LinkedIn and you see a story about someone building a Linux clone in Visual Basic, right? Chatting to Cloud Code over voice while taking a bath. No prior coding experience. Seen that? Yeah, you know what I mean, right?
And it's true. People do amazing things. But I think the big problem here is something I call the illusion of instant software. Because you see building the first version of something, building the first prototype, that was never ever the hard problem in software. We know how to do that. We knew how to do that. The problem is that 95% of the costs for any software project comes after the first release.
And this 95 part, that's where code quality maintainability architecture, all that stuff starts to matter. And this is this illusion of instant software has been shown at work and demonstrated in a new line of research. And this is one of my absolute favorite research papers over the past year. It's a paper from the Alibaba research team and it's really cool in what that what they do is that they focus on the maintenability aspect of software. So they simulate a real software maintenance over six months time. And what they found in that paper is that coding agents work really really well for the first isolated tasks, but they quickly break down over time just as predicted. And they have this amazing quote which I think captures the essence of it all that the consequences of past decisions accumulate.
And this is no longer just some hypothetical academic argument, right? there is production data coming in indicating that this is a very real problem with quality in our industry. So this is something called the AI whiplash, the AI engineering report that came out just a couple of months ago this summer. And what we see here is that bugs and this is by the way this is based on telemetry. So this is real data from real production systems, right? And we see that bugs are up 54%.
Incidents up 242%. So yes, we did see this 3x speed up in production of incidents and that might not be what we intended, right? So no on AI won't generate perfect code and if we want to survive in the aentic
Safeguarding AI generation with MCP feedback loops
age we need to do things slightly differently. We need to focus on fundamentals and to me that means safeguarding. So when we work with aentic coding we safeguard all AI generated and all AI edited code with respect to its health and I want to show you how that can look
3 Highlight · Safeguarding AI code using an MCP code health server
in practice. So please uh ignore the graph for now and focus on the right hand side where you have my claude window. So what I'm going to do now is that I'm going to ask claude to generate some code for me. What I have also done is that that we took our code health metric that we used for research and we packaged it in an MCP server that we added to claude and that means that each time cla generates some code or modifies it, it does a code health review and is instructed to act upon the findings. So let's see how that looks in practice. So look to the right here. I'm asking Claude to do some payment validation functionality and AI happily complies. Of course, first version, fantastic, right? Really low code health, poor feedback. So, Claude acts upon that data, tries to improve, comes back with a slightly better version and it's still not perfect. So, it gets a third attempt and now it nails it. Right?
The nice thing with this is that I as a human I never have to see these intermediate steps, right? The MCP server gives feedback to the AI and AI acts on that and LLM is pretty good at working towards a defined target. So now when it gets to human validation or even human review, I have healthy code that I can start from. So that's a big big benefit. So the process we try to model is what you see here to your left. So if you start on the top you see that each time you ask your agent to generate some code it writes the code and then it calls out to the MCP server and checks is this code healthy and if it isn't MCP server will respond with the list of the code health issues that were introduced various code smells like we saw before lack of modularity deep nesting whatnot and the AI is instructed to go into a self-correcting refactoring loop right where it tries to clean up the mess it just introduced used and then it checks again does a new code health review and if the code is now healthy then we have
safeguarded the code and we can continue to iterate on the feature or build the next thing. So just to sum it up by having this safeguarding in the agentic
loop we can guarantee that all code we write is healthy and remains so. But of course safeguarding new and change code is just the first step to apply AI at scale. We also need to consider what's already there. [clears throat and cough]
Legacy codebases are a bottleneck for enterprise AI adoption
We need to ensure that our existing business critical code is AI friendly. And that begs the question, how healthy is the average code base? Well, I happen to have some numbers on that. So, just to remind you, optimal code 10.0 zero AF friendly code at least 9.5 ideally higher because below that the defect risk climbs steeply.
So what's your guess? Please give me a number. How healthy on a 10-point scale is the average codebase?
Six. That's an optimist. I love that. The average codebase has a code of 5.15, right? That's pretty far from AI friendly. So what's going to happen when you let your agents room freely in that codebase and try to implement new features? You're going to see the bug numbers and incidence numbers that I showed you before from the AI whiplash report.
So to me, my interpretation of this gap between reality and where we need to be is that legacy code and technical depth will be a bottleneck for enterprise adoption of agentic coding. And the consequence is that a lot of companies will have to uplift their code bases in order to benefit from AI to be able to truly accelerate while keeping quality in check. And the good news are that that uplift can and should be agentic.
Uplifting large legacy codebases agentically in days
So that's what I wanted to talk about next because again it's if we want to do an agentic uplift agentic refactoring at scale it's based on the assumption that an AI can indeed refactor code. Now it happens that we also did some research on this. So this is a paper that we published just a couple of months ago and what we did here was that we asked AI to improve the code health of code of in various categories right so you have various bins of code health and uh we tried all the the new models right so what I'm going to show you now is the it's not the latest uh claude opus it's the previous uh previous one that we had a couple of months ago. And on the y-axis, you see the number of code health issues that the AI managed to fix on its own when instructed to do so.
And we see here that in a best case scenario, an agent can fix roughly 50% of the code health issues. And that's true for code of lower code health. And I think that's simply because there are so many obvious lowhanging fruits, right? Like it's never a good idea to have a function that's 600 lines of code, right? So the AI understands it can motoriize that but once the code health improves a little bit the AI performance drops sharply right and in many situations cloud can only correctly refactor 20% of the code and if you look under the hood then you will see that what the AI actually does is that it becomes super conservative. So it tries to do small small changes like renaming a variable, right? Things that don't really fundamentally move the needle on code quality.
So this didn't look so promising. But then we tried an a variation on this experiment. We rerun exactly the same uh study, the same experimental design with one modification. We dropped in the MCP with code health data and look what happened. And this is not so surprising because the moment AI gets a direction, the moment it gets a goal and it gets a description of what's preventing you from reaching that goal, the AI can form a plan and start to address these findings, right? And measure its way towards healthy code.
So when I saw this, I thought, okay, this is really, really interesting. Can we apply this at scale?
4 Highlight · Agentic refactoring of a 300k-line legacy C codebase
And our research team recently did a case study around this. So this is brand new to me as well. This is something that we wrapped up just last week. So this is the first time I talk about that. Now what we wanted to do was to find a legacy codebase that we could uh uplift. And the goal here was to take a legacy codebase with lots of complicated code and make it healthy enough that we could add new features via coding agents.
Right? So we wanted to get to a code health of 10.0 zero and we wanted to do so automatically and at scale and the case study we choose was Street Fighter 3 the game because Street Fighter 3 it's a it's a domain that I think everyone can understand right and it's also an old codebase it's 25 years old it's written in C and it consists of 300,000 lines of code so it's large enough to be interesting and if I look at the Code health. Here we see a code health visualization of the Street Fighter codebase, right?
Where each bubble indicates a file with C code and the colors indicate the code health category. And you see that it seems quite challenging, right? This is code that's far from a friendly. So how can we uplift this with agents at scale? The process we used was first of all to set quite an ambitious goal. We want to wanted to eradicate all technical depth in the application code.
We wanted to get to code off of 10.0. We also wanted to maintain functional correctness, right? Because otherwise you get a game that is most likely not particularly enjoyable to play, right? Imagine that you hit someone and that hit is not being registered. You're not being awarded points or damage for it. So the way we did this was uh that my colleague used a replay uh trace harness, right? So after doing a refactoring uh you simply took uh screenshots, frames and calculated a hash of that and made sure that they were identical to what was seen before the refactoring.
And of course my colleagues also did a bunch of play tests, right? So each time we uh wrapped up an iteration, they played the actual game, right? Built it and played it. And then we used the code health as the objective quality signal and the direction for the AI. But something that turned out to be quite critical to make this work at scale was this final bullet point that the AI was instructed to build up a refactoring playbook as it went along.
So the moment AI found a refactoring that was successful, it was instructed to document the conditions when it applied the constraints and actual mechanism. Right? And the idea here originally was that by doing so, refactoring should become progressively more effective as the AI is capable of falling back on existing knowledge that it builds up itself. I'm going to talk a bit more about that refactoring book because to me I found some really surprising things in it. But before that, how long did we need to do this? Well, it wasn't a full-time job. I had two of my colleagues were working on this. This was not a full-time task for them. It was something they did on the side. And we spent roughly two weeks trying to define their process and you know get automated tests to run and all that stuff. And we also experimented with various LLMs. So the different colors shows which uh large language models that we used as the backbone. And
early on we tried with the various GPT models. And the GPT models are good at refactoring, right? They can do a refactoring. But we found that they were lacking they were lack the capability of documenting the refactorings properly. Right? So they didn't really contribute to this playbook that we wanted. So we ended up settling on Claude Opus 5.5 uh because it did a much better job at capturing the playbook as well. And then after two weeks the actual uplift of 300,000 lines of uh legacy C code took a couple of days. So less than a week.
And I'm going to show you where we ended up. So let's first look at the code quality aspect. So this is where we started out. Remember each bubble is a file. The color shows the code. After one week of this agentic refactoring, we had a code base that was a perfect 10, meaning every single file in here was 10.0. But and this I mean if someone would have told me this a couple of years ago I wouldn't have believed it because I've been working on you know legacy rewrite projects large scale refactorings project like that in the past and I would have guessed that doing this work manually would have required 12 to 18 months it would be a high-risk project with a large likelihood of failure and it would have required a team of five six sea level experts that's also knew
the domain really really well and now we could pull off with agents within a couple of days. That's absolutely crazy to me. But what's even more crazy is what we found under under the hood. So this refactoring playbook once the AI has had finished the task, we took a look at it
Codebase-specific refactoring playbooks and token savings
5 Highlight · AI-generated codebase-specific refactoring playbooks
and the playbook contains some you know the first parts of it were quite familiar. These were like uh generic refactorings, right? stuff like extract function, introduce god clauses and so on, right? Extract class stuff that we all know about general refactoring knowledge. But once I got a little bit deeper into reading that playbook, I found something surprising.
I found that the AI had the scarabboard certain refactoring receipts and it had not only named them, it also has also up uh documented very specific constraints and context on where these patterns apply. And you see some examples here, shared index range, action parameter and so on. And this really really got my attention because they were so detailed. So I wanted to understand okay what is that and how does it help us? Is there a more general lesson buried in this playbook.
So let me give you an example and make it a bit more concrete. Let's look at the action parameter pattern that the AI innovated. So the action parameter it's a pattern for dduplicating code. And I want to show you an example. So you get to see a lot of really cool code today, aren't you? Right. Yeah. So, h and I know that it's really hard to see in the back of the room. So, I'm going to walk you through it. So, here are two functions.
If you look at them, you will see that there it's vertical. It's virtually exactly the same process being modeled twice. The only difference is that one of them is calculating on X offset, you know, the X coordinate space, and the other one is doing exactly the same transformation in the Y-coordinate space. So I like to claim that this is a deplication of knowledge and the AI figured out that as well. So the code health issue is the application and the solution was the action parameter.
So what is the action parameter? Well, what the AI did was this. So this is the refactoring. So the code at the top is a new function and that new function encapsulates the commonalities, the stuff that's common between these other two functions. And the two originally duplicated functions now turn into oneliners and the only thing they do is to dispatch to that internal representation and then they pass along the concept that varies right and that's the action parameter. So action parameter is like a like it's a function pointer. It's a method reference if you like.
And when I saw this I thought okay this is pretty cool. Uh but isn't it just the strategy design pattern done with C function pointers [snorts] and I like to say that well yes and no. The mechanism is certainly strategy or refactor into strategy. But the innovation here driven by the AI is something very different because if I would give an AI, I would attempt to do refactoring at scale and I would give it a broadex instruction to apply strategy to dduplicate code, it just wouldn't fly, right? It would be a complete mess.
What's interesting about these codebase specific refactorings is that they are so specific and narrow and the the benefit of that is that being so narrow means that these codebase specific refactoring receipts can serve as a bridge from specific problems that might be local to just your codebase. They can take that specific problem and bridge it to more general knowledge encoded in the LLM.
So the way I think about it is that generic refactorings and P patterns they offer the foundations they offer the actual code transformations but the codebase specific refactoring receipts they adapt to the local problems and at hand right and this makes it possible to mechanically map specific problems in your codebase to generic design patterns and refactorings and that is the mechanism that allows agents to recognize and reuse those receipts at scale.
So at the end we found that agentic refactoring turned out to be a predictable and mechanical process at scale and a large part of that was
thanks to this innovation that AI did of codebase specific refactorings. Now I'm going to mention a little bit about the engineering outcomes because it's one thing to have a codebase of 10.0 but I don't want that to just be a vanity metric. It has to carry some meaning to the business as well. And to do this, I need to introduce another piece of data for you. So, in doing all this research, we have collected a lot of statistics on token spend. And when we start to compare, you know, what it costs in terms of tokens to implement a feature in healthy code versus unhealthy code, we see quite an interesting uh distinction.
So, it turns out that healthy code is also financially viable code. The actual numbers differ between the different programming languages, but on average, if you have healthy code, you can usually do things with 40 to 45% fewer tokens, right? So healthy code is token optimized. And this meant that we can start to reason about the engineering outcomes for our large scale uplift of 300,000 lines of code. So the outcome here is not only that you have a better looking codebase. The actual outcome is that there's now a 17% reduction in AI induced defects. Right? So when you continue to develop this codebase, the risk is much much lower. This is AR friendly. And we can also see in more concrete terms that whatever new features we add to it are going to be 45% cheaper in terms of token cost. And how much did all of this cost us in total to do this uplift?
Well, it cost the API token cost is roughly $4,000 US for doing this complete uplift. So once we have that number, we can make
Architectural god classes cause agentic merge conflicts
a very concrete business case. Right, I'm going to wrap up but talk a little bit about the the next steps what lies beyond the code. And I started out by saying that agentic workflows are reinforcing the fundamentals of software engineering and uh that includes behavioral code analysis. And I think nowhere is that as obvious as when we talk about merge conflicts and code reviews, right?
So a couple of months ago I was visiting a bunch of uh presentations in New York about agentic coding and all technica presentations came back to the same thing that when coding accelerates they keep seeing recurring merge conflicts right at enterprise scale and a lot of the solutions were were quite creative right because the response to on increasing merge conflicts seems to be to double down on better PR tools so that might be stacked PRs, multiQ systems for PRs, intelligent conflict resolution and whatnot. And I I think it's it's all interesting, right? I mean, who doesn't love a new toy? But we're actually patching symptoms. We're not fixing the root cause here. So, what is the root cause? Let me go out on a limb and claim that recurring merge conflicts is a clear sign of social technical problems. And what that means is simply that there's a disconnect between the work your agent is doing
versus what the architecture supports. And when I have a recurring problem like that, I always like to understand and verify it. And one way of doing that is by returning to behavior code analysis and use a technique I've been using for years to figure out where's the the coordination clashes in the code. And I want to show you how that works in practice. So here's a visualization of a codebase. It's a real co real codebase.
It's folly from meta. And it's the same visualization style as I showed you before. The only difference now is the color of each of these circles. Remember circles represent a file. The large circles represent folders. So it's a hierarchical visualization. The difference here is that the color indicates the offer density. So in the past this was the number of developers that had to work in the same parts of the code. Now it's the number of agents that work in the same parts of the code.
And back in the times of manual coding I used a time window of maybe six months to do this analysis. Now with agents I tend to restrict it to just look at data from the past month and see okay how many different agents collide in the code. And when we see something that is glow glowing red here, it's definitely a cause for concern. So I always like to dig deeper into that and investigate it and understand why that is happening from a design perspective. And I see here that there's one part of uh the folly codebase that is really really red and that is a part that is responsible for implementing asynchronous socket communication. And I see that the the file with the most offer the highest offer density has been modified by 58 offers, right? And that's over a period of 6 months.
So how does that happen? What's the reason that a piece of code attracts contributions from 58 different offers? When I find something surprising like that, I almost always start by looking at the code. So let's do that quickly. So this is a small piece of what that uh file looks like. And if we review this code again from a code quality perspective, there are a number of things that pop out to me here and not in a good way. One of them is this pattern. This is something that I'm always worried about. So what you see here is that you have a fairly complicated if statements followed by a lengthy comment block. To me, that's a clear indication that it most likely is a completely separate responsibility that doesn't belong with the rest of the code here, right? Maybe it should be expressed in a separate function or maybe in a separate module.
I also see as we noticed before in earlier code examples, I see these chunks of deep nested complexity right spaced out throughout the function. And these are definitely cognitive barriers to any human reviewing the code. and they are very likely to lead an identic coding agent wrong. And if we combine this, these are just a few examples, but if we combine these code smells with the fact that this file consists of hundreds of functions that look just like that and thousand lines of code, I think it's fair to say that we have met our god clause, right? Ever heard that term before? Yeah. So a god clause is simply you know it's a module that compulsively collects behaviors and responsibilities and the consequence is that no matter what you work in that system you will end up in the god class. So you and me we might work on entirely different features but we are both forced to modify the same code because the god clause becomes so centric to the system.
So gut clauses are our traffic jams and gut clauses will force agents working on what should be independent features into the same parts of the code. And of course what I want to show you here is that refactoring is not only about addressing immediate code quality concern. By refactoring code, modularizing it and making it healthy, we can also reduce organizational friction.
And to me, there's a lot more work to be done in that area. And the next frontier that I see is that what I see in my day job is that code quality at the module level is largely solved problem with the automation we have talked about today. But I think scaling AI friendliness to the design and architectural level is the next big challenge and I've started to work on that. I've started to think about it and if you want to follow along then please take a look at my uh Substack where I try to evolve this and I'd be happy to discuss it throughout the conference. But now I'm going to wrap up by giving you a couple of references if you want to read more about this. So these are some of the studies I talked about and also some my my blog posts that cover the research.
And if you would like to try this on your own code, I really hope you'll be interested. Then feel free to grab a copy of the MCP server and uh try it out as well because I'm pretty convinced that AI there are organizations that are already thriving in this age and these are organizations with strong existing engineering practices that took code quality seriously. And I'm also seeing other organizations that are really really struggling with adopting AI and getting any benefits out of it. And I think that focusing engineering fundamentals is an absolute prerequisite. So thank you so much for showing up and may the code be with you.
Thank you.