AI is making software delivery less stable. DORA's Nathen Harvey on the fix | EVOLVE 2026

5,737 words · about 23 min to read

AI amplifies existing organizational capabilities and flaws

0:05 So, I come to you from the land of Dora. Uh, of course you all know Dora, and I don't mean the little girl with the backpack. Although, when I every time I think about her and watch an episode, have you watched episodes of Dora? Like, three steps with a map every time. Every time. I mean, Swiper comes in and creates a little bit of chaos, but you know, she's always going to achieve that objective. I wish our organizations worked as well. They don't.

0:31 [sighs and gasps] But it's fine. It's fine. Here we are. So, as you know, Dora is research into the capabilities and conditions that technologydriven teams and organizations need to thrive. AI is absolutely changing the game. It's changing everything about how we work and we've just seen that. But I I just want to echo what they've just told us. The organizations that are going to win are those that are embracing continuous improvement. And that's our tagline with Dora. Get better at getting better.

0:59 That's what we want to help you do. Now, Dora, of course, has done a ton of research, and I I wouldn't be a good representative of Dora if I didn't share some of our data with you. What this slide shows us, um, well, actually, let me just ask a quick question. How does Dora get most of its data? Anyone know? Say it louder with your chest.

1:20 Surveys.

1:21 Surveys. Surveys. Yeah. And you know, for over a decade, Dora has collected data through surveys, and we hear this really consistent push back survey. That's just like your opinion, man. Well, last year, we figured out exactly how many people lied on our surveys. And that's what this chart shows us. We asked this question, uh, are you using AI professionally? And what we found out was 10% of people lie on our surveys. So we we discount we discount 10% of all of the data that we collect.

1:56 So just bear that in mind as we go through the rest of the presentation and as I share more data with you. So AI adoption it's universal. Everyone is using it and this was a while ago. So if we ask this question today well we might find that 10% of people still lie but everyone is using AI. When we ask teams and individuals how is this impacting your productivity on the whole everyone feels a lot better. I am more productive because I'm using AI. And this is software developers, it's SRRES, it's product managers, it's across the board.

2:28 Anyone really involved in the software development life cycle. Now, of course, there are those one or 2% that say actually AI has extremely decreased my productivity. I feel worse because of that. And honestly, I've had those days. Have you had those days that AI was not super helpful for me? Yeah. But it also has improved the uh code quality. So when when we step out beyond just you, how is it going with your codebase? Is the codebase getting better because your teams are using AI? And again, we've got some people that say actually it's getting a little bit worse. And again, you've probably had those days, but on the whole, it feels like our code bases are getting better. So it's nearly universal. It feels better when I'm using AI. Our code bases are getting better. But that leads us to this other really really important question and again as a representative of Dora I feel like I must collect data. So I need your help. I would like you at this moment

3:29 and this is this is a really important point. So first this is the one of three times I'm going to ask for your data. But more important than that kind of a meta point. One of the things you see on the screen right now is a QR code. I'm going to give you a a tip as an audience of any presentation. If ever there is a QR code on the screen, I don't really care what you do as long as you hold up your phone. Because when there is a QR code on the screen and you hold up your phone, the presenter says, "Oh, I'm connecting with the audience. I don't care if you're taking a selfie or checking your email. Just hold it up because that makes me feel good." Okay.

4:09 So, uh, thank you for all of you that are participating. I love collecting data. It's kind of my jam. So, we're asking this question. In the last three months, how much did you trust the quality of the output that AI generated for you? And let's see. Uh, all right. We've got most of you are a lot. Uh, that's pretty good. Pretty good. All right. We're going to let the the answers continue to come in, but uh there are I'm I'm really happy to see uh that none of you answered not at all. Uh that's that's good. But 13% of you are answering a great deal. I want to have lunch with you. Uh don't stop stop trusting it a great deal unless well you can trust it a great deal if you've put all the right guard rails and feedback mechanisms around that. That's what I want to understand. How are you doing that? And those of you that are trusting it a great deal, make sure that you share how you're able to do that with everyone else that's here. As for me, team somewhat team somewhat. I trust it

5:11 somewhat, right? We have to trust but verify. We have to build the right mechanisms around what we're generating.

1 Highlight · AI acts as an organizational amplifier

5:18 What we've found with Dora is this. AI's role in software development is primarily that of a of an amplifier. It will amplify the things that are good in your organization and across your SDLC and it will amplify those things that are bad in your organization and across your SDLC. You really have to have a strategic focus on the full underlying organizational systems. Without this focus, you end up with localized pockets of productivity that are eaten away by downstream chaos. It's what happens.

5:55 We're all feeling this right now. If you

5:57 go on LinkedIn uh or Twitter, I can't call it the other name yet. Um you you will see this in everyone talking about the code review process and how broken it is today. And frankly, I I agree the code review process is broken, but it shouldn't surprise us, right? What have we done with AI? We've used AI to generate a ton more code. Did we scale our review process or our feedback processes on that code? Not nearly enough. And so we're sending more stuff through this same bottleneck. And I apologize. I don't apologize. I used some uh explicit language there. I hope that's okay. This is New York after all.

6:35 That's probably okay. Um and it's who I am. But look, this is the stage that we're in. Like, do you all remember working like this? No. Neither do I. Neither do I. But here's a factory back in the day when when we had steam engines. And you know what happened? Steam got replaced by electricity. And what happened in those factories? Watch. Look, we went from a steam engine in the back, this big steam engine putting off well, a lot of steam, uh, to this electric motor. We amplified the way things work. We put like electric lighting in, so the lighting got a little bit better for the workers, but all we did was go harder. And that's what we're doing with AI. This is where we are with AI across the SDLC today. We aren't reimagining the SDLC. Today, that factory looks unrecognizable to the workers that were in that steam factory.

7:31 It took us many, many years to evolve and understand how does this new technology change the way that we work. We, many of us in this room lived through that same change with the journey to the cloud. We moved to the cloud and many of our organizations did not change how the systems worked at all. Before the cloud, if I wanted access to a server, I had to open up a ticket and wait for someone to provision it and then tell me that the server was there. And then I couldn't log in because I asked for a server, not a login on that server. And then we moved to the cloud. And how did our process change? Not at all. So I had to open up a ticket, wait for someone to provision me a new cloud instance. It didn't feel any different as a developer.

8:15 But it felt different to the organization because it got more expensive. We started paying opex instead of capex. It maybe even slowed us down. Like what where's the promise of that? We need to get to this within our SDLC with AI. I don't know exactly how we get there. I think that that's what's so exciting is that we all get to learn this and build this together. So I want to talk a little bit more about these relationships that we're seeing and the patterns that we're seeing in the data. How does AI impact these downstream outcomes that we really care about? So the way that you read this slide, we collected a bunch of data. We build a model of how the world works and then we run a bunch of simulations across that model and we say what if we increased AI adoption across all of these teams, how would it impact these outcomes? And if the dot appears to the left of that dash line, that outcome increased as you increased AI adoption. And if it appears to the right, it decreased. So, I want to just

9:17 zoom in really quickly. You've all taken a picture of this. And thank you. I love seeing you take pictures of the slides.

Increased AI adoption correlates with delivery instability

2 Highlight · Increased AI adoption drives software delivery instability

9:22 Also, we'll release all of the slides later. Don't let that stop you from taking pictures. It makes me feel really good. I can go home to my kids and say, "Man, I really connected with the audience today. The phones kept coming out. It's great." But I want to zoom in. The two biggest impacts that we see as you increase AI adoption, individual effectiveness goes up. Hell yeah, that's awesome. I feel much more productive at work. You know what else goes up?

9:48 Software delivery instability. Yeah. Oh, that's not what we want. What is software delivery instability for Dora? It's a measure of how frequently are you rolling back the changes that you've pushed out or how many deployments are unplanned deployments that are reworked that we had to do. This is not good. But this is kind of the state that we're in. Yes, we're moving faster, but we have not increased stability. In fact, we're having negative impacts on stability.

10:14 Not super great. The other thing that AI has done is it

10:20 has democratized who can build. Welcome builders. Uh look, all of us in the room before today, before AI, we were builders. But maybe now you have colleagues across the organization that are also building. How many of you have folks in marketing or finance or sales that are building applications right now? Yeah. Yeah. Some of you lie. Some of you don't like raising your hands. And the rest of you are honest. Thank you. Thank you. I like to collect all of that data. So, another question for you.

10:52 They're out there building. Why aren't you shipping it? What's stopping you from taking that thing that Joanne in accounting just built? Why aren't you shipping that to production? Why not? What's wrong with you? Security risk orchestration up to and including termination. Nice. I think that's the first time I've had that one. Too many dependencies, cost, accountability, the the same things that stopped us before.

11:23 risk fear. Yeah, this is great. Who is Joanne? Uh, very good question. You can trust her. I'm sure. Yeah. No guardrails, an outage, quality, risk, risk insecurity, risk insecurity, risk insecurity. You know, these things show up commonly when I ask this question. But here's my advice to you. I want you to I want you to look at this list.

11:52 That's your road map. All of those fears that you have, why can't you just ship that thing to production? You you should be building that into your platform right now. That's the platform that we have to build, that we get to build for our organizations. The best leverage that we can get out of our software engineering talent in our organizations today is asking them to stop building features and start building the guard rails, start building the platforms and investing in those platforms that make it easy for the people that are closer to the problems that need solving to go solve those problems. Let's enable that work. Let's get closer to our customers. I think that's what's really really exciting

Seven capabilities drive successful AI adoption outcomes

12:34 here. I also want to give you some concrete capabilities that you can focus on. So as we look at this widespread adoption, we also see a huge level of variance in the outcomes that teams are getting. Some teams are doing really really well, some teams not so much. So in as part of our research, we wanted to understand like what are the differences? Are there things that those teams that are really excelling are doing that are different than those that aren't? And that led us to develop what we call the Dora AI capabilities model. These are seven capabilities that when paired together with your AI adoption lead to the best outcomes.

13:18 And we're going to talk through each one of these capabilities here a little bit. We'll spend a little bit more time on some of them. The first is a clear and communicated stance. If it's not clearly understood how and where and why and which AI tools you can use, that leads to a lot of friction across your organization. In fact, one of the things that happens if you don't have a clear and well-communicated stance is you have developers just like yourselves who at home have played with Claude and said, "Well, this is really fun. I'm going to bring this to work." without a clear and communicated stance. They might not know what what are the ramifications of me using this at work. We saw one of those risks was up to and including termination. Maybe that's the risk that you're facing without a clear and communicated stance. I'm going to I'm going to be fearful that I might get caught. How many of you have felt like, "Oh, I better not get caught using this AI here at work." Don't answer that question. Don't answer that question.

14:16 But that shame and that worry that you can carry that with you. So we have to be very explicit within our organizations. Next, we have to have a healthy data ecosystem and give AI access to that data. The organization that you work in has specific policies that you follow. You might be in a regulated industry. You might have other security or other constraints that you've placed on your organization that the frontier models simply don't know about. They don't understand how your organization works. They're trained on this huge corpus of data but not your organizational specific information. We have to make that data both healthy and have a good healthy data ecosystem and make that accessible to all of the AI.

14:58 We need strong version control practices. This is a really interesting one because uh kind of twofold. One, I think that we're we're probably at a state where it's time for us to reimagine what does it mean to have version control. But what we know is that when we have good version control practices that those version control practices provide a safety net. If the AI goes off on a wild tangent and builds something ridiculous, we can at least roll back to something that was sane, something that worked well for our organization. So we have to have those strong version control practices in place. I've talked to more engineers than I would like to count who've come to me and said, you know, ever since I started using AI, I'm committing to version control a lot more frequently.

15:42 And the reason I that blows me away is like what were you doing before? Like why weren't you committing every like all the time always be committing lots and lots and lots of small commits do that? But speaking of small commits, working in small batches is another thing. And this this idea I mean this is I think the hardest one for us to hold on to because AI fights us so hard against this. But the reality is as engineers one of the things that we are skilled at is taking a big problem and breaking that down into smaller things that we can go and solve. And as we solve a small piece of that problem we can get feedback on that small piece of

3 Highlight · Large prompt batches destroy code review effectiveness

16:22 the problem and we can understand are we headed in the right direction. Well, AI's come along and I can build a whole application with one prompt. I'm sorry to tell you that's not a small batch. That is a big batch. And we we see this in code review, right? What happens when you get a a code review, a PR that you have to review and there's maybe, I don't know, 15 20 lines of change in that PR, you're going to look at that. You're going to inspect it. You're probably going to provide some real feedback to the user who opened that PR.

16:52 And then the next day you get a PR that's 2,000 lines of change. What do you do with that PR? LGTM, baby. Let's go. Ship it. It's good. It's I ain't got time to read two I'll I'll critique your eight lines pretty hard, but those 2,000 lines, ship it. It's fine. Nobody's got time to read that. Nobody's got time to understand. That's bad. It's bad. Look, as humans, we're better at solving small problems.

17:19 Guess what? So is the AI. Let's give it smaller problems to solve. Let's ship smaller items so that we can get that feedback. Next up, we have user centric focus and quality internal platforms. I want to come back to those. But first, I want to show you what the model actually looks like and I'm going to take you through an interactive version of this model. So, the way you read this model is basically with AI adoption, which we all have, we want to get to outcomes.

17:44 Like none of us are using AI because AI, we're all using AI because we want better team performance or higher code quality, better product performance and so forth. So how do we get there? AI adoption combined with these seven capabilities lead to those outcomes improving. So you pick an outcome and you work backwards. I'll show you what that looks like. I'm just going to jump over here to dora.dev. Whoop, that's not the right place, but you know, there dora.dev. Uh if you've not been to dora.dev, dev. Uh I highly encourage you to head there. Uh and from here, I'm just going to go to the AI tab here. And we see the uh capabilities model. Let's zoom in just a little bit. Make that a little bit easier. Let's say that your team decides we want to improve product performance. What can we do to make sure that AI is driving better product performance outcomes? Well, the first thing you might have or the first question you might have is what do you mean by product performance? Well, we show you exactly what we mean in the land of Dora. If you click on product performance, you can come down here and

18:44 see how did we assess product performance. And this is where we publish all of the questions from our survey. So you could go and figure these things out for yourself. Product performance means that the product does what it's supposed to do that the performance metrics uh like latency are improving and so forth. So we figure out we want to improve product performance with AI adoption. What do we need to have in place? We need to have AI access to our internal data. we need to be working in small batches and we need to have that clear and communicated stance.

19:13 Well, how do we get to any of those things? We can pick, I don't know, working in small batches. And again, you can see here how to assess it and you can learn more about what that particular capability is. And here we give you advice on what that capability is, how you can put that capability into practice, some of the pitfalls that you're going to run into as you try to work in small batches, and even some new measures that you can use to sort of understand what is my actual batch size.

19:39 Uh, so really helpful resources here that you can use over on Dora.dev. So, uh, oh, that's the Dora AI capabilities model in action. And now

Quality internal platforms and user focus matter

19:49 we're back. Okay, so I want to talk specifically about this idea of user- centric focus and quality internal platforms. And we'll start with the user because well, I mean that's always the best place to start, right? Who's the user? Do you know what a user doesn't want? More features. It's hard to say, right? Our whole job is to build features, isn't it? But go talk to some of your users. They don't want more features. They don't want more features. I mean, some of them do for sure, but what they really want is to solve their problem better. I'll go back to an early early early phase of my career. I was working at in an organization where um we were we were moving our architecture to client server.

20:33 Any of you remember, some of you remember those days, but most of you don't even know what I'm talking about. Well, so we went and we worked with the operators that were using these applications and we moved them off of the mainframe where they had a dumb terminal and they sat down at a typewriter all day basically and like did their data processing and then we moved them to client server and we handed them a mouse and they're like, "Well, this is ridiculous. I know exactly how many times to flex my pinky to tab through all of this uh fields on this screen and now you've handed me a mouse and I I'm working at about uh half half the speed. I cannot work very well.

21:10 All of that muscle memory is lost. The same thing is happening with your users. How many of you uh I don't know, everyone in this room has probably been on Facebook and Facebook changed their algorithm and changed the layout and then everyone had to field calls from your mother about how dumb Facebook just got and you're like, "No, no, no, mom. It's cool. It's better this way." No, they don't want it to change, right? So, but it is interesting. It is really important that we have to go figure out who are our users and why. What what problems are they trying to solve? How do we help them? It's not always with more features, but every time an engineer gets closer to a user, that is time well spent. And every piece of software that we build is for another human. Every piece of software that you build, even if the thing that's going to consume that soft, like maybe you're building an API, you're like that API isn't for a human, it's for another computer. Yeah, but there's a human at the other end of that. It's always for another human. And so it's really important that we not lose sight of that. Next up is our quality internal platforms. And this goes right back to

22:12 that. Why won't you yolo that thing into production? We have to build the platform that enables that for everyone. The platform is what gives us confidence and trust that the AI is building what we want. The platform is what reduces friction. It improves our operational excellence. And it's so important that we really focus and dedicate time to building out that platform. Uh so please invest in your platform health. And I'm sorry, you're the choir. I'm the preacher. Like, you're here at Cortex Evolve because you're investing in the platform health. Take this message back to your team. Uh your leaders need to hear this as well. When it comes to platforms, we treat platforms like products. Every product has a customer.

22:59 Who's the customer of the platform? Well, it used to be only other engineers, so we can relate to them pretty well. But today, it's not just other engineers. It's also agents. Agents are consuming our platform or using our platform. So we need to build for them. The beautiful thing is that anytime you invest in developer experience, it's going to pay off in your agentic experience as well. And vice versa. Think about this. Documentation quality is something that helps both the agents and the human users, the developers. But now we have the builders coming along as well. How do we help those builders? And oh I forgot this. Not everyone wants to be a builder.

23:40 Did you know that? I didn't know that. Look, we're in an echo chamber here and like AI has come along. It's going to change everything. Everyone's going to build anything that they want with AI. Let me share a text I got with you from a friend of mine that's in finance. This text came in last week. You can look at the time stamp. This is a real text. Turns out not everyone wants to be a builder. Who knew? Why wouldn't she want to build? I promised her I would not attribute this to her by name. Uh I told her I was going to make her internet famous for not wanting to build and not knowing what agent who doesn't know what agentic AI is. I'll tell you what, a lot of people don't know what agentic AI is.

24:24 They think we're talking in this foreign language thinking magic phrases. I don't know. Uh, so this is this is but this is real. So they're not builders, but you know who they are? They're solvers. They are people that like she may not want to build something with AI, but I can guarantee you she runs into friction every day in the work that she does and she would like to solve for that friction. So maybe the best thing that we can do is go sit with her and understand what friction she's running into and not ask her to build a solution for it, but let's solve that problem for her. Let's solve that problem with her.

24:59 And if we can do that with more and more of our users, we're going to provide more and more value not only to those

Expect a productivity J-curve during adoption

25:05 users but also to our organization. As you go through this journey, there are patterns that are repeating. These patterns happened whether it was steam engines moving to electric engines, whether it was data centers moving to the cloud, whether it was client server moving to web services. All of this happens. There's a J curve of impact.

25:32 Anytime you adopt something new, whether it's a new technology, a new process, a new way of working, you should expect productivity to dip before it recovers and before you start to see the real gains from that. That dip is important. If we don't model for it, if we don't expect it, like uh Ganesh told us, we can't just put our head in the sand. We have to have that leadership honesty. We have to know that this is coming. Now there are things that we can do to both diminish the width and the depth of that dip. But we have to be intentional about them. We have to understand what are the things that are causing that dip today.

26:07 It's the learning curve. It's the verification tax. It's pipeline adaptation that's happening. How do we think about and sort of put countermeasures in place for all of these things? You can't just give AI licenses to everyone and then expect tomorrow you're going to get 10x better productivity from everyone. There has to be an enablement phase and a an experimentation phase and it has to be safe to fail because that's what we're going to do. We're setting out to fail.

26:33 No, no, no. We're setting out to get better, but along the way we're going to fail. It's just it's how it's going to be. Okay, last question for you today. The last bit of data I want to collect and I will apologize in advance. This is a multiplechoice question and there is no other. There is no fill-in- thelank. You got to pick one of these. So, of these four or five, what's the biggest friction point that you're running into today across your SDLC, right? AI is clearly doing its job.

27:13 Look, 0% of you said building is is the friction point. Great. the majority of you although it's a kind of neck and neck here reviewing changes I don't believe in two years time that's going to be the bottleneck I think we're going to solve for that we're going to solve for that either by shipping smaller features by changing our review practices for understanding more about how to improve that maybe even leveraging AI to help us review changes maybe we should have started there from the first place Forget using AI to build. Let's use AI to review changes first. Why start there? Well, it's safer.

28:00 It allows us to understand how these models work and what the tools do. And by using them, we're building more trust. Then we can go on to crazier and more risky things like building with AI. So, I think as an industry, we did it backwards. We should have focused on [snorts] review and feedback first and then come to building. But clearly, building solved. So reviewing changes is our next challenge. It's not the only challenge. It's just the next one that we're going to run into. And and thanks for keeping that neck and neck.

28:29 Measuring impact. Yeah. Uh guess what?

AI increases optionality under existing system constraints

28:32 AI didn't make that harder. Uh it was already very difficult to do to measure impact. All right. So I'll I'll close out with just a couple of quick thoughts here. The ROI of AI is optionality. How many options do you have now with AI? You know, when it comes to building things for users, we always are trying to figure out what is the right thing to build for users. Well, with AI, because we can build so quickly, we can get into a room and we can prototype eight things in the time that it took us to argue about two things.

29:08 Let's feel them. Let's see what they look like. Let's maybe even imagine this, put it in front of a real user and see how they respond. And let's use that information as feedback so that we can get better and better at building the right things. Not just building the things right, but building the right things as well. And AI really opens up that optionality for us. It's really huge. I also have to have a Deming quote in any presentation I give. So a bad system will beat a good person every time. AI didn't change this. AI amplified this. So if you have a bad system, it's going to beat down those people even harder. Now, so this is the thought I'd like to leave you with. This is, as Ganesha Anish mentioned, this is a a a change management problem. This is a cultural change problem. We need to build a sustainable system. That sustainable system needs to be sustainable from the systems like the computers that you go

30:10 and kick. Well, you can't kick them anymore because they're in the cloud. But also for more importantly the people that are part of this system. We have to build a sustainable system for our people. How many of you in your organizations please don't answer this question? Boss came to you and said here's an AI license and then came to you the next day and like where's the 10x productivity and now you feel like yeah I I can do more but I have to work more. That's not how it's supposed to be. It's supposed to give us more time for more creative thinking, not feel like we're on this uh treadmill even harder. So, please take this with you. This is all about how do we get better at getting better.

30:49 Learn from each other. You're here in the room today. Have those great conversations. Continue to get better at getting better. You cannot do that alone. You're here with a community today. We have a community at Dora where we're learning and talking about how do we put all of these insights into practice? And then most importantly, and this is where you get to hold up three phones, uh, read our research. I think that you'll find it really helpful. Uh, but the most important bit is that you take that research and you put it into practice. Thank you so much.