The Downside of AI Coding No One Talks About

11,207 words · about 45 min to read · YouTube’s automatic captions

0:09 Back in November, we hosted the first ever in-person AI native DevCon in New York. This June 1st and 2nd, we're bringing it to London. It's two days built for AI Native developers and engineering teams. One day full of hands-on workshops, and one day full of practical talks on agent skills, context engineering, agent orchestration and enablement platforms, and how teams are actually shipping AI in production. Join us at the Brewery in London near the Barbacan for all of that, plus networking, parties, giveaways, and a room full of people building the future of AI native development. You can also join us from anywhere in the world via the live stream. As you're listening to this podcast, you get 30% off your ticket with code pod30. Just head to a nativedevcon.io and we'll see you in London.

1:02 Hello and welcome to a milestone episode of the AI native dev. My name is Simon Maple, one of the hosts of this show. And joining me is

1:13 Guy Pajari. I'm the co-host, Simon's co-host and sidekick overall. And uh today's a milestone episode because do you know guy we've uh already done already shot and and released 100 episodes of the AI native dev.

1:26 That's insane.

1:27 How does that feel?

1:27 Like I don't know how we get anything else done.

1:29 I know. Just sort of all the time just sort of coming here and talking to interesting people which actually is not a bad reality.

1:34 Yeah. We're so we're a podcast company, right? And we and we we do this other thing on the side.

1:39 It's the fallback. It's the safety net. I don't know how safe it is but

1:43 it is interesting.

1:44 Amazing. And so this episode is really going to look back at the previous 100 episodes and we're going to take sound bites uh from some of the most amazing speakers that like you say we've had uh on the on the show.

1:57 So it's going to be about 45 hours this uh

1:59 something around then. Yeah, it it'll definitely be a week at least. Um so so yeah, make sure you you've got a long drive or and we're going to we're going to add our commentary um and also see how things have changed. for example, let's run the first clip, guys, because I think we're going to find this very, very interesting.

2:16 Okay,

2:17 if anyone who's predicting what that looks like in a year or two is going to be fundamentally wrong. And I think that that's really what makes that so exciting. We don't know where we're going other than that we do know that we're going to keep building software while maintaining, you know, billions, if not trillions of lines of code that are already out there. Thomas Donker of course the uh the previous CEO of GitHub and now I believe founder of a new company new startup in in the AI space called entire. So if anyone's going to be fundamentally wrong about a prediction in one to two years let's go back maybe let's say one to two years guy to episode one where you predicted something around AI development.

2:57 Yeah I never claimed to be too wise. No, no, no, no, no, no. Caveats now, guy.

Moving from specentric to context-centric development

3:01 Let's hear what you have to say and then we'll see how accurate you were.

3:05 I think the biggest change that we should expect that I feel I feel fairly convinced will happen is the move from codeentric to specentric development. Um, when you think about how you develop software today, uh, you write you get some requirements, you write some code, you make 100 decisions in that code that never leave the code. Uh, and the code very quickly becomes the source of truth. uh and we focus on evolving it and growing it. I think in the world of AI native development, we will have a means of specifying what it is that we need and AI will provide the implementation.

3:39 So guy tell us uh tell us your new take how how accurate were you? Yeah, I mean I think so so something's spot on uh some aspect of it in that uh I think I think today it's a much less controversial statement to say that you know code will become disposable and that uh implementations uh will will be you know something that is just kind of created for it uh and so that you will revolve around around intent you know around what it is that you want to build. Um so I think it was like not far off on that

4:09 bit that bit was good. Tell me about the bit that's fundamentally wrong. As Thomas would say,

4:14 I think the the the primary kind of change I'd like to think evolution kind of in in my thinking, our thinking here at Tessle, uh is this move from spec to context, right? It's when when we recorded that episode and we're you know talking about specentric software uh we talked about how you want to make sure that LLMs this is pre-agents you know now agents will uh make sure to update to kind of capture you will you will convey what it is that you want to build and then the LLMs will build that and they will fill in the gaps that's what we talked about back then and while that is not wrong you know it's it's true LLM need to do that it's just such a small part of development when you think about your dev team and what do you expect of them you You don't really say, "Hey, make sure that every time you follow you read this doc and you follow the exact instructions and you update that." You know, sometimes you want them to do that, but generally you want them to to make good decisions, you know, including choosing when to update documents and read them.

5:07 But that's just like a portion of what you want from a great developer in your team. you know, you want them to uh to, you know, have all sorts of preferences

1 Highlight · Moving from specking the program to specking the programmer

5:15 around how they build and how do they collaborate on it and how do they test and make sure it didn't break and how do they balance kind of quality and how do they and then sort of speed and how do they kind of use your existing infrastructure and how do they troubleshoot issues and so all of that is really about training the developer and so I guess my my kind of catchphrase today is to think that uh we've moved from specking the program to specking the programmer so it's much more about how do you want a developer a great developer in your team to behave and and again there's a portion like I don't think specentric software or specdriven development as is the kind of the term today goes away like I think that is important I just think it is a subset of how you want today agents to build and it's more important when you think about the new software development it should revolve around um training these uh AI agents to build the way you want them to build

6:09 such a Such a good way of putting it.

6:11 And the way one one way that resonates with me is is onboarding. And it's like if you was to hire an individual to work on your team, you would spend time onboarding them, training them. They they could be the most senior developer on your team like that you've hired, but they still need to know how your team works, your organization works, and take on those those policies, ways of working to actually use their skills in the new environment. And there's no reason why we shouldn't do that with an agent as well. And it's that onboarding of that agent and very very very nice.

6:43 Yeah, absolutely. And that would sort of adds it's like onboarding and then continual training. No, you want to make sure that they you know as as a group you learn things. So some of it is you individually learn but a lot of it is the the group learns you know the team learns and you want to make sure you accumulate that and retrain the team. So both those are human analogies, right? About how do you want uh people in the team to to work? You want them to on board and then you want them to kind of keep keep with it and then sort of adapt and evolve and be aligned with the rest of the organization how they build. So we need to build that. So all of that today manifests in the form of context engineering. Uh and I guess the unit of software today or unit of context today that is most discussed is skills. And so I'd say if you take the clip, you can swap the word spec for skill and you know it will still be natural language words. It will still be uh a capture of intent. Uh but it is not a program spec.

7:36 It is a programming spec. You know how do you how do you want to develop software here?

7:40 So it was a it was quite a subtle thing that you were fundamentally wrong about.

7:44 I was fundamentally wrong. I had to be fundamentally wrong about something

Overcoming the human code review bottleneck

7:47 otherwise we wouldn't be learning anything. Would be boring.

7:49 Simon I mean we can agree you and I can agree that you were fundamentally wrong about that. So that that's fine. Okay. Next up, you know, Thomas didn't just mention that you were fundament fundamentally wrong about something. He also talked about the changes that developers uh need to go through. Let's uh let's hear from Thomas.

8:03 Simon Wilson on his on his blog recently had like a statement. I'm paraphrasing um but this was like we have to get comfortable with not reviewing every single line of code before we deploy them into production because the sheer amount of code that a single agent can write 247 now you paralyze that into running 10 agents in parallel all the time. you're just never going to be able to review all the code without the one human doing that becoming the ma major bottleneck and effectively erasing some of the productivity gains from this agent.

8:33 Yeah, very interesting take there from from Thomas and actually also supported by a couple of other folks on the on the podcast. Uh Peter Guagenti from episode three way back uh also talked about a potential future whereby uh agents might actually write code that isn't even human readable. So maybe something close to a bite code or or a machine style code. Uh Bea also from Thoughtworks in episode 69 also said she ran into situations where she actually has done so much almost like a vibe coding but you know allowing the agent to to write that code that she feels so disconnected from the code. She isn't necessarily the most appropriate person to do the code reviews. What's your take on this?

9:12 Yeah, I think uh so super interesting fully aligned you know with sort of the Thomas' view in general like that the human is the bottleneck. And uh there was like all sorts of similar statements in the AI world. You had sort of Elon Musk talk about how he had some weird spreadsheet analogy on it, but he was saying if you have like a gazillion spreadsheets on it and just a few cells need to be filled in by humans, you've reduced it down to human speed. uh uh and I think that's what we're sort of seeing with code review uh that you know if humans are the bottleneck then you know you get human traits like review fatigue but then you also just sort of see slowness right this is the the human speed you know is limiting uh how much how much can AI kind of code and build up and so you're just moving the bottom neck I think this will actually extend further so once you get through and you deploy now the problem will be that the system changes too quickly and we can't uh do uh like uh observability and root cause analysis and handle those. We can't figure out that a bug needs to come back and uh and be resolved and then resolve that and deploy the fix. So

10:14 really throughout the end to end software development life cycle, our aspiration really has to be and I think we're seeing this in the advanced organizations to really identify every single step of that process. Um and I think that is the only way to to move forward. uh and when we think about okay but but these things are unreliable right they don't do you know that they build uh your the right thing is it it comes back to context right and uh you

2 Highlight · The developer as a second-line manager

10:40 know at Tesla we talk about the the context development life cycle right and I think agents should operate the SDLC and the human should live in the context development life cycle in the CDLC they should really be a guide and I think that the human analogy is really a manager and if you're you know at the moment we're like the frontline manager with the player coach you know you're you're sort of giving the tasks you're monitoring and you're you know getting things done and increasingly if you want to increase your scale and have more employees in this case AI employees or sort of you know AI labor that is getting things done then then you need to become you know a second line manager and a third line manager and I think those are those really predicate on your ability to define what correct looks like to convey instructions to uh to identify that mistakes were were made and be able to resolve them and so you need to move up the stack and the way you communicate with agents is through context. It's not just context, it's context is the tools you make available

11:36 to them. It is the um the like the the foundation to be able to observe what has happened. And so that's why when we think about context development life cycle, we think about kind of generating some piece of context like it it's actually we talked about that in one of those episodes as well about how you know knowing what you want is actually hard and so like figure out what it is that you want, what is good, what is correct behavior and then you want to be able to test and evaluate. This is maybe the equivalent of testing humans but in a little bit more software of like your tests to know whether these instructions are well adhered to. So we think about eval. You want to be able to disseminate and communicate to that to the uh horde of agents you know that you have the right communication to the right agent at the right time and then you want to observe what has happened and once you've observed you will have learned a bunch of things also a bunch of like facts in the world have changed know maybe some codes uh have has changed maybe a pricing model a new model came out and so you want to learn from that come back and sort of update your instructions and so all of that is what we think of as the context development life cycle so I think that is that is

12:36 very kind of aligned uh with this destination and I think what we've it's interesting that the world of AI uh you know I I I I like my crystal ball you know like I like these you know fundamentally wrong uh predictions uh but uh and and and looking ahead but I think one of the things that I find most interesting about the way AI has evolved is uh it's oftentimes around solve the next problem solve the next problem solve the next problem so I think the next problem to solve right now is code review because we're filling it there I think subsequently we'll get into those deployment uh threats Because one of the biggest worries is of course the quickest, cheapest, shortterm fix is just to allow things to happen and for developers to go if I'm the bottleneck, I can just eliminate myself from being the bottleneck by just either just checking, yeah, this is fine or just not doing it at all.

13:25 So it's about doing things. How can we speed things up, continue at the pace that AI wants to generate at

13:31 while doing things accurately and properly and adding that context in to almost take take things away from the developer so the developer the human can really look at the things that are most important rather than as code changes scale up to try and do things faster by you know

13:48 by learning how to code review faster that doesn't make sense. But adding this context actually can take a lot of that away from us and for us to be that second line manager to understand it to to to manage it makes it a lot more sense.

13:59 Yeah. And I think the cloud analogy kind of works pretty well here which is you know when we were in waterfall era a lot of these processes and a lot of these reviews were human and as we moved into cloud we had to automate them right we had to define

14:11 to do policy as code as the sort of the meta name for it but you know infrastructure as code to deploy your infrastructure automated testing you know and dev QA became a thing it wasn't a thing before on it and so we we actually have have gone through a fairly similar uh shift in when we moved to clouds to say things are moving faster the bottleneck is the human let's automate this action of the human and now we're we're doing another another leap over there and it was mighty uncomfortable at the time to say oh what you mean I I'll automate this and I will deploy it and I will deploy more bugs

14:46 and you know the time has shown that if you close the loop then yes you might even deploy more bugs but because you're able to identify those and then fix them and then come back and roll those out the systems that were continuously deploying doing, you know, today's well known. They are more secure. They are they have higher quality, you know, they have all of those, but you have to go end to end. If all you've done is remove the gates, right, and just said just just let it let it flow, then you end up with a system that is more broken. Uh so a lot of that cloud learning actually applies here substantially and and uh you have to think about the end to end uh uh operation. It it is happening faster than in cloud. And so what we don't have is the benefit of uh of like trying it out and seeing if it works. I have one more small comment just on on what you said about uh non-readable code which is interestingly something that is now starting to surface as a safety concern. Mhm.

15:41 Uh so there's there's actually some significant call outs to say that actually a safety requirement should be that LLMs are not allowed to produce things that are not human reviewable

15:52 uh like binary code or you know their own languages even if it is more efficient for safety uh as part of being able to supervise agents. So that's interesting. I don't know if it's an over kind of an exaggerated concern or not. uh but I think right now more likely you know in the prediction realm that we will uh produce readable output readable code and then it'll get compiled.

16:16 So following on from code review I guess

Shifting focus from code craftsmanship to architectural decisions

16:18 next up we have uh Annie Vela who's a distinguished engineer from Westpacking and um really interesting blog and discussion that I had with her around the future of developers. Let's hear from Annie. There are literally people who enjoy going through those sorts of elite coding exercises on there's a reason that website exists, right? There's leaderboards of people who can solve the most complex problems the fastest. So that I don't think will be very important anymore. That's a skill that a lot of software engineers have taken to be the core of being a really strong software engineer. That is a skill I think we're going to need to learn to let go of both in interviews and personally. It just it won't make much sense anymore. Why would you need to be an expert at writing that level of code when an AI can probably generate something good enough for you?

17:12 Really important point and I think it's something that I think we're most I think most of the industry is aligned on it, but it's possibly one of the things that we are least comfortable giving away because it's a thing that we enjoy. Yeah. Uh it's a thing that we have become it's it's part of what excellence is in or or seen as in development in programming to write the nicest code the cleanest code that solves the deepest most complex problems

17:39 and be fast about it

17:40 and be fast about it. Is this going to be a problem for developers adapting

17:45 uh in the future? I mean I think for some for sure it I I like the mental model that says like some developers I'm generalizing here but like some developers are more motivated by the craft of coding

17:57 while others are more motivated by the impact right or by the creation. uh and so I think if you're motivated by the creation side then you know agents are amazing you can produce so much the shortening that journey from idea to ex execution even if you were are very very fast as a human developer it still takes you that time and while in the meantime you could have thought of other things right

18:20 it's the means to the end of the coding versus so so the fun is uh having an idea and bringing it to life and tuning it and orienting it and even making some architectural decisions and things like that right around what is elegant and what's not but like writing the code uh step in the journey right is not the goal while for others writing the code is the craft you know and I I do think there's beauty in the architecture of you know like writing correct code you know well-crafted short simple code on it all of those are

18:49 good abstracted Java classes and interfaces

18:52 exactly yeah and also in like modern languages but uh but uh you know you you um so so there's there's beauty in it you know so there is there's probably room for it from an aesthetic perspective and a skill, but it it's slightly becoming like a a factory manufactured furniture versus kind of handcrafted ones, right? You can create things that are that are very local. They might even be able to better represent very very specific preferences, very specific decisions that are almost hard to convey in any way that isn't code. But for the vast vast majority of furniture, you're totally comfortable with them being factory-made, right? With them being u built by machines. Uh and and so I think that is true for code as well and for increasingly sophisticated and complicated code. And the advantage of doing it by hand is is so minuscule compared to the advantage of being able to produce it quickly, cheaply by more people. Like think about the the loss of

19:54 knowledge when you have one person with the idea and they need to convey that to you know a person to convey that to a person you know to eventually get to someone who writes the code. So

20:05 dropping some of these middlemen. Um so all of those are just so much more valuable. So the reality is I think there will be room for handcrafted code in some niches in some very specific spots and some artistal element and I think it can remain a hobby just like today. There's a bunch of these, you know, hobbyist hardware, hobbyist, like it's okay for that to to happen. But I think as a profession, you got to let go.

20:29 So, so do you think first of all the de the best developers five years ago will continue to be the best developers in five years? What do you think the the the key skills need to be for a successful developer?

20:43 It's a good question. I don't know what the sort of the percentage is of people that are more the best developers because of their ability to craft code versus the best developers otherwise. I'd say that even like throughout the years, the best developers are the ones that truly understood the problem to be solved and the architectural tradeoffs to be made. And then they they just saw the codew writing as a uh as a pass through, right? they knew how to then translate the English into code quickly.

21:15 Um, and so I think it's the same best developers, but I think there's uh um if you were to contrast or like compare, you know, different developers, for some of them, the strength of code writing uh is was a a bigger part of that mix. Uh and that will diminish. I do there's another element of just speed of decision-m and so writing code is slow and because it is slow you actually get

3 Highlight · The importance of fast iteration and rebuilding

21:41 to think about the problem and these architectural decisions and all of those as you as you code

21:47 and I think that is an interesting thing that is lost a little bit when you are coding quickly because you need to decide more things up front

21:56 and and that's hard because sometimes you don't need to do it uh I see the pattern that uh that is emerging to to try and tackle that is okay as you get the agent to code be explicit around architecture. Even if it's not you who made the decision but the agent have the agent be explicit about the architecture review that don't be afraid to tell the agent scrap all of that and build again. So this ability to rebuild

22:21 uh is is really more important than the ability to build. How do you capture that? Slightly comes back to context of you know in this case maybe it is sometimes specs and architecture is a specific type of context to say this is what we've built. This is these are the emphasis points about how we built it.

22:36 Now remove the whole thing and write it in you know I don't know Java or again rust something

22:44 uh per your requirements

22:45 and I love that I love that thought where you know as coding becomes something that's very fast and actually taken away from us slightly we do lose that time to think as we're building but what we do gain is that that faster iteration which allows us to gain feedback faster and throw away quicker. Yeah,

23:04 we can throw away we can and we can use that feedback to build something faster with real, you know, stronger knowledge about what works and what doesn't.

23:12 Precisely. Which which by the way, like

Context is critical for DevOps and root cause analysis

23:13 the a recent episode with Chad Fowler about regenerative software and the Phoenix architecture is really all about that.

23:19 Amazing. So, next up, let's jump into the world of from dev to ops via DevOps. uh alka

23:28 you might hire the world's best s sur who knows terraform and cloud inside and out but when they join your organization they don't know anything about your organization so they don't know do you run windows in production rail in production Ubuntu in production now they might be power users at all of those things but the context of knowing hey as an organization we standardized on rail that's critical piece of context that you have to be a useful sority at the or right you don't want to hire someone the rest of my stack is windows and all of a sudden this person's writing Linux for me but doesn't help me right they need to understand how does the organization work what have we standardized on what does my existing infrastructure look like for them to generate things that are then contextually relevant to my environment

24:08 and you can tell from your from your youthful looks there guy that was that was back in episode 19 uh Arman of course the co-founder of Hashorp um so yeah context is the difference between great AI and useless AI and it's not just development it's generally AI right

24:26 in ops I think what we're seeing in first of all context is probably the most abused word right now it's like anything you need to know you know and I think the the meta perspective over there is to separate intelligence from knowledge you can be massively intelligent but what do you know and and I think there's a bunch of things you need to know to operate in an or uh I I really liked the the episode with Armon which was you know early in in in DevOps we also had a recent one with uh Mirao who's the founder of D-Zero kind of a modern observability play also talking about context and how it's important from an observability uh perspective and the ability to troubleshoot and do root cause analysis. Also in the world of DevOps also highlighting the the key is context. I I think uh ops is an interesting place because when you think about who in your team like which developers are you know carrying the pager and able to troubleshoot the things or even like the superhero sRes in uh in some organizations who are the ones that will decult to firefight you know a really bad uh production outage those people they have a very broad

25:26 understanding of the system right they they know what are the types of applications what are the patterns that are deployed what is the infrastructure that they are running on So you know all the way from like operating system to the types of scaling historical like recent uh or past outages like known weaknesses within the infrastructure that have not yet been prioritized. You know there's there's all this knowledge and because of all that knowledge they can look at a piece of of of current data of like this system crashed or that you know those requests are slow or whatever it is. uh and they're able to kind of quickly translate that into oh it's probably this problem and it's therefore probably this solution uh and so I think it is very uh uh like context is very very important because often times the the production system is where it all comes together right it is the infrastructure you've deployed it is the system that you are connected to it is of course your code and your application

26:24 and what you're seeing in in so so definitely seeing in the world of devops that context is important important than any any player kind of on the on the planet will will talk about uh about context uh when when it comes to DevOps and what we're seeing is uh root cause analysis capability is the one that is manifesting as the as probably the most useful in the DevOps world to be able to look at these kind of vast amounts of data that exist about how your system operates which is a strength of AI and agents uh to understand it combine that with context in the organization where you're not just blindly looking at the logs, but you don't know anything else.

27:02 You're you're actually able to to gather more of that information and then you go off to uh to understand it. And the advantage of root cause analysis is that it's it's kind of harmless. um where you're actually seeing still very cautious use of AI which was still the case like back in you know uh in episode 19 you know with uh with Armon uh is the use of it for actual ops uh activity like people are still very uncomfortable with an agent deploying a system slightly slightly more comfortable with writing you know some Terraform code

27:40 that's another interesting piece because like everything that you talked about up until the Terraform code uh is is really pulling sensitive information I guess but information that has come out of your ops infrastructure whether it's troubleshooting pulling data and logs and things like that but when we talk about creating the creation of Terraform code uh enterprise terraform code that's not just something that you find on someone's GitHub account right whereas typically there is so much more you know for a developer there's tons of code that that that you know is that the agent can train on uh the LLM can train on it and will provide you with good suggestions and good alternatives to what you're trying to do. But is there really the the the right training data there for an agent to use or an LM to use when generating enterprise quality and enterprisegrade uh Terraform scripts for example?

28:30 Yeah, I think I think there was definitely a hold back in the early days. There's still a decent amount uh out there. you know there's a bunch of public modules and things like that and you know there's there's good documentation for it so it can follow it and eventually you know declarative languages like HCL the hashorp language or or rego and such are a little bit easier there's still probably behind the like a lot of commentary aboutgo the open policy agent you know not being you know agents not being great at creating it so there's probably room to improve I I I I do I have seen substantial improvements on it or like hear you know from from uh various users and you you see it probably for two primary reasons One is there's some data aggregation that happens just as as these things are actually used in in the real world and so there's more data accumulated from private sources. Uh not all of it can be trained right away but still there's signals that can be learned. And the second is the ability to create synthetic data and so the labs are increasingly relying not just on public data but on synthetic data and I do think that infrastructures code has the same kind of attribute that regular code has and sometimes even more which is you

29:31 can generate something and you can check if it actually works. So you can deploy it and sort of see whether the result is correct or not. Uh so that becomes just a kind of a data generation exercise.

29:41 But still that is one kind of limiting factor is indeed the sort of the public private and maybe the the price of generating it. Another big uh concern or like limitation is the ability to deal with time series data and so when you analyze observability data it's very important the sequence you know of things that that happen. And the NLMs are like a little bit more kind of meta uh uh thinkers, you know, of how it is. And you've seen a variety. We had Ollie from Data Dog on the podcast on it. You know, we talked about Toto, their uh little dog, but also their sort of time series uh model. And you see some of the startups like traceable uh in the space and all that actually look to create models or or at least fine-tune kind of RL models around that. And so the ability to deal with those volumes and then on top of all of those there's just like sheer price like when you're producing a lot of data you're not going to you're going to flow that through opus and you'll be happy you know until it hits your bank account like it's not uh uh so you you need to be smarter and you know there's much many more attempts of sort of using open and cheap models.

30:41 So I'd say overall context is key in DevOps as much as it is anywhere else and maybe even more so. Um, and I still think the DevOps world, and we've had in, you know, like we've mentioned three episodes over here, we've had a few other references to it, still ends up being a little bit more cautious, mostly because of some of these sort of technical limitations. But I'd also say the blast radius of, you know, if they got it wrong, you know, I guess what what are the horror scenarios that we sort of hear about agents running systems is they drop the database in production, right? They those are are kind of very hard to recover from uh type setups. So, it's an interesting let's solve code review. and then you know kind of evolve from there.

31:20 Uh but it's still it's still making progress.

AI amplifies both good and bad engineering practices

31:23 You mentioned meta thinkers. Let's switch. This is so seamless guy Ian Thomas who's a meta thinker because he he works for Meta. Um he he he has some thoughts about the adoption of AI within the Meta organization and I caught up with him at uh at Cucon AI in in New York. And um actually to set this up, Peter uh Guanti who who's at mine also mentioned a stat that he gave was uh that Gartner uh say that 10% of enterprise devs use AI tools today but by 2027 that number hits 80%. Let's see what Ian thinks about that.

32:04 We had a few people that were really keen and and sort of experimenting and they were they were finding value of these tools outside of work. And then we had a bunch of really senior engineers who were perhaps a little bit more skeptical. And then gradually over time, we've seen adoption grow. Last time I checked, we were over 80% week active users.

32:20 Oh wow.

32:21 80% over 80% weekly active users within meta, which okay, is a little bit biased perhaps uh given how deeply involved they are with AI, but over 80% weekly active users within meta using AI dev tools today. Does that surprise you? um it it doesn't but I feel like I have a few grains of salt to kind of throw into that statement you know uh one is this is this is a world of of kind of halves and have nots you know on it and what you're seeing is this sort of big polarity between organizations that have embraced and invested and you know have uh leadership that is both bought in and able to execute on it and you know driven and sort of businesses that are in a state to push the use of agents on it which is often times times, you know, disruptive, expensive, you know, it's it's like it's not an easy thing to doing it. And and so those organizations are are ahead. And we're seeing a lot of other organizations, you know, and we engaged with a lot of them here in which you'd sort of say this organization uses

33:21 agents,

33:23 but really when you pierce through and we talked to a lot of these orgs, there's like a cohort of forerunners that they are uh embracing agents and some of them are actually very sophisticated in how they use it. But then the vast majority of the organization is still very nent you know everybody's using code completion and and the likes. So anyways that's so that's one is like I think there's a difference between the sort of the 10% stat which is also about a year earlier.

33:48 Oh yes

33:48 but uh between that stat and the 80% stat just between organizations and meta is still a very AI forward it is a large organization but it is an AI forward very driven very technology organization. The second thing is I think we don't yet have good enough metrics to say what does it mean that you're using agents uh and I feel and I see that the majority of uh developers today that are using agents they're using agents for smaller tasks. They're still very supervisory.

34:20 So they're they're they're delegating but they're delegating small capabilities. They give very detailed instructions of like write this piece. They use it for things that are immediately available or immediately useful like learning some use of an API or um you know just sort of building things that are very repeatable. They build small things with it.

34:41 What are your thoughts on adoption through uh looking at organizations costs of using models or tokens from using models? Some people are more and more doing that these days. Yeah, there's a very controversial view there, right, of whether you should basically require people to spend more on the models. Um, I think it's a very shortlived thing. So, I I do see it today. People talk about how uh that that measure is is like just a means of driving adoption. Um, I think everybody that says it has a bit of an apology, you know, with it. Um, in part because I think the models are actually getting very expensive. And one of the competencies of being good at using agents is knowing when do you need to use opus and when can use sonnet and when can you use an open model you know those are actually part of the sort of the strength right and when do you give the right instructions so the agent doesn't need you know a thousand turns to get it right but it can actually you know get it get it done in one go and and so there's actually an efficiency

35:42 element uh the metric that I like the most and I think maybe has a shelf life for a bit uh is the notion of merged multiple requests. Uh and then some people, Lemonade is a good example, actually introduce the complexity measure on top of the pull request. So for starters, just

36:01 just kind of assess, you know, rate of pull request merge rates, fine. Maybe these are small pull requests, but as as long as you kind of have established a point in time and you're kind of seeing the trend moving up, uh that is I think a better measure for developers to say, well, how many poll requests have you open and how many were merged? And from that point on, even if you cheated at the beginning and you can kind of get to like some small poll request because you just started pushing things in that way, you can only do that a little bit at the beginning and then you still have to accelerate that number over time. Uh so I think that measure is probably a slightly better one.

36:34 Mhm. So, so next up after thinking about a potential future where you know 80% plus in 2027 of of enterprise developers

4 Highlight · AI amplifies engineering practices indiscriminately

36:45 and developers generally are going to be using AI. Um we're going to switch to biga uh in episode 69 talks about how AI amplifies everything. Good engineering practices bad engineering practices it does them equally as well you know whatever it sees. Let's hear from

37:02 Bakita amplifies indiscriminately, right? So if you have like a bad setup, you might just amplify that bad setup, right? And if you have a good one, everything might go well, right? Can you actually in your organization with your pipelines, with your processes, can you support an increase in throughput? I think a lot of organizations are still underestimating this foundation that you need.

37:25 Really interesting because it kind of means, you know, you can't just put your cowboy outfit on and just create software. you still need to use good practices. You need to guide and have the right context to be able to get an agent to do what you want the way you want to do it. And that's really kind of like the crux of what Bit is saying,

37:43 right?

37:43 Yeah, I agree. I I feel I really like the idea of AI as a as a multiplier, an amplifier of it. I it actually makes me remember one of the early episodes we had with with Dez uh from Intercom, one of the co-founders of Intercom, who talked even in the support world of about garbage in, garbage out, right? or you came in and if you were to say like hey just replicate what's in you and like learn from my knowledge base and quickly you learn that well your knowledge base has a whole bunch of outdated stuff and kind of things that are no longer correct. It's the same for your codebase. If the agents just replicate the codebase well a lot of the things that you've done before in your codebase you don't actually want to repeat. So uh you you really have to be uh mindful of uh of that. Um and so I I think we we've kind of gone through that phase a little bit in the AI world in which the thinking was we will just learn from your code

38:30 uh and we will replicate what development and you don't actually hear that uh much anymore. Um I think today it is a lot more about context engineering and about the definition of what is the correct behavior to uh to build out.

38:43 Uh and then you want to codify that and you want to be consistent and first and even if the agents are building things from scratch then they might build it um you might want them to be explicit about the way and the decision that that they've made as they've built it so that future versions of that agent uh is able to to build them out. So there's there's more and more of of those types of like definitions of what good looks like of history of decisions you know of uh building those out. I still think that there's a a gap which makes sense just with time around looping in the learning into that of saying fine you've made these decisions what would you see in the deployed system that would make you think well that decision was maybe not very good or it's no longer correct can I can I go and adapt so we still have to evolve that

39:30 and also and also I guess what you need to change as a result did that decision come from context that needs to be changed or is that a human decision that needs to be changed

39:39 it's it's pulling feedback or data in and understanding what was the root cause of that issue.

39:46 Yeah. Yeah. I I agree. And actually, it's interesting to think about also making the migration to the change and making that be incremental. You know, I I saw a recent uh post from Intercom about how they manage skills and all that and they have a a test generator or like something that tests their systems as a as just sort of part of software development. And when it finds a bug, it looks for that pattern elsewhere in the system and then it modifies it in those areas as well. I don't know how thoroughly and autonomously they do that. But I think that type of pattern has to be there. It has to be a sort of self-healing, self-improving system. Um, and it all comes back to that like the whole SDLC, the end-to-end SDLC has to be handled by the agents and what we need is we need to really focus on what correct behavior is and what are the mechanisms and what are the things we need to review versus can you know be made autonomous around modifications to that context and then how how does the context meet the code when you modify the context doing it and it's interesting because people I think what we're seeing today is like we're moving so quickly but nobody has had

40:48 substantial mileage with this yet. And so people haven't experienced yet what happens when their context goes stale. You know what happens when you know like a practice like we think today about make sure that it applies the right practices but what happens when a practice changes right when a security issue changes. And so that is that is very interesting

41:10 and also that the fact that we're never going to be as manual as we are right now with agents. Everything is, you know, agents are getting better, but we're also trusting them more. There's going to be more autonomy than in future than than than less.

41:23 Yeah. Great.

41:23 And and and in terms of that, we need, you know, things to act correctly and we need things to to to to be as we need it. And and that's so so important

41:32 indeed.

41:33 Hey everyone, hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about Agentic development. Whether that's talking about the latest tools, the most efficient workflows, or defining best practices. But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button. It really does make a difference and lets us

Balancing automated stability with emerging security risks

42:03 continue to improve the quality of our guests and build an even better product for you. All right, back to the episode. Okay, and then one of our actually one of our many most popular episodes in terms of uh in terms of the number number of views was Olivia Pomel, Data Dog CEO, who talked about uh the the wonderful dream of no one waking up at 3:00 a.m. because of a critical situation. Let's hear from Olivia. At the end of the day, we think just everybody will want automation,

42:34 right?

42:34 Every single company in the world will want automation. That's the dream, right? The dream is you never have to wake up in the middle of the night ever again to fix an issue. The issue has been for you.

42:44 And so the bar for precision needs to be very high on the on anything that has to that relates to mobility. And the good news is that I would say three four years ago being completely right about root cause most of the time was science fiction and I think now it's definitely within reach and we can clearly see on the horizon the moment where the this technology is good enough to be put into the hands of customers on in in a large amount large number of situations.

43:14 What's the worst that can happen guy? Yeah, I think you know we spoke a little bit about root cause analysis and how that is the sort of the dominant case and I think it it just really lends itself well to you have to sift through a lot of data, you have to build those and if you got it wrong then you've wasted time as opposed to you've taken out a system. Uh and so I think I think those are useful and they have a little bit of the coding assistant version over there which is like surface candidates for me right be an assistant for root cause analysis now that technology is improving to be able to uh to even like get you good good inklings right of what that is you know with context and so I do think that this is you know we touched on this already you know very much the sort of the dream of not getting woken up there's a there's a nightmare version of this that we're seeing and actually has manifested a lot recently with all the supply chain attacks which is uh what we're also seeing is that attackers are becoming increasingly agentic uh and there is a marked rise in uh fishing attempts you know that are scaling and are going downstream and going into SMBs we're seeing agents finding vulnerabilities in

44:15 open source libraries and so there was a big push around uh a variety of supply chain vulnerabilities supply chain vulnerabilities are multiplied in in different ways you know in part is because attackers can find vulnerabilities more easily uh and that on top of that they can actually fish like the Axios vulnerability that happened right now had a fishing element which sort of combines you know the other capabilities of AI but then also the agents are pulling in a lot of these libraries and they're done with less supervision so there's just this trifecta of of of whatever multiffactor you know of uh of uh of reasons that drive it uh and so I think in in the in the DevOps world as is often the case right in the dev and devops world it's a very positive message it's like you can do this and we can all sing kumbaya together and you know sleep well and doing it because the systems will will kind of be self-healing and we'll build that and self-healing is is a DevOps term right that has existed for a while.

45:06 In the security world there's a bit of a you know if you don't then uh type of of of risk scenario which is we have to level up security handling to not just find vulnerabilities but actually fix them and then go all the way to deploy to production and that forcing function really again drives that sort of end to end speed. What I like about it is that it's actually a case in which you know you really you should embrace full end toend agentic capabilities so that you are able to respond this quickly. So again an extreme version of the the DevOps kind of cloud move in which initially it's like well it's insecure to deploy multiple times uh a day or whatever to have continuous deployment and then it became well it is insecure not to do that because the vulnerabilities will happen the systems do exist on it so the question is how quickly can you respond and reach out and so I think we'll see we'll see a lot of that and I would expect this year to be

46:04 security [clears throat] to be very much at the forefront as agents are um are being used for much more sort of real production cases. Uh so you you'll see both of those. You'll see the security issues because agents were not used in uh doing it like people were just sort of slow attackers are becoming faster than defenders

46:23 and there's a lot of understanding and awareness of this in the security industry but not real solutions yet. And the second is you'll see security issues because of of failures on the agent side and unreliability. We've seen some leaks from Enthropic, right? Whether it's clot code that they've quickly clarified is a human error. It's like it's the button pusher human uh who's being, you know, thrown under the bus. And we've seen sort of the leak of of Mythos somewhat ironic, you know, like super cyber security model having a cyber security leak on it. But there will be more and more of those.

46:52 Yeah. And do you think, you know, for the 3:00 a.m. kind of wake up, uh, do you feel like that will turn more into a human in the loop validation versus actually, like you say, the important piece is the activity, the action, the root cause analysis, the fix, and getting all of that done quick. Do you still feel like there's, you know, amount of time where we need human in the loop to to build that trust or do you feel people will

47:15 Yeah.

47:16 be more immediate in

47:17 I think really the the the best setup is to make yourself error tolerant. And so what you want is you want to have an environment in which if the you let the agent do it but you build the infrastructure such that if the agent has made the wrong decision it is reversible. Uh and I think that's really what we aspire. I increasingly think about the mental model in which uh you know if if you think there were some things that were tolerated in waterfall and were no longer tolerated in the cloud era. Uh and the best teams in waterfall were automating things or finding issues early like shift left in security is a good example of that

47:49 and then they were you know no longer tolerated in cloud like you could not get by without you know shifting security left or by you know having automated uh tests of some capacity on because you just couldn't deal with the speed. I think there's similarly now a few things that are tolerated in the cloud era and will be not tolerated in the AI era largely due to speed. Uh and many of them come down to the same pattern of these were human actions, human reviews that had to be done and the best teams were automating these things in the cloud era but you know in the uh AI era the average team needs to be doing that otherwise you just it would not be viable for you to build in this world. Let's switch gears a little bit now to Matty Statashevki who is the uh the the CEO and founder of 11 Labs.

48:37 He has some opinions on how we interact with uh AI. Let's hear from Matty.

48:42 Like in general we think voice will be the future of interactions, the digital interactions of how you interact with interfaces of of the digital world and it can carry so much more emotion so much understanding than text. So it it will read your mind to an extent but I know like you know maybe go to Elon's kind of a neural link or some scary options over there. Um yeah I mean I think voice is definitely prevalent on it. It's interesting because it's less about just typing speed and more about our human tendencies when we speak over here. There's there's there's a you know there's you know you know you know and and all those

49:19 the audience don't know that that gets edited out guy. So you know in our automation technology makes that go away and when you when you type something then you bother fixing these things you pause you think. So it it's actually not just related to your time of sort of typing down

49:37 it is a lot more uh kind of related to how how much do you need to phrase how much can you ramble on on it. And so it's been really interesting to see how voice and speech have become uh uh common in the world of AI uh in in the coding because coding requires that precision. You know when you code you can't you can't say you know you can have an a you know in uh in that interpretation but when you are discussing things right and talking to an agent then that's okay. the agent can remove those things and they can take that away and so it reduces the cognitive load. It just allows you to to ramble on. There's still value in precision when you're instructing something to the agent. But if context is managed well, in this case it's inline context. You ramble something, the agent might create some instruction files. That instruction file might be the thing that you've passed to a sub agent to implement things. Then it's

50:38 again if you use the human analogy a little bit this is the sort of the person the project manager right that you are talking to and you can ramble on to them and part of their job is to sort of streamline your ramblings to uh to be able to translate that into uh real conversations. So it it's interesting I think Mati is is right. It was clearly a little bit self- serving to talk about sort of voice uh being the future of it. You know 11 Labs continues to sort of shine and thrive and know since we've had that episode. Interestingly, they've actually expanded past audio and so now they don't just do audio. Uh but uh I I think I think you'll see more and more of that text in the AI dev world as well. You're seeing there's an interesting pattern in cloud code and I think it probably will be if not already mimicked by others to delegate to use sub agents a lot more. Mhm.

51:24 That to me is that analogy of the the the the front agent that you have there is the project manager and you're interacting with it and then it will delegate activity to sub agents to to do the work. Um and I will say that just like with project management, there's a caveat about losing visibility. So when you're in the terminal and you're giving those instructions, it's a lot harder to know what happens when you're when you're making that delegation. So it's interesting but I think generally correct that voice will be uh

51:52 and of course I think Claude code released a feature actually probably nearly a month ago or so where if you put there's a slash command now slashv voice I think or something along those lines where then it actually listens to the mic and you can actually put uh from your you know laptop and you can actually just chat with claude code and it feels quite natural actually for me I quite I enjoy doing that naturally with claude desktop actually anyway so it was a natural addition for me so I'm definitely in agreement. And it raises the like there's a a very it's a very different question to say will a thing happen like what voice kind of be more dominant than text and who would be the players that win in that space. So it is it is very interesting. I will say that as we as we record this you know there's been sort of recent updates on the cloud side in which they've tightened up a lot of where can claude be used and they really want their harness. want their interfaces. And so uh we're we're probably starting to toy a little bit with you know as companies get more capable and entropic right now is in that state do they seek to form more of

52:55 a closed garden around it and you know hey you just need to use our systems and run everything within our systems versus the other trends where you see for instance Nvidia committing to open code and open claw and uh you know building things that are very open and composable system uh which once again feels a little bit cloudesque a lot more web like and we had in an early episode we had uh Matt from Netlifi the co-founder and CEO of Netifi talk about the criticality of the open web and you know how do we how do we build over there so they've continued to invest in that as sort of being a composable infrastructure you know while you're seeing you know Versell actually like definitely uh supporting some elements that are open but also mostly investing in sort of the end-to-end experience and making a very good end toend experience within their kind of walled gardens, right, of uh of it. So, it's it it's a it's a just different questions. What will happen and who will win or what does it take to win are just very very different questions at the pace that this world moves.

53:56 Amazing. Well, look forward to chatting about what's next in the in the next 100 episodes, but for now, that's all we have time for. really fun to look back and kind of like, you know, think about some of the topics that were raised by such amazing guests that we've had on the podcast.

54:12 And I want to extend a huge thanks to to all of you listeners who are uh really kind of the reason that these guests, you know, come on to sort of have these conversations with us, right? And they uh uh and just all the feedback that you get and the suggestions for different speakers and different guests uh that we will have. So, thanks, keep on listening, subscribe if you haven't uh already on it. And if you have suggestions for guests that we should have in the next hundred episodes and beyond on it, send them over. And of course, you know, to all these amazing guests that have sort of spent their time sharing their learnings, their perspectives, you know, with with us, with you. Uh big thanks, very humbled, very much enjoy the conversations, learned from the conversations. And you know at this pace uh a lot of the kind of one of the critical elements of of being able to deal with this space of change is community is about sort of learning together sharing perspectives sharing these learnings. Um so we're very uh grateful for you for sharing on uh those uh those perspectives and we'll continue to provide a stage uh for that to happen.

55:11 Seconded. Well put. Thank you very much for listening and tune in to the next episode. Thanks guy.

55:16 Thank you. Hey