Cobots, Dark Modules and Architecting For AI

5,717 words · about 23 min to read

0:07 Uh normally when I do talks, I like to try and do talks where I can give you like advice or guidance or say, "I did this and it worked." or "I did that and it was terrible." or "Here's what's really great out there right now." This talk's a little bit different because this talk is mostly about the things we don't know. Uh everything's a bit uncertain, everyone's a bit unsure, everyone's a little bit worried about what's going on. And that's kind of what this talk is about, but that's not very interesting.

0:33 So, I thought I'd talk about robots instead cuz robots are cool. And there's something I think is even cooler than robots um because I am at heart somebody who is rooted in the past, I actually quite like modules. So, we'll talk about

AI productivity claims are vastly overstated

0:46 robots and modules and a bit of uncertainty. And it was kind of interesting coming from the talk I was just at, in Dylan's talk. And I also trying to explore a little bit this question of whether or not we're now operating at a new level of abstraction. So, the history of software delivery, uh professional software development from say arguably the mid-1950s has been one about working at higher and higher levels of abstraction. Initially, we would work at the machine code level, where we'd have to have a deep understanding about the underlying physical hardware, working though with fairly limited instruction sets. And then we moved over to assembly, gave us a little bit more power uh because we're working at a higher level of abstraction. And then we had multiple generations of systems and general purpose programming languages.

1:33 And as those new newer languages came along, we found ourselves being a bit more productive. I my first programming language I used that I was paid to use was Fortran 77. That is a programming language that was uh released in the year that I was born. Um I then moved to C, which is slightly more powerful than Fortran 77, arguably too powerful. Um and so then I was quite happy when I then moved to Java where it did things like memory management for me. But I found myself being more productive and getting more done.

2:03 There's a question though, is are we now operating at a new level of abstraction? Is this where we going? Not sure. And even if this is where we going, I think there's no certainty about what this would look like. This is not the first time we've been promised things. Um, I am old enough to remember being promised jetpacks. Didn't get a jetpack. Don't know about any of you. Um, I also remember claims that were made about the software development uh, industry.

2:31 Uh, hands up how many of you have got this book? Keep your hands up if you've read this book. Some of you put your hands down. Very, very honest people. There's a This is a collec- It's actually a fascinating uh, collection of essays, a large amount of which stand up really well despite how old this collection is. But this is in this book is where the myth of the 10X developer comes from. And we all know that 10X developers don't exist, right? What we do know is we have people that claim to be 10X developers, but they are 10 times more painful to work with than anybody else.

3:04 So it's 10X for some definition of 10X. Now of course around the claims of AI and software delivery, we haven't been told, "Oh, it's going to we're going to 10X." No, no, no, no. 10X is 10X is so

1 Highlight · AI productivity gains are closer to 10% than 10x

3:16 1980s. No, no, we're talking about 20X, 30X, 50X. The end of software engineering. This is it. You're out out of a job. We don't need you anymore. Um, turns out when you get down to it, we don't really get 10X. Uh, it's about 10%. Like give or take a little bit. If you look at the studies from both DX and DORA, uh, you look at things like the Whiplash Report, we're getting numbers that broadly speaking across the industry that we have surveyed with a mix of qualitative and quantitative studies, that this is more close to what we're actually seeing.

3:49 Now obviously there are outliers. There are absolute outliers out there and we've heard from some outliers already um at the uh conference today. Um but this is where we kind of are at for a lot of people. Now, 10% that that's not bad. 10% is actually pretty freaking good. Whether or not a 10% improvement in software delivery throughput uh justifies the largest deployment of

4:14 human capital in history, that's a question for another time, right?

4:20 We've got lots of graphs like this, numbers going up and to the right, talking about how much more productive we are. This is from GitClear's data uh data, which is actually pretty good. Um they have data from outside of kind of the GitHub universe. And what we see here is a graph that we all like to see, a graph where numbers go up and to the right. And whenever you see a number go up and to the right, that's normally a good thing. It's talking about how much more code we are generating, and that is a generation you know, that is a measure you can look at. But what is fundamentally the problem, right? We are being a we are generating a lot more code. Well, you already know this because I'm sure you were at the keynote and you've also got brains. And so you realize that just generating more code doesn't necessarily help if your bottlenecks are elsewhere or if increasing one part of the system creates a new bottleneck somewhere else.

5:09 There's a general problem, I think, that we've got, and I can't solve all of this but AI I thought was supposed to free us up to have more time. It actually the opposite seems to be the case. More people are working longer hours. We're seeing that coming out in studies. Uh the DORA numbers are starting to show increased burnout

Surging PR volumes lead to cognitive surrender

5:30 rates, more people spending time context switching. The data from the Whiplash report is especially damning in this area. Now, I can't solve all of this problem for us today, so I thought I'd focus on one aspect of it, which has been touched on by some of the other presentations we've had today, and that's like focusing on our unit of work and thinking about the PR. Now, I hate PRs. I think the whole process of PRs is dumb and stupid, but I lost that argument and you're doing it anyway.

5:58 Um but we accept that the the size and frequency of PRs are increasing. And that's causing us a bit of an issue, but we know this is happening because GitHub, which keeps what it's up, um gives us data about how bad things are getting. So, between April and August of this year, they went from 1.4 billion PRs to 2.9 billion PRs, and that's more than double. Even I can do those maths in less than 4 months. That's not necessarily good if you're the person who has to have all the infrastructure for this, and we see that's causing some issues.

6:33 Now, we're seeing the ability of AI coding agents to generate code and turn around PRs, great. Um and that's now creating bottlenecks elsewhere, and human review is often coming up as being one of the main bottlenecks that we've got. Just saw Dylan's talk where they talked about how they got rid of the human review portion of that process. Not saying that's right or wrong, but it's up for consideration. The problem we've got with um coding agents generating more and larger PRs is the LGTM problem.

7:01 The looks good to me problem. All right, this is I I I first encountered these words at Google. When you do a code review, if you want to get code into trunk, it had to be reviewed by somebody that actually was in the owner's files for the files you were changing. And if you sent a decent-sized code review, you'd get some good conversations around it. Um if you tagged somebody in who was an they'd just reject it cuz they don't like you. Um and if you sent them a really big PR and they were well, big really big change and they were otherwise busy, they would just type in LGTM, which stands for looks good to me, and they'd hit enter. And that would actually be set up programmatically to accept that change. And this is what happens when we get large change sets, we just yelled LGTM. I've heard this phrase referred to as cognitive surrender.

7:44 Right? AI is so good at generating content that we just go, "Oh, yes. Just go away." It's just almost like quiet quitting, but in a in a smaller moment. Now, of course, we are starting to solve this. A lot of people are now using AI for code review, either using commercial tools like Code Rabbit or building their own internal code verification, code change verification bots. Um so, that is kind of interesting. So, now we're in a situation where we've got AI writing the code, right? AI then reviews the code uh to and then AI will then change the code based on the review.

8:20 Right? It's like write the theme tune, sing the theme tune. It's doing a lot now. So, if we've got AI doing all of these things, um so, AI is writing code for AI to look at, uh it does then open up the question, which I think is quite legitimate, is um why are we looking at it again? Forget. And that's the problem. We often forget.

Interrogating the four real reasons for human code review

2 Highlight · Articulating why we perform human code reviews

8:43 I think the human review of diffs has in a lot of organizations become ceremony. And we may not remember why we did it or not. And that was why so, when I say, "Should you stop doing human review of code?" When people say, "No." They often just knee-jerk, "That's a bad idea." But you actually I think to say that, you need to articulate what you're doing a human review of code for. So, why do we do a human review of a diff?

9:11 Now, you might have slightly different variations on that, but I think broadly speaking, there's four main reasons why we have humans look at code. The first thing is just looking at it and going, "Yeah, that looks about right." Right? There is a bit of that that happens. You look at a bit of code, "Yeah. Yeah, that'll do." Uh bit of that. You kick the tires. It's also called kicking the tires, right? You go and buy a car and your dad comes with you, kicks the tires, says, "Yeah, four good shoes. She'll be right."

9:37 Hasn't looked at the engine, but it is a general vibe thing. Um and then we've got the We imported this idea of shared learning. Dylan talked about earlier about uh in the talk before about how AI is not a peer. This is kind of an interesting aspect. When I write a piece of code and I submit that and a human looks at that piece of code, we engage in this two-way conversation. Uh they might spot things that I could do to improve the change I've made, and those are things that I can then learn from.

10:06 But they might also learn from things that I wrote, and they might therefore grow and improve. We can have a conversation. There can be pushback around that, and that's fantastic. That's one of the things I liked about uh a code review. That's why I love pair programming, because I'm doing that all the time. So, the third reason we do a code review is checking that the change aligns to our sort of strategy and practice. So, we can say things like, "Well, looking at this change, you've put a whole load of functional code in the middle of our object-oriented code base. We don't really like that. It's going to be quite disconcerting and quite confusing to people. Or you put a dependency here on a new Amazon services, and we don't want to do that. We want to limit our you know, which of the AWS products we're using, for example." And those are things that do come up in code reviews as well. And finally, a code review is a very useful auditable unit of work.

11:02 Some of you operate in regulated industries where you have internal practices around this. This can be a useful point at which you can say, "This review was done. Here's

11:11 what we saw." Let's look at this now through the lens of uh the current way we develop software using AI to write a lot of our code. Correctness. Well, we've just established that with AI, we're generating a lot more code, and I would say the Mark I human eyeball has always been pretty bad at looking at the code and saying, "Yes, that is correct code." That's why we have tests both pre and post in production. So, I think a human review of AI-written code to assess whether or not it's correct doesn't really scale.

11:44 Right? Doesn't make any sense to me. Shared learning. Humans and AIs are different. Right? We keep falling into the trap of anthropomorphizing the AI agents we use, partly because that's what the vendors do, but they are not human being. They have nothing like the cognition system of we what we do. They're not even an it. They're more an amorphous blob of connected computers. When I give feedback on a code with bit of code I get from an AI agent, does that change the training of that model? Does that tweak the weights of the model? No.

12:17 Because models are trained over months and months and then we get the model we're running through an inference process. Best case scenario, I'm going to update some of the harness or some of the context. That's best case. And again, these change sets are big and quite verbose. So, how am I going to reason about them anyway? So, yeah, doesn't work. Alignment to strategy and practice? Well, I can automate a lot of that. I don't actually care as much if I'm not looking at the code about code stylistic stuff. And even then, I can automate that.

12:47 I can have checks that say how many lines of code do I have in a function? What kind of coding style? I can use things like ArchUnit to implement architectural fitness functions. Those things are kind of interesting, right? Lots of actually static checks that I can do to verify these things. So, I don't need to do that as a human being looking at it. And in fact, if I if I do those things with scripts, they become much more explicit about what we want in the system.

13:13 And as for an auditable unit of work, we can have an auditable unit of work without a human being having to be involved in reviewing it. You're working like in a SOC 2 environment, for example, you can do pre-approved changes. You do not need a human review to make it auditable. In fact, if you have a fully automated process around how a change set is verified, and you can document and reproduce what you did, auditors love that.

13:39 They love that more than a human being looking at it. Right? Whether or not you've had that conversations with your internal controls team is a separate conversation, right? Cuz there's what the regulations say and how your company may have chosen to implement those regulations is often the sticking point. So, if we take this argument, why don't we give our code to AI wholesale,

Evaluating software dark factories and third-party trust

13:59 reduce the load on developers, and allow us to to free up our brains to focus on important stuff? Hold that thought. Let's talk about robots, cuz I'm aware I promised you robots, and I hadn't delivered a robot, and I don't want to be like one of those people, so let's talk about robots for a cool. And until the recent Chinese Robotic Olympiad, I think most of us when we thought about robots were thinking about them as the industrial robots that we'd use in in the context of automation.

14:26 So, things like these robots. These are little blue things underneath the cages. These were Kiva robots that were purchased by the company that was bought by Amazon. Amazon now call these Hercules. They're actually used in their fulfillment centers. So, in a normal warehouse, when you were say sending off an order, you have the pick and the pack. A person would walk around picking things off a shelf, putting it into a box, and then the box would be shipped off. That's picking and packing. Uh in a fulfillment center like this, these cages move around in a in an area just for the robots, and they come towards where the picker is, and they take items off these cages and put them in a box.

15:03 Now, these robots are dangerous. They're actually quite big, and you know, they're moving around large cages. This whole zone here is a human-free zone. Humans do not go in that area, cuz they might get squished and died. So, there's a safety zone around them. So, if they do maintenance, they shut those floors down, and the human being then goes into them. This is kind of a more clear example of an industrial robot. Industrial robots are not designed to work in close contact with humans.

15:29 This one here, this robotic arm, is doing some automation, has a get a fence around it. You open that door, it will shut the robot down. Because the ergonomics of that robot are not designed about safely operating with a human. They are designed around, you know, doing stuff without a human in the mix. When you take these ideas to a logical conclusion, you end up with a concept of light soft facilities or dark factories. These are facilities in the real world where the products or the the items being created is a fully automated process.

16:00 They're often called dark factories because there are no lights on because robots don't need lights, right? Um unless you're, you know, a lot of them are using things like lidar and stuff, so they don't need lights. Now, I went looking for some nice photographs of dark factories for you. Turns out most of the fancy photographs online are AI because the lighting's not good in a dark factory. This is a real picture of a dark factory. This is in Tamil Nadu. Now, they are not These are not as common as people would give them credit for. Uh this particular one here is using the semiconductor industry. Most of them, I believe, are kind of normally doing intermediate products anyway. But we've got this idea now, which is okay. So, why don't we, as a human beings, construct a software dark factory effectively? So, we define how the factory works, and then we put something in the top, plans or specs or whatever you want to call them, and the and the machinery creates it as a black box, and out comes working software.

17:02 So, AI creating software in these dark factories. Now, this, of course, is quite a scary concept, especially if you're not using a lot of AI-written code in your production systems right now, uh or you're not at that you yet where you trust what your AI is producing cuz it does feel really risky. This same idea for as many of you I suspect seems absolutely crazy. Can't do that Sam, it's nuts.

17:28 All right, let's change Let's change things a little bit. Let me ask a question. Has anyone here ever outsourced some code to a third party to write to be developed for you? Okay. I'm going to say it's about almost 10% of the room. Uh now put your hands back up again if you looked at every single line of code you were given from that third-party vendor. I've got one hand. That's a This is a very honest audience. Sometimes I get more hands than that. I've been on the other side. I've worked in with teams in China and India doing the outsourcing for companies. It is incredibly rare that they ever put contractual obligations around code quality for example. We would do good jobs cuz that's what we did. It wasn't checked.

18:05 All right, I not many of you have outsourced systems. Let's change it a little bit. Who here has used a third-party library? The rest of you don't do software development I would imagine. And again, put your hand up if you've gone through every single line of code in those third-party libraries. No, none of you? That's interesting, isn't it? So we've got this kind of interesting challenge here which is ah, we must review the code that we write so the people, your colleagues who you know and theoretically trust because you hired them, you don't you trust them as people but you don't trust their code so a human being has to look at it. But these people on the internet that you've never met, ah be fine. Stick it in YOLO the whole thing right?

18:45 But we've also got the reality that not all code is equal. The systems we build aren't like these sort of homogeneous systems. The code we have does different things. It has different levels of criticality. And yet we still kind of especially in the current day and age when things are really in a fluid state where we don't know what software development's going to look like next month let alone in 5

Modularity lets you choose where to use cobot-style AI

3 Highlight · Using cobots as a model for AI code delegation

19:08 years from now, we still want to be forcing all our code through the same pipe, but code is different. The things the code does is different, different levels of importance, different levels of risk, different levels of criticality. So, we shouldn't really be treating it all the same. Come back to robots. There are different types of robots. Industrial robots not designed to work with humans. There's another type of robot called cobots. And these are robots were designed to work in collaboration with human beings. There are lots of different types of cobots. Probably the most extreme example would be something like this. This is a robot designed to help a surgeon in surgical procedures.

19:48 The surgeon is very much in charge. The robot is there to help and support. This is a particular example is a simulated laparoscopy. That's a world away from a robot that is so dangerous you have to put it in a cage. This is something that's been designed to work closely with a human being around a high-risk procedure. This is something where the robot itself is dangerous, but the work it's doing is probably quite simple.

20:14 So, if we take this metaphor, we can look at a model where for really important code, for the stuff where we're still not sure about, where we've got an uncertainty about like how can we trust the AI, we can still continue to work hand in hand with our AI coding assistants and our agents. But for other less critical code, we just give it to the agent. And maybe the agents we just give it to a different.

20:41 Maybe the models are different. Maybe the harnesses around them are quite different. So, if we start thinking about our systems less as one big box and more as a modular architecture, whether or not you decide that those modules should be on different computers or not, that's not an important question at this point, you can think about the criticality of these different parts of your system. Where can you afford to take risks? Maybe some parts of your code base is so critical or important or hard to maintain or whatever it might be that for right now this is where you need to work hand in hand. You need to be in the detail finding out and working out what's going on. For other bits of the code, maybe you should give it to AI. And then of course you've got the bits of your system where you already let those third-party libraries handle it for you.

21:26 Implicitly, you've already got different ways of working with the code in your system. But I think we often don't intellectualize it like this. The challenge is often you don't have a modular structure which allows you to make different choices for different parts of your system. We see modularity

21:42 like this more as a runtime concern. But I think it's massively a development time concern as well. But now we can move with a modular architecture to outsourcing our modules to AI. Why do we outsource code to third parties or to libraries? Right? A lot of it comes down to to trust.

22:08 We trust them enough. Right? If I'm going to uh go and get a new library that I'm going to improve it that's important to my application, for example, I'm going to pop over to GitHub. I'm going to take a look at how often is that thing forked? Is there activity around it? How many people have starred it? I'm going to go take a look at the issues. A library with zero issues, I'm worried about cuz that implies no community around it or a or a maintainer that just closes tickets straight away. Which is something that the core does actually. If you look at core code, they're always closing issues that aren't fixed.

22:40 But I'm going to go look at the issues. How are they responded to? What are the people like behind the project? It won't take me long, like maybe 5 minutes or so. Like do I get a good vibe? I know this is vibes people, you know. Yeah. And if I'm going to outsource code, I'm going to talk to people, the company. So then the question then becomes is how much do you trust your AI agent? Or more to the point, how much can you trust the output of your AI agent? We're quite fortunate if you think about the software delivery part of the process, which is we are getting a non-deterministic process to create a deterministic output.

23:15 So, how good are you at validating what it does? If you're really good at validating the output, you're going to be able to trust it more. There's a completely separate conversation about do you trust the vendors, cuz the answer is absolutely no, you don't. And if you do, you shouldn't, right? Um you absolutely need to be a in approach where you're being quite open about the models you use, and experimenting with different harnesses, because something bad's coming. I can't tell you what, but it ain't going to look good, and you need to be prepared to have a to to you know, turn on a dime.

23:46 So, try different models, try different vendors, start playing around with the harnesses. I've got a theory that things like pie are going to be much more important in the future than they currently are. So, I come back to this question. Are we at this new level of abstraction? I think I don't know. But I think if we do want to get to this point, we need to start working out what we need to do as individuals, as teams, as organizations to grow that trust.

24:14 Because until we can improve the extent to which we trust what these things are doing, and to an extent by extension the the companies behind them, that's going to be the only thing that's going to make us more relaxed in reducing our human involvement. This process here is very different to a Roomba robot, for example. A Roomba robot is a type of cobot. These robotic hoovers, they are designed to operate safely in the same space as human beings. I would hate to think what a robotic vacuum would look like that wasn't safe to be around. But if I guess this goes wrong, it's not the end of the world.

24:51 But we have to trust these things quite a bit if we're going to

AI must free up time to be more human

24:54 implement these dark factories. But I keep coming back to this idea of freeing time up, freeing time up for us to be more human. Like when I'm thinking about what does that mean, where is the human still vital, I come back to this paper by Peter Naur, which is on the theory of programming as theory building. He said the essential part of any program, the theory of it, is something that could not be conceivably be expressed.

25:23 But it's inextricably bound to human beings. When we build software together, it's about the shared mental model that we create and evolve and iterate. Requires us to talk and communicate and collaborate. This article was from '85, I believe. He talked about this in the context of programmers. Um I think nowadays we would extend this to be everyone involved in delivery, including product. The act of creating something is us working together to create that shared mental model about what we want to build together.

25:54 And then there's what we get in production. There's always conversation about whether or not code or specs are the source of truth. They're both garbage as sources of truth. There's only two sources of truth I care about. It's what's in people's brains and what's in production. Everything else, arguably, is noise, right? So we need to actually use these tools to spend more time talking. Not less time talking. Not less time running off by ourselves.

26:20 We need to use AI to spend more time being human. As an architectural level, right? If I'm thinking about it, I want to spend more time thinking about the boxes and arrows in my system. These are the things which are often hard to change. But I can think about the interface around my modules effectively as a user interface, both of human beings to interact with, but also our agents as well. These units then what are these units? These become a kind of a concrete implementation of that mental model that makes it easier for us to understand how our system works. We need to think about this stuff. The shape of the boxes.

26:57 I don't think we need to worry too much about what happens inside them. I do think that actually given that we don't know what's happening, we don't know where software delivery is going, that a modular architecture is a great hedging instrument. It gives us the space to try different things and to experiment. But I do think if AI is going to work for us, it's got to it's got to free us up to be more human, not less human.

27:23 Not as just people who feed specs into machines. There was a great uh conference I was at a few weeks ago in Dublin. And it was uh ShipItCon. And it was a conference all about friction. And it was actually inspired by the book Frictionless. And the and the expectation was all these talks would be about reducing friction. Most of the talks were about putting friction back in. Right? And there was this fantastic talk by uh Jennifer Cox called Shipping Fast, Burning Out Faster. And she was looking at the human impact of of AI in software delivery.

27:58 And at at the closing part of her talk, she says, "We need to make sure that we ship fast enough to care." Not just caring about the product we're creating, is it good enough, is it safe, is it secure, but also to care about our people. So, ship fast enough to care. And it's up to you to decide what caring means for you. Just to summarize, if you are using doing a human review of code, that's great.

28:24 But have a conversation with your colleagues. Why are we doing it? What's the purpose of it? Break it down. Be really clear. You may keep your human review of code, but maybe now you're going to be more focused on what you're doing and why. Uh have a look at whether or not there are boundaries in your system where you could try different working approaches. You might not be ready to say give up on human review of code, but maybe you want to try out some different ways of doing spectrum development, for example. We'll scope your specs to a module in your system. Give it a go. See what happens.

28:57 If it doesn't work, it's just one bit of your code base. You can try something else elsewhere. Do be very intentional in how you use AI. Have conversations with your teammates about why you're doing what you're doing. Don't just grab the fancy tools. You've got to find a a way to still give yourself rooms for experimentation that keeps you within kind of what your organization wants to let you do. But doing the same thing as before, I don't think is a a legitimate option anymore.

29:25 But more than anything else, free up time to be more human. Um that's me. There are links to the the deck here. I've also written a new book, which has got nothing to do with this talk, but please buy it anyway. I've got a mortgage to pay. If people want information about the book, I've got cards to hand out. Uh but thank you so much indeed.