Why Faster AI Development Often Increases Rework | Cian Clarke

10,743 words · about 43 min to read · YouTube’s automatic captions

0:00 The challenge with incomplete requirements going into a model when it's building a software system is that ambiguity at a minimum becomes some sort of hallucinated feature that the model has made up and potentially becomes masses of rework and wasted tokens and ultimately real kind of code smell in the repository as well. So the ultimate artifacts winds up harmed as well. And so I think of it as as almost like context engineering on steroids for software projects and making sure that the model is as well as possible grounded in the what that it's building.

0:31 Be able to give that steer not in a one-shot prompt, but in some documentation. That's where I see spec driven and BMA being really really strong. What does BMA mean to you? Build more architect dream. It's this really interesting open-source project. What it's about is giving a model the best context possible to go and autonomously build software.

0:58 Before we jump into this episode, I wanted to let you know that this podcast is for developers building with AI at the core. So whether that's exploring the latest tools, the workflows, or the best practices, this podcast is for you. And really quick ask, 90% of people who are listening to this haven't yet subscribed. So if this content has helped you build smarter, hit that subscribe button and maybe a like. All right, back to the episode. Hello and welcome to another episode of the AI Native Dev. My name is Simon Maple and I'm your host uh for this this episode and uh today we're going to have a really fun chat about something called BMA and the BMA method. Um so this is I I believe it's called a Build More Architect Dream, which is a which is a fun way of of really talking about a spec driven approach to to to building software um where we really think about specifications first and then drive the code uh from there. It allows us to really think about how we create uh our intent versus thinking about the how too early. We capture those behaviors and

2:00 those constraints uh very very early in the process. And taking us through this is uh Kean Clark uh from Nearform. And Kean and I have done a a couple of things including actually uh we we we we shared the stage at AI Native Dev uh oh

Nearform adopts spec-driven workflows for AI delivery

2:15 sorry, AI Native Dev Con in New York in May and we also uh we also did a a meetup very recently as well in London. Um Kean, welcome to the podcast. How are you? Yeah, thanks so much for having me. I'm doing well. Brilliant brilliant. And so you're ahead of AI at at Nearform, right? Tell us a little bit about what that entails. Yeah, absolutely. So it's really about realizing customers' ambitions in the world of AI. Uh and more recently of course lots of generative AI. That's been something that's been keeping us pretty busy. Um and so that might look like, you know, unearthing a bunch of proprietary knowledge that some customer might have in their product to their end customers. It might be, you know, building some sort of agentic system to make their teams go faster.

2:56 Um but of late we've also been thinking an awful lot not just about what we build in the space of AI, but also how we build it. Yeah. Um and so Nearform sort of came of age in the era of Node.js and that's been sort of the technology we've been known for. Um and actually we've been thinking a lot lately about AI native delivery and and delivering with AI models and the enhanced capabilities of coding models and spec driven. And so that's really brought our paths together, I suppose.

3:24 Um since we are very interested in all of the tool sets and and ways to to to build software artifacts through AI. So, you know, more about the how we build as well as what we build with AI. Which is super interesting cuz like, you know, there's a lot of obviously kind of like you you know, consultancies and people like that who who are really not adopting AI in a very interesting way, but you're really kind of like leading a lot of the way with with a very you know, spec oriented and and and future thinking path of building with AI. So we'll explore some of this um in just a second. Let What do we first talk a little bit about about BMA? What What What's What does BMA mean to you?

4:00 Yeah, absolutely. So to me it's sort of the best approximation we have right now of what a comprehensive spec driven workflow looks like. Um and so it it's, you know, the the there's a a sort of a tagline attached to BMA of Build More Architect Dream, but it's also sort of named by the the guy who created it, Brian Madsen. So BMA. Um and it's it's this really interesting open-source project, which I think is quite important to us at Nearform since a lot of our pedigree comes from the world of open source and and contributing upstream to things like Fastify and Node core.

4:34 Um and so yeah, the fact that this is an open-source framework that is also open to the selection of model vendor and open to the selection of IDE tool set as well is something that really really appealed to us. Um and in my head actually at its very core what it's about is just like really intelligent context engineering with a model. Um and that's kind of what spec driven is all about. Um at least in my head, you know, giving a model the best context possible to go and autonomously build software. Right.

5:05 And it as I say, best approximation I've seen to date of how to go about doing that. And I didn't know that about the name, the BMA. That's That's Brian Madison, is it? That's it. Yeah. Mad Madison Madson. I think it's Madison.

5:16 Madison, sorry. Madison. So so it so so it started as a name and it was searching for an acronym. That's what That That really I think so. Yeah, exactly. The acronym now is uh is a bit of a stretch, I feel. But Now the Build More Architect Dream makes a ton more sense. I was looking at it and thinking, "Wow, where where where did this come from?" But now it all makes a lot more sense. But it's it's a catchy name, yeah. Um and it started with this person, but you know, there's a huge suite of contributors now on this this tool set. And it's this really vibrant open-source project that's gone through a couple of different versions. Um it's got a really cool the V6 version that's just released has a bunch of really nice developments in terms of uh being able to select the like the scale of project you're working on. It's got a module system where you can bring your own roles. So it's it's really evolved massively over the last kind of couple of months. Yeah, yeah. Let's Let's get into that a little bit. But I guess the one question that I'd love to ask cuz I know Nearform, you know, do a lot of work in kind of like the the discovery of various, you know, a lot of the research and discovery of various tools.

6:12 I know you're you're big fans of Keiro, for example. I know we've talked in the past about about using Tessell for, you know, discovering context and learning about, you know, how agents can be you know, much better enabled. Um where did you Where did you first learn about Where did you How did you kind of like go through the discovery period with with BMA? Yeah, so it sort of came out of our own internal discovery process really. Um and we would do very frequently in consulting this kind of discovery process that we call ignite, which is about aligning a bunch of stakeholders on what it that exactly it is that they want to build.

6:46 Um and you know, in in the world of consulting we call this discovery, but like it happens in every company. Basically a bunch of folk getting together and um talking about what the requirements are for the thing they're looking to build. What's really important, like what levers within the business is it going to impact. Um and being able to document some of that stuff, you know, really really well. And we noticed that we could use things like AI note takers within the meeting to be able to capture a lot of that context and then distill it down into playback decks and also distill it down into requirements documents.

7:19 Um and so it was the folk who were facilitating these sessions um that observed that actually you can get really good requirements documentation and backlogs off the back of these meetings just talking about how and what it is exactly that we want to build. And I know the folks in AWS do this a lot with their pre-read time in meetings when they're doing their PRFAQ's. And now they've moved to this term called squad mobbing. So this is, you know, really big, you know, methodology in industry as well. This idea of gathering requirements um as a group and then using that as an artifact to then later go and build.

7:54 And so we kind of spotted, you know, we had been vibe coding prototypes that we would later throw away and because that's what you tend to do with an artifact that's vibe coded. But actually like could we ground the the the process of using AI to develop with these artifacts?

8:09 [snorts]

8:09 And that kind of aha moment happened probably about 6 months ago and it wasn't me who had it. It's It's a member of my team, James, who who helped helped out there and discovered this tool called BMA along the way that is really really good at taking some of that documentation and using it to hone the output of a model. Um and so I discovered this spec driven thing and as a pretty like skeptic, you know, vibe coding skeptic, that this was sort of a revelation for me because all

Vague requirements cause massive AI rework

8:36 of a sudden you were actually able to steer what the model did. And yeah, that was that was pretty powerful, I think. Yeah, and a lot of the things that you're kind of like talking about here, whether it's spec driven or BMA, they're they're really they're really kind of like smart and and I guess, you know, they're solutions that are well thought out. But I guess when we when we go back to the root

1 Highlight · Forcing vibe coding to grow up with guardrails

8:55 problem, what what would you say is the What would you say is the need for these for these types of solutions? Yeah, for for me it it came down to like forcing the the world of vibe coding to sort of grow up a little bit and have some guardrails. Uh and so distilling that um you know, uncertainty with what the where the model would would would fill in the gap, instead have stakeholders fill in the gap and make sure the requirements were actually complete.

9:20 Um because the the the challenge with incomplete requirements going into a model when it's building a software system is that ambiguity at a minimum becomes some sort of, you know, hallucinated feature that the model has made up. Uh and potentially becomes masses of rework and wasted tokens and ultimately, you know, real kind of code smell in the repository as well. So the ultimate artifacts winds up kind of harmed as well. And so I kind of think of it as as, you know, almost like context engineering on steroids for software projects and making sure that the model is as well as possible grounded in the what that it's building as well as the how you want it to build.

9:59 Being able to give that steer, not in a one-shot prompt, but in some documentation. That that's kind of where I see spec-driven and and BEMAD being really really strong. And what's really interesting here is kind of like when we think about what, you know, the way [clears throat] BEMAD goes around trying to solve this. And so and specs, you know, if we if we think back well part well before kind of the AI boom here, you know, we were doing the we were thinking about these types of these types of things back in the BDD days and things like that. And it's interesting that when we talk about the problem now, the problem is is is very, you know, the problem definition is peppered with AI terms. And and I think like now the problem is actually far greater because of those types of things. So a lot of things that you say about, you know, the issues around vibe coding, those types of things, they magnify the problem which, you know, like you say, vague requirements and things like that that we previously had. It's just magnified so much more because of the additional complexity that that that that that vibe coding and these types of agents can bring along as well. Yeah, like as engineers, if we had a gap in

11:01 requirements documentation that seemed really obvious that would impact users, you know, we would go and ask a clarifying question of a product manager of somebody who's a stakeholder on the project. And we'd be a little annoyed by it. Like, come on, we're mid-sprint, why didn't you think of this? But we'd fill in that gap. The model just makes up, right? That's that's not a great outcome. Yeah. Yeah. Yeah. Sometimes it is, but sometimes a lot of the time it isn't. Yeah. Yeah. Um so so let let let's go ahead and and kind of like talk

11:26 take us through an example kind of like NearForm rollout, I guess, of a BEMAD project, I guess. I guess there are certain times when it's worked better and certain times when maybe you've actually learned actually, you know, it's not quite as good here. Talk us through, you know, one of the one of the most solid kind of BEMAD rollouts that that you guys have done. Yeah, [clears throat] so it's it's sort of to date become the tool of choice for dev teams at NearForm who are looking to progress things in an AI-native manner.

11:53 Probably alongside Kiro as the other strong contender. And all along, the observation I had had was that it is not a great tool for actually kind of rapid prototyping and building, you know, a functional prototype that you plan to throw away at the end because the process is quite heavy. It has actually made great leaps and bounds in V6's release in terms of enabling you to say the sort of level of project that you're looking to build.

12:19 And if you're looking to build a prototype, it simplifies the workflow. Haven't had enough hands-on time with it to to form an opinion but traditionally rapid prototyping still reaching for bold.new or something. But where we're looking to actually build some sort of MVP and something that's actually shippable in the greenfield, we found it absolutely fantastic.

12:37 [cough] [clears throat]

12:38 I think there are probably some challenges to date in the brownfield, although it is the best thing I've come across for working in large brownfield codebases. Nonetheless, you know, there there are challenges with that. But for greenfield projects, it's been fantastic. And the way I kind of try to approach this, so my my background is not in consulting. I'm relatively new to the world of consulting. I've mostly worked in product companies through engineering leadership roles. And I like to think about, you know, rolling out these tool sets to teams of developers through the context of like developer empathy.

13:12 So we talk a lot about customer empathy in the sales process, but I think developer empathy is actually a very important thing to recognize as an engineering leader. And it's like this, when you were, you know, when you were hands-on keyboard walking in the shoes of a developer, what stuff did you just hate? Like, what drove you mad? And so like, you know, writing infrastructure as code stacks only to have the the CloudFormation deploy roll back 15 times in a row and having to go back and delete stuff manually and just that that drudgery I used hate. Or, you know, fixing an automation test that's that's blown up because a pixel moved 10 pixels to the button moved 10 pixels to the right in the middle of the night. Like, that's really frustrating, not a good use of my time.

13:54 Even documentation. So all of those tasks, you know, how can we introduce tooling into the AI development life cycle AI into the development life cycle to make those things a lot less painful. So like, what would developers really gravitate towards, you know, in trying to automate away some of that pain? And to that end, you know, something that I wonder is like, do developers actually really enjoy writing documentation and specs, right?

14:24 Cuz this whole methodology is based on spending a lot of time with a markdown file. And I think some do, some don't. But but as I think about how to roll out, you know, both BEMAD and a broader suite of sort of MCP-enabled tools that

Software engineering shifts from coding to architecture

14:39 are that are connected into an IDE, I like to think about the sort of developer archetypes that are going to be interacting with these systems and what are they actually getting from it? What makes their life like that bit easier? Super interesting. And and yeah, it's it's interesting actually. I remember giving a I think I gave the it was the closing keynote at Devops UK. And I gave I gave a session and and about, you know, you know, what a future world will look like with AI-native development. And they came back they came up to me at the end of the of the session and they and they were like they you know, they had like worry on their face going, you know, you're you're projecting a future here where where developers just don't code. Is that is that true? And I'm like, well, you know, honestly, I I think you'll you'll you'll be hard-pressed to find anyone who will say at some, you know, code will always be a thing. And I and I, you know, while you'll have some skeptics that will say not in my lifetime, it's it's going to happen at some point. It's just a level of abstraction above where we are. The question is, I think, when. And and I think, yeah, it depends, I guess, whether a developer is so attached to

15:41 the coding or whether they're attached to the creation like, you know, the the the build, the complexity of of the problem-solving and the creation. And I always think people who people who care about the creation, the construction almost, the the how I build this thing in terms of how I architect it, how I think about various things fit, you know, integrating and building together, that's all that's going to be there. And hopefully we'll Yeah, hopefully we'll abstract ourselves on top of that pile still and we can still make, you know, have input into those discussions. But I think for people who are like fully attached to code, that's that's, in my opinion, certainly and I think a lot of people's, it's it's absolutely going to be reduced and and someday most likely I think pushing the whole, isn't it? It is. It's a very hard thing.

16:26 is definitely becoming less important. That doesn't mean that the quality of the output of that artifact doesn't matter. In fact, all of the best practices that we followed all these years are just as important as before. And, you know, having code that is linted such that you can actually read it and maintain it is really important. And having code that is incredibly well-structured is really important. And, you know, the art of selecting what dependencies that you're using, you know, don't definitely want to be picking the latest version version of Next.js as as recent events have shown us. But the the the art of like actually picking what modules, dependencies, architecture that you're going to use becomes the part of the craft that becomes more important in my head. And, you know, the actual line-by-line writing and production of code is sort of the thing that that that becomes less.

17:14 One of the funny things about NearForm is as a business we've always hired senior staff. So I think the average experience in the business runs about 8 years. But last, you know, 10, 15, 20 years of experience. And it feels like those types of of of developers are going to have a much more productive time in steering some of these tools, right? Like, it's it's going to be interesting to see over the next couple of years what that does to the shape of general software engineering teams.

17:42 One, you know, really unfortunate artifact we've seen of this AI-native era is probably the the the the kind of reduction in hiring pipelines and also what the what [clears throat] the future looks like for junior developers. It's it's become a bit of a challenging time and, you know, that that's something that probably bums me out a little bit. Yeah, and I guess there's a lot of kind of like, I mean, the fear, uncertainty, and doubt here is quite high in terms of what does this mean for junior developers? But I guess kind of like my opinion almost on this is is I guess senior developers are more likely to be able to naturally switch over because they just have that experience and they have that underlying knowledge. I think the path of a junior developer is just going to be different.

18:23 I think they're going to have to learn different things. And arguably, I could I could almost say, well, actually, by being a junior developer, you're going to be naturally more AI-native because it's that's the path you learn. And so we're going to have probably have this this switch over where the junior developers are actually going to be somewhat more reliant but also more more happy, more more more they will choose to use AI probably with a lower level with a lower bar than more of a senior developer. So they're probably jumping into it more.

18:53 But that senior developer today certainly does have that kind of like more of that knowledge, particularly around when they're writing specs, what they want and why they they know why they want it that way. Yeah.

19:02 [snorts]

19:03 And I'm even not not as worried about this notion of, you know, all of a sudden teams just being seniors and the pipeline of junior talent drying up. I think the reality of market economics is that nobody's going to let that happen. And the need junior team members is going to be very, you know, very quickly solved by you know, hiring pipelines and so on. That that's going to have to be balanced at some point in the future. And there might be a minor reckoning, but we're still going to need junior developers that we grow into senior talent. Maybe that more progressive mindset of somebody who hasn't been doing this for as long is more open to the change that AI-native brings and actually it ends up being an advantage. Who knows, right?

19:37 Yeah. No, I very much agree. So so we talked a little bit about specs there as well. I guess when we think about from the

Specifications must be written for human review

19:44 developer point of view, the developers are going to be potentially writing specs, maybe manually, maybe through agents or or LLMs. The the specs ultimately will absolutely be consumed by agents. Um but we're at that at funny crossroad where kind of like both the agent or both an LLM and and an agent and a human need to need to read this and need to understand it and because one is generating code from it, the other one is trying to maintain it from the point of view of uh you know, making sure the features are are correct and well-written, etc.

20:17 Who's the who would you say needs to be the the primary stakeholder? Who's the person where this specification should be written for? A person. And I guess a little bit of a a little bit of a a comparison here, I guess, is when we write code today, we have two options. We either make it, you know, the most performant, most effective code possible, which means we're writing for a for a you know, maybe it's a JDK to or you know, the machine code that is that is the most performant, most effective possible. That makes it far less maintainable. So, we very often take you know, make decisions to make something more maintainable because we feel well, we're the ones who are trying to maintain this and read this as well.

20:57 Let's make it most readable for us and

2 Highlight · Writing specifications primarily for human review

20:59 then and let the let the compilers and everything else deal with a lot of the efficiencies after the sake. So, yeah, who is this primary artifact, the spec, uh written for uh you know, primarily? Yeah, that's an interesting question and I I like the sort of code optimization comparison. Um I also love that you have to get the JDK and there's somewhere your roots are showing through here, Simon. That's great. Yeah, my my JDK my JDK gray hairs, yeah.

21:27 Yeah, yeah. Um I think that ultimately, for me at least, writing the specs for human consumption is absolutely vital because that review phase is quite important. Um if you are writing outputs purely for the consumption of a model, um you're probably not really reviewing what it is that's that's been generated. You know, if you're generating a PRD off of a transcript and just blindly accepting it, in my eyes, you've probably missed the purpose of spectrum development in the first place. And actually, the honing of that spec, you know, however much some developers may not enjoy, you know, crafting markdown is actually the the really important part of this process.

22:02 Um in terms of like, you know, the model being able to interpret what's been written, I actually don't [clears throat] know how much of a consideration that's been in my experience. Um you know, models are incredibly adaptable, for sure. You know, if we're putting in like, I don't know, a PDF of a Gantt chart, we're probably going to have a bad time. That's not going to go well. Um but, you know, things like Mermaid JS syntax, which I've always done anyway, or or even just the the subset of web sequence diagram syntax, that type of thing in technical documentation is invaluable. If you've been If you've sort of come of age as a developer in the world of building specifications in markdown on GitHub, you're going to have a great time. If you've come of age in the world of oh, I don't know, um you know, Microsoft Word documents and diagrams scaffolded using those constructs in in in in XML, which is ultimately what the Word format breaks down to, probably not going to have as good of a time.

22:59 Um I've found that, you know, the kind of modern way the developers architect their their documentation, models are just really good at reading and that's a really nice coincidence. Yeah. Yeah, no, absolutely. And we talked we we kind of

Combating documentation rot in spec-driven workflows

23:11 like mentioned there a little bit about maintenance. I guess um we're going to have the same issue with specs, right? How do we make sure that specifications don't become stale, that developers are continually upgrading the the the spec and it doesn't it doesn't, you know, get out of sync. Um I guess how when when we think about like BEMAD specifications, the the BEMAD method, how do we how do we make sure that this isn't a turning into a maintenance hell and that we don't get stale documentation that just sits there, nobody reads it, similar to a kind of like what we saw the PRD become? I think there's this like continuum, isn't there? Um in the the world of spectrum and between spec as source as the kind of the first term, um you know, the the spec as the the baseline artifact that the code is generated off such that you throw away the spec and get the exact same code artifact or very similar code artifact as a result when you regenerate your code, versus a a more kind of spec-first um driven methodology where the specs are essentially, you

24:12 know, they last for the lifecycle of the epic or the feature or the task and then get thrown away. And so, you know, tied back to that is this world of maintaining the documentation. And I feel like as the frameworks evolve, we will end up more in a world of spec as source where the really important artifact is the spec. You know, cascading all the way back to the product requirements document. But in the in the immediate term, we are definitely seeing some degree of like documentation rot and documentation going out of out of state or out of out of sync, [clears throat] a little bit stale.

24:45 Um in BEMAD specifically, I've seen this solved for a little bit through the ability to inject into the backlog

24:54 [clears throat]

24:54 new work items that, you know, might might alter the course of development. And within that work log item in the backlog, you will also have a specific spec for the story that talks a little bit about the detailed implementation of that story and what we expect from it. Does that cascade back neatly to the PRD and architecture document if if needed? Sometimes, but not always. Um and that just feels like a a direction of type documentation will wind up also

Spec-driven development requires cultural and tooling shifts

25:23 being kept in sync. Um feels feels like that's the way that that that the tooling is headed, but we're certainly not there yet. Yeah. It's interesting when we talk about it cuz there's like, you know, you talk about tooling there and that's the way the tooling is heading. And and I guess there's like like any major kind of change to it to our industry, there's there's there's different things that need to change. There's tooling, there's practices and process, there's the human cultural side. Um And and like DevOps [clears throat] is a great example of people sometimes initially over over-rotating on the tooling, but actually coming back to realize actually, this is a this is a team approach, it's cultural change that we need to address first. Um I guess BEMAD, let's let's ask the same question of BEMAD. Is this fundamentally um a technological problem or is this a cultural practice that we need to change?

26:10 I think it's a mixture of both, but it sort of starts out in the world as a as a a change in culture. Mhm. Um so, what what it sort of does is it forces you up front to expose a lot of the assumptions that might exist within a software project. Um because, you know, as we discussed earlier, if you if you're unearthing these discoveries and these assumptions um as development is ongoing, you know, results may vary. And so, it's very important to have a well-aligned um piece of documentation with all of the stakeholders that talks about what exactly it is that you're building. And to me, that's, you know, a pure cultural change.

26:45 Um There are also kind of technical practice changes coming about as a result of the use of many of these STD tools, BEMAD included. Um in particular, probably the the worst culture technical change that I've seen is probably the the transition from teams to sort of the sole contributor. And that's one of the things we are most focused in trying to solve for. Um is how do you scale um a spec-driven framework to an entire team of developers rather than a an individual contributor that is taking on the role of DevOps plus front-end engineer plus back-end engineer plus, you know, the list goes on. I think that's that's almost, you know, not a great way to be building within the enterprise and we need to better solve for that collaboration piece.

27:30 Um and although right now the challenges are purely technical, um it feels like ultimately a you know, it's a cultural it impacts the culture because, you know, you're expecting an individual contributor to be able to wear all these hats. The the the limitation is purely technical. It is that right now, we are mixing, you know, the the work of the back-end and the front-end engineer in a decomposed task lists um uh that that that are output of of a lot of these spec-driven tools, assuming full-stack engineering, but in reality, we could be aligning these tasks to individual individual roles and have people contribute. So, I'm um I am optimistic of the future for the sort of more collaborative culture that spec-driven will will bring. Um but the the tooling right now just needs to catch up. Yeah. I love to I love the phrase you used there as well where it's it's like I think you you mentioned forcing function, which I think, you know, problems with a culture change in terms of the way we go about building software or or doing anything in in the space of tech, you you often have all the desire you want in trying

28:31 to make that change, but ultimately, it's the tooling which is the forcing function that encourages and ensures people are actually taking those steps. And I think it's the way it's the tooling that helps train us to make those cultural kind of steps to to to ensure we actually follow that. And over time, yeah, we probably do we'll probably do that without even thinking, but it's that transition whereby the tooling can really help teams grow fast. And I think if we I I really liked what you were talking about with the collaboration and the teams. And I guess I'd love to ask a couple of questions. When First of all, maybe go into more depth on specs and then I'd I'd love to kind of like understand at what point does that does that make it too heavy weight? So, so why don't we start with specs? Um this is a cultural change to actually, you know, get people to sit down and and and and write those specifications and think out what they need to put into those specifications. I guess at what at what depth do we need um detail about

Right-sizing specifications to prevent context degradation

29:30 every aspect of a project uh whereby that's going to maybe we overthink things or it actually starts slowing us down versus versus helping us. Where's the where's the right level for for detail, I guess, in those specifications? Yeah, it's it's interesting because I think in the world of startups, you know, the the idea of overly verbosely

3 Highlight · Optimizing spec size for LLM context windows

29:50 specifying what it is that you want to build is, you know, almost sort of a bad word, right? The super detailed spec can can slow teams down. Um I feel like the right size spec problem is something that is another challenge with the tooling right now. We talked earlier a little bit about rapid prototyping not being a great fit for vibe coding bug fixing not being a great fit for a vibe coding.

30:17 Um And right now it feels like the the sort of the scale of the specs that frameworks like BMAT produce can be quite excessively verbose and heavy. That's one of the things that we're looking to try and trim. Because not only we're looking to right-size the specs for human context, right? But we're also limited in how much the model can can comprehend. And the quality of outputs will go down the more and more the context window we take off with with with just specs, right? Um so that right-size question is probably one of the most important things for these frameworks to get right. Um it feels to me like BMAT is the closest approximation of that I've seen to date.

30:55 But it's not perfect. It does some really clever things around very lengthy documents like PRDs and architecture documents called sharding. Which is super clever. So what it will do is split up a requirements document such that the functional requirements per epic are split. And then a architecture document such that components are split such as front end and back end. And only subsets of those shared documents are included in the context window when interacting with the model.

31:23 And it's some of the cleverness like that that makes me think this is like the best approximation right now of what spec driven looks like in providing the best possible honed context to a model. Yeah. In theory every line of spec helps the model and it's really just about optimizing for context window. But in reality that can become excessive. So like it's one of the most important

31:44 things I think to optimize for that length of a spec and you know, how big can it be before it starts hurting rather than being helpful. That's where these frameworks I think are going to longer term help us. Yeah. And gosh, this resonates so so much. I'm literally doing something right now with a whole bunch of a whole bunch of kind of like spec style documentation markdown files that are kind of like sharing core security practices and I was practices and we're talking like 80 different files of of best practices and policies of how to write secure software. Now if we was to throw all of that directly at a an LLM and say right do a code review here, there's just too much context for it to kind of like hold in and say yeah, I'm going to I'm going to do all of this and I'll do I'll do it very very carefully. But the more I'm thinking about it and and the way I'm trying to the way I'm approaching this maybe even as a code review is to perhaps say well, let's actually let's actually try and target

32:45 you know, various flows whereby I'm sharding is a great way of saying actually. I'm looking at very specific areas. Maybe start off with authentication. Let's let's see let's do a code review for authentication across this across this massive app. Because otherwise you've got so much context and you've got so much code that you're trying to look at. It's just it's not going to go do a good job I don't think. Really interesting to see is like how much So right now a lot of the logic for how this is sharded is said sat in a framework called BMAT. But if you go up the chain, right? So like if you look at the IDE and how it selects the most relevant piece of context to send to the foundation model, then you go up the chain again and you look at the foundation model as it reasons while completing a coding task and decides which bits of context to pull. How much we can align on, you know, standardized specs that we're using to to develop.

33:33 You know, we have some constructs now like agents.md. Um but then we get into, you know, do we use dot cursor rules or a project constitution or a cloud.md. Um

33:44 [clears throat]

33:44 Right now it's a little bit of the wild west. You know, it was nice to see the MCP get get standard get released into a an open source foundation. But there's a lot more to be done on standardizing on this sort of base building blocks of specs across model vendors and across IDEs. And it's it's sort of exciting to me to see how much of the sort of techniques and terminology that we use in the framework land right now all the way up here is going to eventually cascade down into foundation model vendors and also IDE tool vendors. Um so yeah, you know, shout out to the the the the vendors of tools here like let's let's get a bit of standardization going. The more the better, right? Absolutely. And I think that's the key as well because you'll get you'll get people like using whether it's IDEs or agents across a company even wanting the same the same style.

34:31 And that agent like the agent agnostic approach is going to be is going to be

Spec-driven methods give startups a production-ready edge

34:35 so so important for for for for this. So so so you mentioned about startups. I guess the startup culture of of really valuing speed and delivery and experimentation as well. Is this the wrong approach then for for a startup or do you feel like there's a there's a path for where BMAT can be used there as well? That's a great question. And like where on the spectrum does this belong within a startup as well? Yeah. Um I think the sort of you know, the world of the the vibe coder is going to find this very frustrating, right? Because it's going to slow them down. And the idea of having gone from seeing outputs in 20 minutes to an hour to seeing outputs at the end of a

35:15 [clears throat]

35:15 five six hour documentation session will be inherently frustrating to somebody who's used to in a in a really low diligence way just prompting their way to a really cool application. Um but for me at least the penny drop moment was when the quality reflected the fact that you know, that slight increase in effort was was put in. And so you know, when I was able to build something in let's say you know, two days of of spec authoring and iterating through a backlog that I felt like I understood as a and and it felt like an artifact that would have taken me a month to produce otherwise and was probably better code than I would have written, you know, it's that's when the the penny really dropped for me. And a repository with linted code that had you know, two three 400 unit tests, automation tests, you know, an architecturally separate front end and back end, infrastructure as code like just real wow factor. And I think that any startup that isn't leveraging this build methodology

36:16 [snorts]

36:17 is going to find it very hard to compete in a world where you know, the vast majority of for example Y Combinator startups are using AI tooling to a statistic I don't want to misquote but some you know, incredibly high percentage of of startups in Y Combinator are using tools like this. It just gives that sort of competitive edge and the world of building products just looks so different now because anybody can show up with a vibe coder prototype and and show something shiny to investors. But actually transitioning it into something that would scale to the first even 500 or 1,000 users is a very different question. Um and I I feel like spec driven is like the the methodology for doing this. Yeah. And I think it's it's it's interesting I see when I see spec driven approaches and I see kind of more prompt driven approaches. You almost like start looking at a prompt driven approach being a very proto type style where doing things. I've even known someone to

37:18 start with a prompt driven approach and they send their kind of like code logs or whatever it is over and say look, build me a a spec based on these prompts. You know from my prompts where my intents are. You can see from the code that's been generated what has been built, what I'm happy with. Build a spec that represents this. And I I guess kind of like do you feel like that if you were to like go fully vibe in like prompts without without building a spec.

37:44 Do you feel like that would be a like a strong limiter for people who actually wants to go much further, much closer to production, those who want to scale AI within their organizations? As you've probably gathered like that's kind of the framework of how I think of things that have purely been prompted. Like that's not to say that you couldn't construct a perfect prompt that actually encapsulates all of the context from your specs. And just perfectly right-sizes your context window such that it's built, you know, task by task a bunch of different a a bunch of different smaller stories in just you know, very well crafted prompts. That's definitely possible. But the methodology of spec driven with the combination of a framework makes you more likely to be successful. Yeah.

38:29 What's also interesting is like you know, is spec driven what the future of AI native engineering looks like? Like I'm not actually that dogged on the fact that it is. I'm not completely convinced. Um it feels as though Cursor in particular has been slow to adopt a lot of the primitives and have stayed gone through the

38:47 [clears throat]

38:47 the the recent release with plan mode feels like a a spec similar to spec driven or spec driven compatible way of operation. But it's not exactly that. Um so like you know, it feels like the best approximation right now how to work with these models this this this spec driven methodology and also the BMAT framework itself. But that's not to say that you know, next year that's that's what things are going to look like. My if I've learned one thing in the world of the in the kind of world wind that's been the last nine months, it is that my crystal ball is inherently faulty and I will make terrible

39:22 Who knows? Yeah. The crystal ball will be I I I saw an eight ball the other day where it's an LLM it is an LLM implemented eight ball which is like oh yeah, everything's everything's going in the future. So the crystal ball will be

Inadequate requirements create severe technical debt

39:35 repeatedly say You're absolutely right.

39:40 So so when we when we think about how LLMs, you know, typically are are kind of like better with the the more structured specs, with with kind of like clear requirements. When we think about those specifications or those systems that are poorly specified, what do they become? Do they become something which just, you know, holds technical debt? Or are they now a liability because because of the way an LLM is going to is is essentially going to implement that and and and make assumptions?

40:08 Yeah, it's a an interesting kind of question, isn't it? Like what becomes of technical debt? Does, you know, do we have this new thing that is spec debt or requirements debt or, you know, inadequate requirements? Because when you think about a tech debt, surely should at least fade into the background um because it's now easier to tackle backlogs of tech debt through agentic coding, right? You know, the the sort of drudgery of Well, actually, it doesn't feel like drudgery sometimes. It's really rewarding. But but the the art of like eliminating tech debt is now much easier to to go after with an agentic coding system typically. So, you know, does that become really important um when you can solve for tech debt much easier? And in fact, it's actually something that's not really aligned to what you wanted to build that becomes a, you know, the problem. So, like poorly defined requirements, um does it become harder after going through, you know, three, four days worth of of spec-driven development where you suddenly have this 20, 30,000 line repository? Is it

41:10 actually harder to recover from the fact that you missed some requirements along the way? And is there flexibility sufficient in the workflow to be able to make up for that? Um certainly some of the techniques I've tried have been things like injecting new backlog items to the end of a a backlog to try and recover and still following the spec-driven methodology rather than sort of bailing to vibe coding, which is definitely a temptation, but following the the methodology to sort of steer towards the the desired output. Um but yeah, it's it's definitely tricky.

41:42 Um the the sort of robustness as well in that kind of requirements documentation artifact. Like I know the the Kiro team are doing loads of really interesting research at the moment on what makes a like really good, well-defined, structured requirements document through I think the use of like formal methods um to to ensure that that that that the requirements are are um

Balancing human oversight with agentic autonomy

42:06 well-formed and that they complete and that the gaps are actually identifiable by the the model as well. Yeah. Yeah. And and I guess, you know, the goal the goal here um of of B mad when we think about agents as they are today, um agents are becoming more and more autonomous. Uh are we trying to make them more autonomous through things like specifications and through uh B mad methods because that allows us to create a a more detailed brain dump of what we're thinking? Or is it actually are we still aiming for that very assisted back-and-forth chat style flow uh whereby it just makes it safer because we're giving it the guardrails it needs? Is there a Is there a kind of like a north star that we're we're trying to get to or both very applicable?

42:57 Yeah, I think it's a bit of both. I need to get off my fence, don't I?

4 Highlight · Scaling developer autonomy with sandboxed backlogs

43:02 If you had to choose one though, Kieran. I refuse.

43:07 But backlog decomposition for me feels like the place where there's a little bit of room now to speed up with the sort of um kind of safety we've injected and guardrails we've injected into some of these frameworks. So, like the iteration of an architecture document and a requirements document, that feels like a pretty important stage to still have a human in the loop. But when you've decomposed a backlog and reordered it to your heart's content and you feel like the the stories that a an agent is going to go after one by one with a new context window for each one is in good shape, I'm not sure that me sitting there and trusting every single command that the model suggests I run is necessarily a great use of my time when I can, you know, run the agent in a sandbox and force it between each story to get commit, get push. And if it decides to rm -rf / star, that's okay by me cuz I have everything in Git and it's in a container and who cares, right? Um you know, you see all the memes of of folk vibe coding and seeing it rm -rf their their entire directory or rm -rf.

44:04 Um yeah, the that becomes less of a problem when you've got sufficient guardrails and you're operating in a nice sandbox VM and so on. Um and so it just feels like sat there iterating through every single story maybe less useful. I'd love to be able to accelerate through a bunch of stories and just review the output of epics maybe. And so you get towards more of that autonomy um when it comes to, you know, as I said that that earlier phase of actually writing the specs, authoring the specs, aligning on the specs, I feel [gasps] like there's less room for autonomy

44:35 right now. Um but but if if we were able to also make that highly autonomous, like where do we end up? Are we actually back to vibe coding somehow? I don't know.

44:44 It kind of like relies a lot on trust, right? Cuz like the if we don't have trust, we just plain don't have autonomy because the the the the trust is that piece in the culture that allows us to allow the AI agents to to to to stretch those legs and you know, run run as fast as Yeah, again, right? It's like having the agent enabling you to go after CI/CD and and then, yeah. Yeah, yeah. So, so So, does B mad I guess B mad changes that model then when we think about trust in terms of it, you know, does it provide us with more trust and maybe we kind of like lean a little bit more into um I guess quality, the quality of of of AI output and and how specifications can you know, can [clears throat] provide that greater quality because it's providing those guardrails and as a result, knowing if those guardrails can be whether there's automation in the validation and things like that so that, you know, things can be raised to us to say, "No, this is actually going against the things that we've said in the spec." We can not just recognize we know there's high quality here, but we then have greater trust. Is that Is that Is that a path where we can, you

45:46 know, raise the autonomy bar? Yeah, I think that that is exactly the lever that we need to be able to make the transition in autonomy is you know, being able to trust the outputs of the agent. And so, like if in the short term it looks like, you know, on a per context window, per story basis we're reviewing outputs, so be it. But the more that the sort of quality metric is pushed further to the right, um the more that we can afford that autonomy and the more that we can afford to accelerate. And so, I think what will get very interesting is as we transition um as models are writing more and more tests to cover what it is they've done. It's funny, you know, that the same tricks ring true like TDD and so on. The the more that models are able to actually test their outputs um and prove that what they claim is working is is in fact truth um through a completely separate artifact to it just interpreting the code, um the more that we can trust that autonomy. Um and it's like it's the traditional software engineering techniques that we've always used to build trust in a in an artifact

46:49 and build trust in the deployment of an artifact just being used by by agents. Like I do sometimes wonder is there a a world where there's some new construct that that models use to test the validity of code that we don't traditionally do in a in a software development life cycle that can be, you know, that can be used here. Maybe that's where the research goes, I don't know. Um the the the Kiro folks again, shout-out to to them have probably done the most in this space with their Oh, what do they call them? Checks, I think. Um but they have a new, you know,

Bridging specifications and test-driven development

47:21 testing methodology that checks for completeness in a in a way that's not TDD and is not BDD, but is something different. Yeah. And it's funny when we talk about TDD and like when we think about specs and I remember in the early days of Tessel when we were when we were looking at what goes into a spec and and how should it be how should it be phrased? What I what I found like immediately very interesting was how, well, if you were to write down a capability in a specification, you know, you can think about that in two ways. You can think about that as this is something I want the application to do, but also just slightly written differently, it's it can be written as a test. You can say when when this happens, I want you to provide this back. And that, you know, it actually turns it into just a slight rephrasing turns it into actually a test-driven test-driven development style approach where actually your specification is is a list of is a list of things that you want to be true,

5 Highlight · Writing specs as testable requirements

48:16 um essentially test cases. So, I guess, you know, what came first, the chicken or the egg? What what should come first in the spec, the the code or rather the capability or the test? That requirements writing style, uh EARS, is that what it's called? EARS requirements? Um it is setting you up quite nicely for a specification of what a test to subsequently validate that requirement could be. And so it feels like that is vastly more important than I don't know, you know, your your dump from your brainstorming session earlier on. So, actually getting a succinct statement of what that requirement is and being able to use it as a testable artifact, highly valuable. Yeah. Um you know, maybe maybe we'll we'll be in a world where a Playwright automation test is written to validate a piece of functionality purely off of observation of the produced interface plus that statement with no insight into the code. The code is a black box. And the methodology is, here's the interface, here's the requirement, prove that this actually works. Model, off you go. Yeah. And it's

49:17 just a LLM as a judge LLM as a judge style just trying to look at what's been created. Yeah, absolutely. And I think actually there's been there are several papers that actually show that is a valid that is a very valid way of of, you know, running evals and and actually very,

Enterprise experimentation with spec-driven frameworks

49:32 very accurate as well. Um yeah, cool. I guess what do we wrap up with kind of like a little bit more on some of the some of the things that Nearform have done. Um I guess you know, you've run a number of experiments certainly around Be Mad and SDD and I think you're from talking with a you know, a large number of people. I think you're probably from from a consultancy point of view definitely the the the the group that have you know, jumped in most with this kind of thing. So I'd love to kind of hear what are what are some of the most interesting experiments I guess that that that you have that you've played around in New Zealand.

50:06 Yeah, we've definitely gone hell for leather. And so I suppose some of the kind of challenges that we found in rolling out a framework like Be Mad to a team of people since that's how we tend to operate have forced us to sort of experiment a little bit with you know, how we overcome that. And so I've done a little bit of spiking on how to codify some of the rules of our team within Be Mad. A colleague of mine Luca, shout out to Luca is doing incredible things at the moment in forcing the backlog generator from Be Mad to be able to sync with a an issue management system like GitHub issues.

50:41 And also being able to orchestrate these workflows not just locally on a developer's machine but through something like a GitHub actions workflow. So a super cool way of running spec-driven development tasks through a more shared interface where the sort of work log is visible by all members of the team through a web browser rather than an IDE. Helps with that kind of going parallel as well. And that's been most of our experimentation throughout the kind of latter half of 2025.

51:11 And so what we're now toying with as a business is do we just you know, embrace chaos and let you know, a bunch of really clever people drive their own experiments which yes, absolutely.

51:24 The open source way. And then 2026 you know, the hope is that we'll be able to harvest from a bunch of these really clever ideas a more kind of unified Nearform way of delivering at scale overcoming some of the challenges that we have with the core framework. You know, does that one up a module within Be Mad? Do we roll our own framework? Do we contribute back to Be Mad? You know, that very much remains up for grabs. But it feels like an area where we're going to need to contribute something.

51:52 Um you know, the hope would be something in the open source domain to to try and solve in particular if we're going parallel across multiple teams. That that's been the big area focus. Yeah. Super interesting. And and to be honest, I love I love working with you guys and I I can't say enough nice things about about you and the team. For for folks who want to learn more about or first of all, keep up to date with you know, some of the great things that Nearform are doing or you know, reach out to work with you. What what's the best ways of of getting in contact and learning more about Nearform?

52:26 Yeah, absolutely. I mean great place to start nearform.com. The jobs board is up there. In terms of what we've been experimenting with spec-driven, I'm the worst on our blog. Nothing. I think I've written more on your blog than I have on my own at the moment. But you know, shameless plug come follow me on LinkedIn. I tend to you know, talk post fairly techy stuff on the world of spec-driven development pretty sporadically.

52:52 Our own Nearform insights account on on LinkedIn also post a bunch of great content. A couple of my colleagues post a lot retweeted by the main account in the world of spec-driven. And so we're pretty active bunch on LinkedIn and that tends to be where we you know, where we talk the most about our experiments in spec-driven and what's working and also what's not working. You know, we're pretty we're pretty open book when things don't go well as well. Yeah, yeah. Sounds amazing. Well, Kieran I think we've got about two to three hours of content in somehow around 50 just over 50 minutes.

53:23 So we we did well. We did well. Kieran I think you so much for having me Simon. Oh, it's been an absolute pleasure Kieran. I really enjoyed the conversation and thank you thank you very much for bringing you know, all of your and and and Nearform's expertise and experience to to share with our listeners. So I appreciate it. Thank you very much. Absolutely. I'm sure we'll chat again next time I'm over in London which I've no doubt will be sometime in January. Oh, wonderful. We'll see you there. Yeah, I hope everyone enjoyed that episode.

53:49 Tell us what you thought and speak to you soon and see you in the next tune into the next episode rather. Thank you very much. Bye for now.