The Man Behind World’s Largest Dev Con | Stephan & Devoxx

11,581 words · about 46 min to read

0:00 I'm calling this the renaissance of development. If you didn't do VIP coding, you need to set up your Maven project. Just setting up the build environment can already take you a couple of days to make that work. And with VIP coding, it just creates that for you very rapidly. The project I've done in more or less 5 weeks now. It would have taken me definitely three people 6 months without doing VIP coding. So after 4 months of waiting, I decided to develop my own plug-in in Java which is called DevOps Genie. So I did the first prototype in React. I used that as a code to spec and then for my final implementation I went spec to code. So it's like a round circle so to speak and that really helped a lot. And just last weekend we reached 30,000 downloads already of people using it.

0:45 If I was to go to people and say who is Stefan Yansen, people will think ah Devox pretty much it's the premier uh Java conference worldwide.

0:53 I do think so. We welcome 20,000 developers every year across a small 20 events of which eight deoxes and 12 10 vox days. Before we jump into this episode, I wanted to let you know that this podcast is for developers building with AI at the core. So whether that's exploring the latest tools, the workflows, or the best practices, this podcast's for you.

1:24 Now, we've had amazing guests like Olivier Pomel, the CEO of Data Dog, Matty Stanisowski, the co-founder of 11 Labs, Victor Ripelli, the CEO and co-founder at Synthesia, and of course, Tesla's very own Patrick Dubois. And they're all sharing lessons right from the frontier of our industry. A really quick ask, 90% of people who are listening to this haven't yet subscribed. So, if this content has helped you build smarter, hit that subscribe button and maybe a like. It really does help us keep bringing the

Devoxx scales globally with massive developer enthusiasm

1:52 folks who are actually pushing this space forward and helping build that AI native future. All right, back to the episode. Hello and welcome to the AI Native Dev podcast. Uh my name is Simon Maple and on joining me on this episode is uh Stefan. Stefan, how are you?

2:09 I'm very good, Simon, thank you. And Stefan, you are the what is your official title, Stefan? You're head of Devox. Oh,

2:18 head is a I I founded Devox or Java Polish uh in 2001 and I basically run the Devox Belgium edition. Um and I try to get at least the coordination or know what's happening in the other Devoxes and Vox days which I co-ounded with Mark Hazel.

2:35 Yeah, absolutely. and and so Javopouloolis which is now DevOps um I'll go as far as to say pretty much it's the premier uh Java conference worldwide I think it's it's reasonably fair to say probably the biggest Java based uh uh group of conferences across the world as well

2:55 I do think so I mean we welcome 20,000 developers every year across a small 20 events of which eight deoxes and 12 10 vox So if you group them together, it's a substantial amount of developers that we reach uh every year.

3:12 Absolutely. It's incredible. And and of course, I would say your flagship event is probably as as you just mentioned there, Belgium. Um it's it's an event which has three and a half thousand folks come to Antworp every year in in October. Um in fact, Tesla are very very happily sponsoring this year and we have a we have a couple of sessions there as well. So really looking forward to that in in October this year. And we were talking off air just then how fast the tickets go. It's basically like a rock concert, right? It's seconds and all tickets disappear. It's pretty incredible.

3:46 It's It's crazy. I mean, because of that, we do two batches. So we have one batch last Monday and then another batch on the 1st of September because we have actually people calling us crying literally crying on the phone saying like my boss told me to get 10 tickets and I didn't get even in. So for those type of scenarios at least we have a second smaller batch to to you know uh solve some some issues from registration's point of view. But it's just crazy. I mean, we had 4,600 unique IP addresses refreshing before the registration went open and then people just jump on those. Um, just crazy. But luckily, I mean, people do release them once they have like a conference ticket.

4:28 They still do release some tickets. So, currently we still have deep dive tickets and comi tickets. Uh, but the conference tickets, they're all gone.

4:37 It's it's absolutely incredible. I like particularly postcoid as well. I hear so many conferences that are not struggling necessarily, but they definitely see a dip. Um, and I'm sure it's probably similar with yourself, but it's incredible to see that that demand is still so high for the for the kind of like what I would call I guess the the the premium conference, the highest uh the hardest conference to get in, not just as a speaker, but definitely as a as a participant. I think a lot of people at companies because they work at home after co a lot of them use it also as a social gathering where they're get together for five days you know go to great talks and in the evening have a beer or two and talk about tech so you know it's a great uh week having fun

5:18 uh incredible so when we say like if I was to go to people and say who is

Building an open-source Java AI assistant plugin

5:21 Stefan Yansen people will think uh Devox it's like oh he yeah he runs Devox he he he he hosts and runs Devox Belgium Jim, but behind all that, you you're a very active developer. You've been tinkering with AI dev tools and and things like that for actually, you know, a significant amount of time. This isn't something that you've just got into. I think I've been following on Twitter for for for ages and and and you've been like playing with lots of lots of stuff like loads of image recognition stuff in the early kind earlier days. I remember was it basketball you were kind of I remember all that kind of cool stuff.

5:57 What have you been doing more recently?

5:59 Well, the beauty of organizing DevOps Belgium, it's not a full-time job. You know, it it gradually picks up in pace and like now the last months it's really hectic. But once Devox Belgium is finished, I have like four, five months where I can just do pure research and development and tinkering as you said with with different tooling and and frameworks and things that I saw at DevOps which inspires me and then I want to play with it. And one of those things was two years ago. I mean, Jet Brains announced this AI assistant uh and they actually demoed it at Devox Belgium during the opening keynote. And one of my questions to Anton was like, hey, does it work with local models? Is it open source? Uh can I choose any other cloud model? And those answers were constantly negative. So after four months of waiting um I decided to develop my own plug-in which I've never done before using um you know in Java.

6:51 So I wanted to build a fully open-source Java plug-in for code assistance which is called Devox Genie. And just last weekend we reached 30,000 so 30,000 downloads already of people using it. And the aim was to really support local models. like Lama, LM Studio, YAN, Lama CPP, all of those, but also the cloud models. And the crazy thing is is that you can switch between the different models. So you can work first with a, for example, local model, then have a cloud model, review the local model response, etc. And that's what I I've been working on quite a bit for the last eight months or so. Um and then of course there were other tools coming out especially CLI command line interface tools like cloth codes and so you know the DevOps Genie it works people are using it happy starting to focus a bit more on the CLI type of tooling uh which is of course you know code gemini and codeex I think those are the three

7:51 frontier ones and and for me I had like okay maybe I need to do a new experiment in July because it's still a bit not too busy pets. Um, and I decided to do a VIP coding experiment.

8:04 This is super interesting. Now, and this is what the sessions this this uh podcast episode is going to really be about. We're going to talk through how this experiment that you built, and it's a it's a a neat little app that we're going to uh we're going to talk through uh for as a as an assistant, as a helper to the to the DevOps conference in in October. And we're going to talk about the approaches and the tools and the processes that you use to build this. Um, you started with uh a traditional well I say traditional but given vibe coding has been a term that's been uh probably only popularized for the last year or so. You started with that vibe coding approach using using the kind of like tradition well using the the the better of the vibe coding tools I would say and then you you hit your limits I guess on on that vibe coding and we'll talk through what they are and you then move more into a spec driven approach.

8:54 So how can you actually use specifications that allow the LLM to kind of like build the same app uh but but through the specifications and we'll

Vibe coding a React companion app hits a wall

9:03 talk through the various successes and pain points of each of these. So first of all Stefan talk us through what this app is what the need of this of this application was for you. So we already have for about two years an also an open source project called Foxring which was a tool developed by Fri Caml and beautiful application it uses Firebase at the back end etc. But for him, it's more like a a hobby project, an evening project. And I was really starting to miss certain features. For example, searching speakers, introducing the photos, which I'm also using face recognition to show the speaker photos.

9:43 Um, and then one of the other more important features like authentication, so people could authenticate themselves. So their favorite could also be shared on their mobile phone etc. So it's basically the the Devox companion application, which I I said like, okay, you know what? I'll try to redevelop that using just f coding. And the the the limitation was really I do not touch any code. I can look at it, but I can only prompt it. That was the limitation I I enforced myself,

10:11 which really is what vibe coding is, right? It's it's it's you code and you you you're effectively well, I suppose you're using you're using your vibe or you're using those small messages to kind of like suggest changes, but a lot of that is really you are testing through almost click. you're not necessarily, you know, creating, well, you could, I guess, build a number of tests, but it's typically not how things are done. People will typically say, I need this change, it makes that change, you test that change just through clicking around and and with an app with a front end. Is is that typically how you is that typically your flow?

10:44 Yes. So, well, the other limitation I said is that I will do it in React, which I don't really know. So that already forced me to not even look at the code because I don't know React that I've never done a React application.

10:57 And the reason you used React there, the reason that choice was made wasn't because of a technical choice. It was because you didn't know it.

11:03 Two things because I don't know it. And I've seen other VIP coding posts and people do know that React is very well indexed by these LLM. So the LLMs really know it very well. you know, if you go to more exotic languages like Go or or maybe Rust, you might not have that much support from the frontier LLMs, but React is up there, you know, one of the top technologies supported. And so that that was how I wanted to start. And and so um I already had of course some

1 Highlight · Vibe coding delivers rapid UI but ends in spaghetti code

11:32 features in my head of how the existing application was that I wanted to mimic, but I wanted to go beyond. And what you really see is that in the first couple of days you get an enormous boost. I mean the the the UI that it creates the first thing is like you know it creates a design that a backend developer I'm more a backend developer I could never produce such a beautiful design and then if you do look at the CSS you know the CSS is beautifully done it's responsive etc. So you get an enormous boost immediately. Um and so I spent two weeks doing pure vibe coding and I must say I got all the way till 90% of the functionality. Uh but the last 10% it was really struggling and then I started to to look closer of like what's happening and doing more like um reflection type of prompts and what I noticed is that well there were a couple of things. One was like I was ah I need to support dark and light themes. So in

12:33 a way they was refactoring the the the CSS to support with variables to support dark and light. Then I was like oh I need to support internationalization. So then it was introducing resource bundance and so on. Then I was already starting using Firebase but then you go like okay I need fire store I need storage maybe I want to do the real time database. So you're basically prototyping and refactor or letting the LLM refactor your code based on your chains requirements. Um so that wasn't cut stone before I started. So that might have also resulted in a few strange behaviors but nevertheless after two weeks 90% worked 10% of the application was really broken and what it's uh after some investigation it had created too many redirections and too many abstraction layers that it was just basically spaghetti code. um it could really not be deployed to production because it was not fast enough. It had

13:33 broken caches etc. And it was basically I I've basically created a Frankenstein in two weeks to be honest.

13:41 Yeah. And and a lot and and a lot of

13:44 that is because it's iterative. When you say Frankenstein, it's because you're making changes. You're putting whole extra pieces bolted on almost versus being built through from scratch. And that that it's that iterative uh aspect of it whereby um if a change gets made, it does its best to to adapt what is already there versus recognize if it needs to kind of like you know regenerate versus things and and and and do that across an entire application and build it from scratch as if it were using that from first principle. I think you have some things to share as well.

14:15 Uh you have the you have the application uh that we can kind of like we can kind of like view and talk us through some of

Troubleshooting complex bugs with multiple AI CLI tools

14:21 the tools that you used. um as part of this entire process.

14:25 So the the tools that I've I've used throughout the the VIP coding experiment uh was mainly cloth quotes which which is a CLI. I bought a $100 subscription which was actually more than enough. Uh I must say during that period I did three blocks of five hours every day. Um so I wasn't working 15 hours but it was like three hours then I ran out of tokens. I had to wait two hours to start a new uh chunk and so often it was like you know around 10 hours a day. So those were long days but very fun uh to do. Um and so the the that was my main tool close. uh but then I often actually used uh especially in the second phase which we will come back to but in the first phase sometimes I got stuck um or or the the prompt didn't find the the solution and then I had like I went to Gemini and actually gave Gemini CLI access and because Gemini is Google Firebase is Google so it might actually know a bit

15:26 better the the documentation around Firebase it I think two times it actually helped me uh fix a solution which cloud cards couldn't uh provide. So, Gemini definitely helped there and I just used the API key for that. So, I was paying per token and then often I also went to Deepseek to actually do some brainstorming where I actually had a chat I gave it to Deepseek and Deepseek would actually do

Transitioning from vibe coding to spec-driven development

15:52 some deep thinking or reasoning and come back uh with some other suggestion or approaches. Um but that was basically the first version. uh when I noticed that I was completely stuck after two weeks, I actually decided to do a control alt delete and restart the

2 Highlight · Using multiple frontier LLMs to generate robust feature specifications

16:10 experiment uh but now using a technology that I know a bit better which is Angular. So that was my my focus. I would I would redo it in Angular, be a bit more aggressive in looking at the code what it produces and really be more I would say a technical project lead where I would really guide it and and I would be the guard rail of what it gets generated. But what is more important and I think that definitely helps the success is that I took the react code as the basis to create specifications because the first phase I did not use any specs I just started prompting um and but but large prompts but I had no real specifications. So what I did is that I fed the react application into cloth codes and per feature I asked clothes codes Gemini and sometimes even chat GPT to come up with a specification for a certain feature. So having those

17:10 three models reviewing each other really um creates a crystal clear spec for a specific feature. That feature was then saved in a markdown in my docs directory. And so I had a markdown per feature.

17:26 And and in terms of that specification, what would you say were the um the key the key parts of that specification that allowed um you know the next generation of of the code to effectively um understand what it needed to to build? Were there were there headers with specific information in in each of those markdowns? Yeah, it was very in it was a really detailed specification. I mean it was not only describing the feature but it would even discuss uh in through for example the model it would need to use the even certain snippets of implementations uh were included. It's really like it would be a spec if you give it to a senior developer he or she would know exactly what to implement. uh and that was only possible if you have those three different frontier models reviewing or even four sometimes reviewing that. I I I would definitely recommend having at least minimum two

18:27 frontier models reviewing it. But you have three or four reviewing it and like Deepseek is even free. I mean I I never paid for Deepseek and I even used the the free version of ChachiPT as well. I only paid for clothes code and Gemini. But that collaboration between those four they create beautiful specs uh which is really sufficient then to take that and and include it into uh you know VIP coding uh exercise.

18:52 Yeah. And it's really interesting the the the way you talk about it there

18:55 where the specification doesn't just so so one of the ways that we think about specifications is it separates the the the what I guess from the application. Uh whereas today the code uh very often describes the both the what and the how. the the the how something should be written as well as the what it's actually doing. And it's hard to to kind of separate those. And it sounds like what you've done, possibly because you started from the uh from the from the the code to generate the specification, is you've taken that what out of the code, but peppered it a little bit with the with the how, I guess. So, you've got the intentions of what you want the application to do.

19:31 Yeah.

19:32 And then there's then there's pepperings of the how, which I guess then it pivots to a specific implementation. Uh but that's okay if I suspect you know you only want it to be written in the application because this is you know an app specifically for for your needs. I mean it was exactly I mean I did basically code well so I did the first prototype in React. So I had a a first implementation. I used that as a code to spec and then for my final implementation I went spec to code. So it's like a round you know the round circle so to speak and that really helped a lot. Um and and and so there I I I really saw that the code I mean I had like also cloth MD file where I said like okay you need to use internationalization these are the languages you need to support I need to have theming um you you need to use for example the new iterative for and if statements in angular etc. So I really

20:32 created also a code style specification how it should behave. I introduced the llinter. Um and what's I think really key which is missing up till now in this story is developing the the testing the unit tests but also the integration tests. I didn't do test driven development. I I I let the VIP LLMs first create the code. Then I looked at the functionality visually if it worked.

21:01 Then I had it make unit tests. I had it made integration tests. And those were just added each time new features were were developed. And that's really really key because what you do notice is that once you start adding a new feature, it might actually accidentally break another feature without knowing. And so you have a new feature but all of a sudden an older feature is broken. And if you haven't covered that with a unit test, you would just you wouldn't know you wouldn't know that that happened. So having good integration test and unit tests really are also your guard rails to make sure that your VIP coding doesn't go off rails. Super super interesting because I think that's what also a specification allows you when you state this is what I want my application to do. It's very easy as part of that specification to effectively and and almost maybe even lead with your with your tests in the specification when you describe a capability because a capability can either be written in natural language as to how you want it

22:02 to behave or effectively detailed in tests which describe its behavior because you're effectively saying look when I get this kind of input these are the types of things I want here are some here are some kind of like unexpected outputs that I want you to be able to to predict and and and and react act in this way, you're effectively defining that capability. And and like you say, um when you make a change, you don't want that to have a knock-on uh effect on other parts of the application. And going back to the to the pure vibe coding, you know, pre premoving to a specification. This is the kind of like problems that you were having with that 10%, right? if there was and I think you've got a great example of of um you know when when there was a problem how it really struggled to to kind of um to to to find a solution um um you know for that and it it it lost confidence in itself. Take us through that.

22:58 Yeah, that that was really crazy. So I got stuck uh with cloth code. So, I went to Gemini because it was a fire fire store issue and the reaction I got from Gemini and I'll show you the different responses is just was really unbelievable. I mean, as you can see here, this is incredible frustrating. It's it replied and I sincerely apologize for the repeated failures. It's clear that my approach to modifying the rules is flawed and I'm not currently identifying the syntax error.

23:27 And it goes on. Then I was just chatting and said, "No, no, you can find it, etc." And then he went like, "No, no, this is beyond embarrassing. I am deeply sorry for the repeated failures. I'm clearly not the right tool for this job." He go like, "Gemini is having an identity crisis." And it went just back went back back. And then I said, "No, no, I'm going to revert the the changes. I'm really not able to find it." And then of course I had to step in and really look at the problem. It was just one character it had to change in the uh fire store collection path where you see here in the first part it has interactions and an underscore and it had to be interactions and a slash

24:08 and it was just both Claude and Gemini were unable to find that problem.

24:14 At what point did you feel like an LLM counselor trying to say no come on Gemini you are good enough you can do this.

24:20 Yeah it's crazy. I mean I I I have seen online that they've started to fix this type of apologetic behavior of Gemini. So I mean they are also Google is also learning and improving these things and you can also see that by you know the different CLI versions that are being released on an almost daily basis. So this is just getting better and better uh for sure. But yeah, that that was that was very uh was a very I mean the fact that I took screenshots already showed you how surprised I was.

24:52 Yeah. Yeah. And and I guess as a user, you kind of like end up losing confidence in this kind of thing when it when it's almost like pleading that it's it's it's not the right tool and it can't do it. You as a as a driver of this kind of a platform or tool lose massive copy. You start believing it, thinking, "No, maybe you can't do this. Maybe I do need to to switch to a different approach." Now moving to the to the spec approach when we let's let's contrast these now because you say of all of those integration tests those unit tests and things like that um it actually provides that much more debug information and and you know it it allows it it gives it more of those um more of those clues as to well okay what's working not working and potentially identifies these types of things in a in a better way right what what would you say were the major flaws of um of the kind of like approach of the vibe coding and what would you say

Enforcing guardrails and markdown backlogs for AI projects

3 Highlight · Maintaining living documentation with local markdown backlog tools

25:46 in spec driven was the key unlocks that you needed? Was it the tests? Was it something else? What what what really was that uh defining moment?

25:57 Yeah. Well, I mean, if you just do pure vibe coding and and just some prompting, it's just way too freestyle. There's no guardrails. That that's really not sufficient, right? You need to introduce these guardrails. And for me the guards was a good llinter to to start with. So that the llinter goes through the full code every time you do a deployment. That's one. The unit test and the integration test but also the management of your specs. Um and I think that's where there's a huge uh opportunity. I know you guys are clearly looking into that. Um what I used was a an open-source project and I've even invited the developer I believe he's from Austria uh to Devox Belgium called backlo.md and what backlod does he vcoded that tube also but you basically install it locally and it creates like this conbon uh UI in the browser and there you can create an issue and then that issue produces a markdown file and that issue

26:59 can then be referenced into cloud codes where you say okay now work on issue number one and because you're saying issue number one it will go to the file system try to find issue number one MD which the backlog creates and it will then work on that issue but what it also does is that once it has uh finished that issue it will document what it has done and include that documentation into the existing issue on backlog. So when you revisit that feature or that issue, it already knows the original complaint or feature, what it has done and what it still needs to be do doing. And it reminded me a bit of gyra in the good old days. Um, but this is like a lightweight version of that locally so the CLI can access it very rapidly and updated it and it then moves it from state of where it says created uh what is it um working or in progress and done. So it has like multiple states and

28:00 and that definitely helped um because you have your original regional specs but this allows you to actually have conversations on top of that spec or next to that spec which is a living document and I think that's definitely the way to go. Um as a result of course

28:18 I I I think it was a month or so uh Amazon came out with Cairo or Cairo or Cairo which is also spec to code. I had a quick look at that. Um, yeah, I wasn't fully convinced yet, but but it's definitely obvious that this is a a void that needs to be filled. Um, and and I think that type of combination makes it a bit more interesting because then you have living documentations and features and bugs and issues which can be accessed by the LLM.

28:49 Yeah. from the the the the the point of

Vibe coding delivers rapid prototyping but introduces risks

28:54 view of being a user, being a developer in each of these scenarios, you mentioned how fast you were able to kind of develop, create, and and and show something working and show something actually being able to deliver on your kind of 80 90% of of functionality. um for anyone having that feedback and that fast creation is is extremely fun.

29:24 It's it's it's very validating of you know you being able to kind of like create. You mentioned that kind of like top that that ceiling I guess that you hit after 90%. But from a specdriven approach, you have to do a lot more of the the kind of okay, let me write some stuff in natural language, let me now think about the tests, let me think about all these, you know, other things that you need to do which effectively will slow you down in that kind of like rapid application development uh approach if that's what you were trying to go for. How did you feel as a developer um the I guess enjoyment um of doing vibe coding versus spec coding? But also from a confidence point of view from a uh you know being involved in the process did you did you have more confidence with one over the other? Talk to me through that. I'm I'm calling this the the renaissance of developments or or you know because you you clearly are getting lots more uh

30:27 enjoyment uh and pleasure of doing these type of projects because you know if you didn't do VIP coding you need to set up your Maven projects. Just setting up the the build environment can already take you a couple of days to to make that work. And with VIP coding you know it it just creates that for you very rapidly. you get instant satisfaction so to speak. Um and and that that feedback loop is very is much shorter. I think in general just for prototyping quick prototyping this is clearly the way to go. Uh if you really want to do a real project that

4 Highlight · Accidental production deployment from deployment script bugs

31:02 you want to deploy for thousands and thousands of users you know is VIP deployment VIP debugging is that is that the way to go? Um I mean just to to to tell you or share you some anecdotes. I mean I had a development environment, I have a staging environment and I have a production environment. And so I went like okay I'm I'm creating a task file. This task file creates tasks which allows you to deploy etc. And it obviously had missed the the settings from production to staging and when I was deploying to staging it was actually deploying to production. So these are the type of things where you go like oh wow you know um you you do need to still have knowledgeable people looking into what it is producing not only looking at the codes but also looking at the scripts etc. uh because I can imagine if you're a a non- techy guy, a non-developer doing this stuff, you're

31:57 going to run into problems because you just don't know the scale of what it can produce and and and you just don't know all the effects and side effects of things that you are asking it to do. So I think being a more senior developer, it it it makes me definitely be more productive. I could have never created the UI that it has produced for me. Um, I mean that replaces a web designer for sure. Um, but the project I've now done in more or less five weeks now because I've I've been doing this for five weeks, it would have taken me definitely three people six months without doing VIP coding. I'm I'm I'm fully convinced about that. Um, so it's definitely a gamecher. it is more productive but you still need some senior developers guiding this uh because if you don't you're going to run into problems for example like Angular has guard uh rules so we have routing and you can put

32:58 guards in front of the routing if you don't know that concept you're going to give access to certain pages u as a junior or or non techy guy uh without knowing or you might actually expose

5 Highlight · Using LLMs for white-hat hacking and scaling analysis

33:11 those uh API keys or really security issues. And that's why I've also did certain exercises where I said like to close code, hey, you're a white white hat hacker. Try to hack my system. And it actually goes through your codes, goes to your website and tries to hack your system. And those are really valuable exercises because it will actually highlight, oh, I was able to see the admin page without being logged in. Okay. Well, fix it, you know, look at it and go deeper. Or I did another exercise where I went I went where I went like, okay, we have 3,000 unique users during DevOps Belgium. Uh, how will the system scale? How will be performance? Where are the bottlenecks?

33:56 And it does this exercise where it really then goes like well your Firebase functions, you should have better memory or more memory, you need to allow more instances, etc. So having these other

Comprehensive test coverage prevents silent refactoring regressions

34:08 type of discussions which are not so much development oriented but more devops oriented is definitely also a step you need to do within that process.

34:20 I'd love to break that down a little further. We there's a couple of things there that I'd love to talk through. The first one is you kind of you kind of alluded to the fact that you need those senior developers or or people who really understand have that sympathy of what is being built, what's happening to to review essentially or or to make sure to validate that what is being created um is actually correct. It's the right way of doing things. It's the right implementation so to speak. an amount of that can be done, I guess, with with things like tests and things like that.

34:54 Um, how how much did you do a lot of those reviews versus what it sounds like allowed the the LLMs or different LLMs to kind of like jump in and do those reviews for you? Were you comfortable if you just let DeepSeek or something else come through and say, "Tell me if this is okay and I'll trust you." Where was your limit?

35:14 No, I I was not fully 100% comfortable for sure. Um, just this morning I actually found another bug in the deployed application now because it's already being used now by by hundreds hundreds of people uh for DevOps Belgium. And so I did a a refactoring using prompting where the speaker can actually edit his pictures which are identified from Flickr using face recognition and he or she can decide which picture he wants to show on his speaker profile. And so that worked, but it actually broke a feature where the pictures are not shown anymore uh on the public page. And I went like what happened? And so I could not have I mean I could have only fixed that or anticipated that with using uh end to end uh testing using Cypress. I've chosen Cypress, but I had no test for that specific scenario. So it basically broke something in production and where I went like but to be honest this can

36:16 happen also in normal teams without doing V coding right I mean this is reality it these things do happen um so my target is I would love to get a 80% coverage with the unit tests I would love to have more end toend testing as well but that just takes still a lot of effort even if it's just prompting but it would increase my confidence that yeah if I do deploy it and I have those type of coverages um it it would increase my confidence to to to move forward. Um so I think those are definitely critical to to have um without you're just walking in a dark room and you're going to hit your head against the wall for sure, right? Uh so I think test driven or or just unit testing integration testing um is is becoming more and more important to make sure to enforce the specs that you've defined and I think that collaboration will will be a success combi.

Automating synchronization between code changes and specs

37:15 Yeah absolutely I guess like going forward with future development as well um let's say let me give you a couple of uh of of scenarios. If you were to create a new feature, um, would you create a new spec to do that or would you update and and append make changes to an existing specification?

37:40 So, I guess the question is

37:42 the life the the the the lifetime of your the life cycle of your spec. Does your spec follow the change or does your spec follow the application?

37:50 Yeah, currently not. Um, so the the spec does not follow any changes that I've made. um uh which is which is maybe a flaw in in my process. I should probably keep it alive and up to date. I do review and up to I keep the readme up to date or the cloth.mmd up to date um because you can easily do that but having tool tools which would help me keep that in sync even if it happens in the background I think that would be amazing feature to have from a tool perspective. giving you some hints here, Simon.

38:25 But that would be amazing because now I have to explicitly say it. I need to really say, okay, if I want to like this feature with the broken photos, um, that should be something which would need to be covered by some integration test. So it actually validates that that isn't broken, but it should be reflected in the spec. And because I'm lazy or I'm not thinking about that, the spec gets out of date. And if there are tools who would keep that in sync, that would be amazing. uh for sure. Um and and of course all these CLI tools they have rules which they go back to to follow that uh you know like the claude MD file or the Gemini MD file etc. Um so there's definitely still some room for for improvements but there's huge opportunities for product companies to fill that void.

39:10 Yeah. And so and so you're using specifications really to um to drive your development, but it's not necessarily specentric in the sense of any change you would make beyond that initial generation would more likely be directly into the code versus into the spec first. Correct.

39:28 Um and of course, like you say, that then puts the spec out of out of sync, I guess. And then it it almost like pushes you back into that vibe coding mode versus the the spec centricity that you found valuable the first time.

39:41 And I think that's a discipline I need to change. I need to have the discipline to actually make sure that the spec kept is kept into sync and that it's also always referenced when you start a new feature or enhance an existing feature.

39:56 Mhm.

39:56 Because that's a bit of the problem now is that you know we used to do I I've done several talks on rack but what you're seeing now with these CLI tools is that I call it file system rack. they do some kind of you know pattern recognition on files and the the results are then injected into the window context but if certain keywords are not included in certain specs it might not actually find these and include it in the window context so it doesn't know about it so I think that type of behavior either the developer should should have the the discipline to say hey no always include this specification like we do now with cloth and the I actually explicit ly say hey lots load cloud MD so it's part of the window context and just yesterday actually cloud released a new slashcomand called context where you can see how much of your window context is used by which types of documents which is very very interesting. Um so there's definitely

40:58 some some opportunities there to to make sure that it is included in the window context either by the developer which developers are lazy might forget. So I think tooling should should play a role there.

41:10 Yeah. But e even that though even allowing I even if your code isn't you know every time directly generated from the specification allowing giving access to you know your full set of specifications to from um you know to to your agent in this case Claude that that I'm sure that supercharges the way Claude makes its changes by truly understanding the application and how you know what your intentions are and by making certain changes knowing okay if I do that will probably break or whatever.

41:42 It's amazing how we talk about it these days like it's truly thinking like we are as well. It's it's

41:47 well it's very knowledgeable, right? I mean it really does know all the the syntaxes etc. But the other thing that just comes into my mind is like I do feature branches uh on GitHub. So I create a new feature branch and then I start working on that. But what actually would make more sense, I've of course integrated cloth codes using GitHub actions into GitHub. But what would make more sense is that you create first an issue on GitHub and then use that issue as your feature branch. But you can have cloth code codes or any other ALM already first do some pre-investigation to actually supplement your feature uh in GitHub and already use that as your window context information to start a feature. And but that's I I believe currently not possible. It's only be after the facts. Um but that would be actually I mean that could be a project on its own. Um because you then let let the LLM investigate and see what context

42:49 it would need to actually implement that

Pragmatic architectural choices ensure cloud scalability

42:50 feature. Then you can start doing the feature and and it already has the the context.

42:55 Yeah. Very interesting. I I I I really like the the I think context has always been king and I think what you've also discussed with the file system context is very very very very interesting from that point of view as well. I want to switch more into the kind of talking a little bit about maybe some of the non-functional uh requirements and aspects and I think this probably maybe when we when we're using LLMs drops us into more of a kind of like a rolebased assessment of what is being built. Um, so nonfunctional requirements, maybe it's security, maybe it's performance, maybe it's availability, these types of questions. Um, how did you obviously what you're building right now as a production application uh that needs to be needs to be up and alive for the for you know pre pre-Deox as well as uh certainly the days of DevOps and then probably an amount of time afterwards.

43:44 um you need to make sure that you know when 3,500 people are hitting it potentially potentially you know a lot of people at the same time for very time specific things like end of a session people are going to jump to the schedule uh maybe do maybe jump to the reviews and things like that per session um how do you how does how good is the LLM without specification I guess or or rather without you specifying it needs to be performant it needs to expect a large number of users at a a specific time. How good was it without specifying these are the requirements and and how did you get it to to um be conscious of you know these requirements so that when it's making implementation decisions and choices it does it in a way that satisfies these non-functional requirements.

44:35 But I think there I'm I'm guilty because I have of course a bit of experience of building scalable systems etc. And I I don't know if you remembered but during COVID we uh Celestino and I developed rockets which was an online platform to do online events etc. And there we we had the same thing. We had sometimes thousands of people online using it and I think just knowing the architectural choices as a more senior developer plays a big role there. and and so from the get-go I immediately said no I want to use Firebase which is basically Google cloud right uh I want to use lambdas or functions um because those are very very scalable especially if you then run them on cloud run etc so just that architectural choice of using firebase fire store functions or lambdas uh cloud run etc you already know that if it runs on GCP it it it can be very scalable because Google basically does that for you. So if I would have

45:36 started without that knowledge and it would have maybe created this more fast API using Python, I might have really problems, right? Because now it's just a configuration. I mean you say from the get-go how many instances you you allow, what the time out is, what the memory it gets from functions, and it's it's basically a stateless function serverless backends on a scalable cloud environment. So I could have if I had no experience at all I could have had sessions with LLMs asking like what is the best scalable solution and it might have come up with that type of recommendation but for me that was no no that was a an architectural decision that I wanted to have from the beginning knowing that that will scale and I'm pretty confident it will

46:26 and it's interesting that you've kind of alluded to the fact that you know as a

LLMs accelerate junior developer growth into senior roles

46:30 senior Being a developer, there are a number of choices that you've made that you know had you not had that experience in coding perhaps it would they're nonobvious kind of choices that that you would potentially make. Um a lot of people kind of like say well how in the future are we going to get to those those senior uh those senior engineers to know that. Um I kind of question that a little bit because I feel like well you know we all have to learn through our mistakes. We all have to learn through advice. Um, from advice from an advice point of view, we we've never been more capable of asking questions to whether it's LLMs or people uh through just because of the the the you know the new UIs and the new aggregation of data and information about how to develop, what good development looks like and so forth. We're also able to hit our mistakes quicker because we are able to vibe code faster, deploy faster. I question whether the gap is actually going to be there in the future because

47:33 isn't as a junior developer wouldn't I just make my mistakes faster and get to that end learning of okay this isn't scalable I need to use a more scalable scenario and make you know create those learnings faster versus as a junior developer making my code changes and finding this issue in six months.

47:53 There's two angles on this. um one is as a business owner I need a business solution as fast as possible as cheap as possible but with quality right and I think as a business owner having one senior developer even if it's myself now in this situation I can achieve that much faster than if I would have some juniors um so that's from a business perspective but if you look at from a technical point of view as a junior you now have a universal tutor you have a coding assistant which is out of this world so to speak. You can ask all the questions you want. Even the most stupidest questions which we would never have been willing to ask to our colleagues. You can now ask any question and you will get really knowledgeable answers. So if you're a motivated, dedicated junior developer, you can go up that ladder and become a senior developer very fast because you now have these LLMs which can guide you and help you. So this is something we didn't have, you know, a couple of years ago.

48:54 Um so you have to be motivated. The only problem is then the business is the business willing to actually hire these juniors and invest in them to become a senior if I cannot just accomplish a project with just one senior developer who would maybe be the project lead of these of these juniors. So I think that's a bit of the yeah the the balance act that that will need to happen moving forward.

49:20 Yeah. and and and also as a as a any developer senior or junior I can kind of say look to to an LM you are a performance expert you know you've got 10 20 30 years of of experience in creating performance systems maybe for a fintech maybe for a a site that is going to get huge numbers of of visitors on specific days you know add good coding practices for these types of systems And is that something that you did with this application with those types of roles uh role- based agents I guess?

49:58 So it was actually uh I think it was four weeks ago that cloth uh released soup agents. So what it allows you to do is actually create a prompt um for an agent that you give it a name which is called basically a soup agent and it's basically a markdown file of a certain character of a certain profile uh which it will use when you activate that soup agent. So you could say like hey you're for example a tester and you explain what the tester should do etc. And then when you actually do a prompt, it will look into its super agents and activate that type of avatar, that type of profile uh where it already sets the scene of that role and it will behave as that role for that specific request that you have.

50:44 Mh.

50:45 So that's why also you know for Devox Belgium I mean our our theme for Devox Belgium is basically Java agents. Um and you know we didn't mention that but I think DevOps Belgium is the biggest AI conference now because almost 75% are all about AI large language models but also using Java on GPU doing inference on GPUs etc. But mainly the the the theme if you go to the website it goes agents and I think that's where the excitement and the new opportunities are as well is that you cannot only create agents in CLIs where it has a certain role but you can also have asynchronous agents which work in the background while you're sleeping or while you're walking the dog and it's doing certain actions for you within that specific role. And that's where it's very exciting and it's also where even Java like frames frameworks like lang forj spring AI they're all going to this next layer of abstraction which are agents

Deploying AI-driven features for a massive live conference

51:47 and I think I think there we're going to see lots of new innovations happening in the next 12 months.

51:52 Absolutely. Now I hate to scare you um Stefan but what are we now we're recording this the 22nd of August. Uh you've got Devox Belgium. Uh what are the dates of Devox Belgium? Is it 7

52:04 from the 6th till the 10th of October?

52:06 6 to 6 till the 10th of October. Uh so what are we looking at here? Um about a month and a half really till till I have no idea why you're wasting time on podcasts right now Stefan. You should be there's probably a ton of stuff you should be doing uh for DevOps. But in a month and a half you're going to have three and a half thousand people probably more accessing this application. Uh a lot of them um a lot of them you know at the same time accessing the same data. What are your biggest concerns? Uh, and and I guess confident confidence-wise, how confident are you that this app is going to is going to work flawlessly? What are your biggest concerns about going live with this at a major conference?

52:47 So, I add I added some very interesting interaction possibilities, new interaction possibilities. Well, first is that like people can now finally have free text comments privately feedback to the speaker. But what we actually get often is that there are some less professional people who enter like okay certain texts that aren't really following our code of conduct. So what I added is that I'm using Gemini Flash 2.5 light to review all the comments and see if it follows the code of conduct.

53:20 Nice.

53:21 Right. And it really even if you're just using emoticons emojis and they're not appropriate, it will even find the emojis say like no this is not following our code of conduct. Right? So but that's it works but it's like okay how will it work if 10,000 people give feedback? Will it be fast enough? So those are like concerns I have from that point of view because it's it's a bit more difficult to simulate and and expensive if you want to simulate 2,000 comments at the same time. So th those are like my biggest concerns. Uh I know the application will run. I mean it's Angular. It's hosted on Google Cloud.

53:56 It's just JavaScript being returned. So that's going to work. Fire store is going to work. I don't see any issues there. Uh but like I have other things for example a feature where people can now say how full the room is. So they can say the occupancy of the room is like 90%.

54:13 And that information is now is now shown on the digital signage. So you will get immediately feedback in the digital signage saying, "Oh yeah, I don't have to go into that room because it's packed." And the cool thing about having that feature is that I don't have to go because I like to know how many people are in the room for a certain topic and speaker. Now if that works, I don't have to do that anymore. I can just sit at the registration, look at my dashboard, and see how full the rooms are based off communitydriven feedback. kind of feels like uh you know the how when you do a Google directions and then it kind of like says oh yeah this train is busier than normal and things like that. It's it's it's exactly that, right? That's that sounds awesome,

54:52 but it's communitydriven. So, the thing is, if that breaks, I'm

6 Highlight · Generating real-time conference talk summaries and transcripts

54:56 going to be vibe coding my

55:00 you know, I'm trying to fix it on the spot, of course. And I've got another crazy feature which I was thinking of only announcing at the keynote, but I want to give you the premiere is that I I vipcoded another feature where I'm doing uh real time transcripts. So I've already was doing real-time transcripts where you see the the transcript on the screen right I'm already doing that but now I'm capturing the transcripts from re real time from this talk and every two minutes I'm creating the key takeaways of the talk a cloud keyword clouds and also a summary of the talk while the talk is going on

55:38 amazing

55:39 so you can go to the mobile app and just see five minutes in the talk ah these are the key takeers ways. These are the keywords he or she is using and that's the summary. And I'm just hoping that's going to work. I mean, I've been testing

55:52 this, etc., but there's there's a few moving parts. And that so that's where I'm going to get nervous. It's like, will this work? That's interesting as well because as an attendee, if I w if I were to see something like that, I might scroll through like halfway through a talk, if I'm not maybe enjoying a talk or if I if maybe it's like not the right talk for me, I could actually scroll through and see what others are talking about what the summaries are and go, "Oh, actually that looks really interesting. Let me let me jump into that one." It gives people that more that real time that real time vibe. Super cool.

56:23 So, I've got real time transcripts and just imagine what you could do. I could go even next level. Take that transcript when the talk is finished, give it to an LLM and say, "Hey, create a script where two people are talking to each other." Then take that script, then go to OpenAI, create two voices talking to each other like Notesbook LM is doing. I could totally do that. Techn taking it taking from a technology point of view, that's totally possible. You know, the only thing is that some speakers have concerns about that because they don't like a real podcast from their talk, the accuracy, etc. So, I've put that in the fridge for now. But from a technology point of view, that's just like it's like science fiction. How the hell, you know, two years ago that was not even

Developer conferences must embrace AI while keeping low-level depth

57:08 possible and now you can do that.

57:10 Let's talk about I want to talk I want to kind of like finish up with two two last kind of questions. I want to I was going to originally we talked a little bit about the role of of of a developer a little bit. I want to talk about the role of a conference Stefan when we talk about you know in the day jobopolis moving to DevOps it was it was a very languageoriented conference what's what do you think the future of a langu of a developer conference looks like because we've already talked just now about you know your and I love the I love the framing of saying you know DevOps is one of the largest or the largest you know AI developer conferences in the world and it's Like do do conferences need to evolve now? Because as the role of a developer changes, the skills that a developer needs to learn to actually become an effective developer isn't now or is going to be less and less oriented around how they use a language or you know or their capabilities of of whether it's design or architecture because we

58:12 will lean more and more into AI. So is it more a combination of our best usage of AI and some of that mechanical sympathy knowledge of of how applications should be built? What what's your vision for DevOps going forward?

58:27 Yeah. So I always say I mean I spend almost nine months producing and organizing this event and it goes just like that. It's not full-time nine months but you know so I I always say like this is our party you know this five days event is a party that we organize where we handp speak and topics etc and so I want to reflect the content from the conference where the excitement is where I'm getting excited and so my excitement is clearly in AI with large language models etc and I want to make sure that that is reflected in the conference so that's that's one thing but the other thing is also yeah if up till now my excitement matches with the excitement of other developers and they're curious to learn these new things etc. And so I think we did pivot a bit already this year because I clearly took the card of like okay AI agents and LLM and coding assistants is where the buzz is and the excitement is let's go full-blown for that for Devox Belgium. And to be honest, on Monday I

59:28 was really nervous because I it could be that people find it as a as a how do you call that their their job security and as a danger and they might just be reacting on the as a oppositely of what I was going for. Um but it looks like people are excited about this. They want to learn.

59:49 Well, you only had a few seconds to wait before getting your answer at least.

59:54 But it's also a reputation a bit, right? people do know what to expect from Devox. I mean, we've been doing this for 25 years now. Uh so people know what to expect. But but you're definitely right. I I do if you just look at the content already for this year. There is there's going to be we we need to embrace this new possibility and I think the excitement is there. The productivity is there as well. There are dangers involved for sure, but as a conference that's something we need to highlight uh make sure that we reflect it from the speakers and content point of view. I mean, I I even have Rod Johnson, the founder of Spring, you know, he lives in Australia. He's coming over to do a hands-on lab session. He's doing a keynote and he's doing a conference talk about his new AI agent Java framework called Embab. So, it's like that's where the excitement is. We want to know what's possible, etc. And if you reflect that as a conference, I think you still have a few additions to to to stay alive. Um, I've already seen a few other conferences doing the same strategy in

1:00:55 the US. I mean, they're already going, okay, yeah, we need to embrace AI. But what is also interesting, even from a low-level point of view, there's still lots to talk about. I mean, we have Paul Sundos who works on Babylon who is working on Java and GPU interference. We have the tornado VM guys from Manchester University. So there's still lots of low-level stuff that needs to be discussed and explored and and and do knowledge transfer on. So it's going to be mixture of those two. Uh you might have people who are no developers at all who might actually get some benefit of going to DevOps. But you still have people who want to know what's under that layer, what's underneath that framework, what's actually, you know, opening up the car as a small boy and knowing what's in there. I think those types of people need to be addressed at a conference.

1:01:43 Yeah. And if you do, great. Um, I need to stay excited and have fun organizing it. If that stops, then Devox Belgium will stop for me as well because it's like I I don't have to do this, right? Um, and the the only thing I really hate is that you create this great party, but I can't really enjoy my own party because I'm running around like a headless chicken chicken like from one fire to the other making sure everything runs smoothly. Um, and so that's the biggest disappointment is that I can't enjoy the conference as a normal Latin D. Uh, which which is a bit of a shame, but luckily we record everything so I can see it online.

1:02:21 And I think that's the that's the joy of it, right? If you know you're enjoying it, you know there's a really strong, you know,

1:02:27 that's what you would expect.

1:02:29 Yeah. Exactly. Exactly. Yeah.

1:02:30 And if that stops, it stops. I mean I mean it would be totally fine to have a few sobatical years and just doing R&D. Uh but I don't think we we're at that point yet.

1:02:41 Stefan, I'm I'm sorry to inform you. I think it's actually going to I think it's going to grow from here. So, you know, your job is well safe and and and yeah, sorry to add more to your plate, but uh absolutely. Yeah. Cool. Stefan, this is this has been this has been brilliant and um a real insight into kind of like how your um your approach uh needed to change uh from from developing an application uh going from kind of more of a a vibe coding to a spec driven approach and and fingers crossed uh DevOps and and and I'm sure it all go well be another amazing conference. I'm looking forward to being there again. I've been there many years and and looking forward to uh to to to seeing you in person again and and and being there in October. So, all the best for Devox. For those uh for those who want to check it out more, uh devox.be, I think, right?

1:03:32 Correct.

1:03:32 Devox.be and um and yeah, all the best. Thanks very much, Stefan. It's been a it's been a joy to chat with you.

1:03:38 Thank you for the opportunity and looking forward to the Tesla product being released uh in the near future.

1:03:44 Thank you. Thank you very much, uh Stefan. And thanks very much for listening. Tune in to the next episode. See you soon.