From Unit Tests to Whole Universe Tests (with Will Wilson)

12,625 words · about 50 min to read

0:00 It's worth testing your software, right? Let's just make sure we agree on that. But how do you do it? How do you make sure that software is correct? I think it's a lot harder than we'd like to admit cuz there's always something you didn't think of. You could go full-on TDD. You could have a complete suite of unit tests with 100% code coverage and your software can still have bugs because largely because unit tests only test components in isolation. And a lot of the complexity in a system comes from the way things interact.

0:32 But we know that. So we write some integration tests, but not too many because they tend to be slower and more brittle. We emphasize unit tests not because they're perfect, but because they're practical. But unfortunately, neither of those are sufficient. Both unit tests and integration tests suffer from a big blind spot. You only write tests for the things you think are going to fail and a lot of bugs come as a complete surprise.

1:00 So what can you do? I think the most promising technique I know of is property testing. We've covered it before on this podcast and the essence of it is every function has some rules that it should obey no matter what arguments you call it with. Things that should always be true. So let's throw random arguments at a function and see if it ever breaks its own rules. There's a lot more to that technique and if you haven't looked into it, I strongly recommend it. It's a great way to test more cases with fewer code, but it's mostly a unit testing technique. Is there the whole system integration testing equivalent? Is there a way to test all the possible things that could happen in the world affecting a system we've created and check the entire system behaves itself? Can you ask a question as innocent as no matter what happens, does the database stay up? Can you test it? I don't think we can. I think there are a lot of things that make that kind of question impractical.

2:02 Apparently, Facebook put a lot of effort into building a system that could try and answer that sort of question, and they seem to have given up. But joining me this week is someone who thinks he's cracked it. testing the properties of entire systems and simulating entire possible worlds. He's gone further down the testing rabbit hole than anyone I've ever met. He's Will Wilson. He's the co-founder of Antithesis and they go all the way down below the operating system to build entire virtual worlds, hypervisors of controlled chaos. It's an absolutely huge project. I love its ambition. I love the problems they've faced and the solutions they've come up with. They're absolutely fascinating.

2:47 And I love the fact that in order to get this working, they had to play a lot of Nintendo games. Let's find out why. I'm your host, Chris Jenkins. This is Developer Voices, and today's voice is Will Wilson. I'm joined this week by Will Wilson.

3:13 Will, how are you? Doing great, thank you. I'm glad we've got you here because you are going We've had episodes on testing before, but I think you may take the biscuit as the most hardcore testing episode we've done so far. You That's That's really funny. I don't really think of what we do as especially hardcore. I think we're just trying to mirror what happens in real life, which is what a test should do. That's why it's hardcore because real life's

Standard test suites miss unexpected production behaviors

3:39 dreadful and goes wrong all the time. That is that is super true. Okay, give us the overview and we will dive deep into the details. What what kind of testing we talking about and why is what we've got today not enough. Sure. So, a a sort of a a cheeky question that I like to ask people sometimes is let's say you make a change to your code. You know, you you modify it. It's some feature that your boss has been asking for for a while and you know it's a pretty big change, right? And you you run your test suite on it and the test suite comes back all green.

4:12 Are you now 100% confident to roll this

1 Highlight · Traditional tests only cover anticipated failures

4:15 out to all of your customers right now? I'm saying no. If it's a large change, no, not just back test and and and so why is that? Well, I think for most people the reason is that actually our test suites don't really cover the full spectrum of behavior that we could see in production. And the reasons for that are actually a little bit deep um almost philosophical. It's that if you are writing a test to go into your test suite, that's some test case, some situation that you've anticipated in your own head. And by the way, if you've anticipated in your own head sufficiently to write a test case for it, there's a very good chance you've also written the code correctly.

4:55 Um, but the things that happen in production, the lovely things that set our pages ringing at 3:00 a.m. and get us angry emails from customers and and and so on, those are the things that we didn't think of usually. And so almost by definition, if you are ahead of the game enough to write the test, it's not actually the thing that's going to cause you a problem. Um, you need something that is able to find the things that you would not have thought to test for. And that can be environmental situations. It can be weird thread interleings, weird timing situations, weird user behavior like weird resource usage patterns. Just all the stuff that that it just never even occurred to you could actually happen. This feels like you're pushing me in the direction of property testing, which I really like.

5:47 That's exactly what we're doing. Yes. Okay. And exact. Exactly. Like this is all motivation for property testing, right? What we want to do is not write a test. What we want to do is write a test generator or like a fake client or, you know, something that we can just run. And the longer we run it, the more different situations it'll come up with. and and throw at our system until we eventually actually hit the crazy thing that some user is going to think to do and you find that you know in your tests long before it rolls out to production and long before your pager goes off. And so that's the basic idea of what we're trying to do. What I think is novel about antithesis and different from normal property testing is that we're able to take that mindset and that approach and extend it from small data structures, you know, small

6:48 pieces or modules of programs to like entire large real world interacting distributed systems. Um, I think that I mean the one of the things that makes get doing property testing difficult is coming up with good

Start with basic crash-prevention invariants

7:03 properties. And how do you that's the first thing I want to know. H how would I specify something that was useful and interesting and non-trivial about a system that large, right? So there's a there's a few there's a few things that are working in your favor here. Um the first one is you can actually catch a vast amount of bad behavior without specifying every single property because most bugs will cause multiple properties to fail. Right? So for example, let's say that I've got some C++ binary and it's got some memory corruption in it, which goodness, very believable.

7:37 Yeah, I've I've done that once or twice. Um so that could result in like a wrong answer. It could result in like a security bug. It could result in a user seeing some nonsensical garbage memory like in their response or it could result in the system crashing and you know so basically if I have a property that's checking for any of those things it could eventually find that bug because that bug can manifest in many many ways and so what we find is that actually getting a very very small number of basic properties in place is often enough to immediately begin finding really high severity, really interesting and important bugs. And so with our customers, we usually just start with properties like should never crash, you know, it should, you know, I shouldn't see random uncaught runtime exceptions, you know, I should like it shouldn't time out and never come back to me again. Like, you know, like pretty basic stuff like that. You know, the server shouldn't be giving me 500

8:38 errors, like whatever. And often we already get to some interesting stuff. And then the next level deeper that you go is like if you're building large real world systems, usually these systems are being built by distributed teams or they're being built by multiple teams of people in the same building but who are still parts of separate organizations. And so there's often a lot of documentation and a lot of either formal API guarantees or like written in a Google doc somewhere guarantees, you know, about what the system is meant to do and about what the different components are meant to do.

9:13 And so you can convert some of that stuff into into properties as well. You can be like, look, the the FUBAR service, you know, it will always return you an integer, you know, within 300 milliseconds, assuming it's under this amount of load. And you know, if it if it loses two replicas, it'll eventually come back once you you know, like that stuff is often written down somewhere in terms of an SLA or in terms of a promise that one part of your company has made to another part. And so, you know, turning that into properties in code is a little bit of work, but it's not that much work. You're making me think of things like um like Selenium, right, where you do browser testing where you're where you're trying to say people are crazy and the system under test is complicated. And I find that the work to set up a good test and maintain it and actually test all that crazy behavior is a big thing. So my next question is how

Relaxed specifications improve long-term test maintainability

10:10 do I specify a test worth running on a distributed system without going insane? Yeah. So, this is a great question. Um, and I'm I'm I'm I know I'm speaking to engineers, so I don't want to sound too positive because if I sound too positive, I will trigger everybody's detector. Um, but I'm going to I'm going to take a risk here. So it turns out that the same thing that makes property based testing or fuzzing or autonomous testing or whatever you want to call this class of techniques really powerful which is that you've relaxed your specification of

2 Highlight · Property tests resist breakage from incidental code changes

10:47 what the test is going to do also makes it more maintainable. And I'll give you a really let me give you a really really concrete example please. Um let's imagine we're testing Super Mario Brothers. Okay. the you've got that in your head, right? Yeah absolutely. The traditional the traditional way to write a test with Selenium or with, you know, whatever your preferred test framework dour is or just, you know, with nothing, right? Just with your programming language of choice.

11:20 Yeah. Is to event basically describe in detail a set of steps and then describe the expected outcome after that set of steps. Yeah. Right. So, in the case of Mario, it's going to be something like, hey, if I hold down the right button for 2.3 seconds, and then I hit the jump button and I hold that down for 1.4 seconds, and then I do something else, I'll wind up in this place, and I'll have bounced on this gloomba, and you know, I'll be I'll be I'll be at this location.

11:47 And I think that makes it really clear like why that's so awful and unmaintainable, right? It's like as soon as I change not just important things about the game, but completely incidental things about the game, my test will break. If I change I've had that experience with browser testing often and I hate it. Exactly. And it's really bad in distributed systems testing because timing and like the timing of network requests and like when different tasks execute can be really flaky from run to run. And so it can even even if you don't change your software, the test can break.

12:21 Yeah. Um, so the cool thing about property based testing is that you're not making that kind of claim. You're basically saying things more like, "Hey, here are the buttons you can press in Mario. Here's the things you can do. Mario should never wind up inside a wall, right? The game should never crash. Like the score counter should never overflow and become negative, you know, like whatever." Um, but what that means is now if I change the game, if I change the level layout, my test actually does just keep working and it just keeps still testing my game in the same way. And so I've simultaneously made my test more powerful because I'm testing many more different possibilities, not just that one path through the game, but I've also made it more maintainable and less fragile. And so to now take this to a more distributed systemsy kind of context, right? Instead of saying, hey, if I send this request and then I wait exactly this amount of time and then the server does this and then this packet gets dropped, then I expect this retry to occur and blah blah blah. If I

13:21 instead say like, hey, you can send requests, you can retry things, the network is spotty. Um, but eventually you will get this response after some number of retries or you know this. And by the way, the server will never crash while you're doing this. And by the way, if there's two clients that are both trying to do this, they're never going to deadlock if they're running the same process or whatever. Um, you you make it so that incidental changes to your program don't just automatically break all your tests and you don't have to spend as much time maintaining them.

13:52 Okay. Okay. I can see how that would

13:55 work if and I'll I'll accept your claim on face value that this does actually find bugs in the wild. I can believe that. But to use your Super Mario example, right, that how well how old is that game now? What 40 years? Yeah, it's pretty old. Speedrunners are still finding glitches and bugs. Yes. How do you when I write this simple property test, how do you not end up spending 40 years going through the random search space to actually find me a useful case where things crash?

14:24 Ah, that is a good question and that is a lot of what makes this hard. Um, so, so, so, so I'll I'll I'll tell you a few things. First, number one, the thing about Mario is that is that people aren't trying really hard to make it bulletproof today, right? No, like nobody's like gone and patched Mario to not have all these glitches and bugs and nobody's like, you know, besides us has thrown a fuzzer at it to try to systematically find all the bugs.

14:58 Wait, are you saying you've actually done that or just hypothetically? Oh, yeah. Yeah. Yeah. No, you can go on our website and you can see Antithesis playing Mario and a lot of other Nintendo games, too. Um, okay. This is good mad science. Yeah. Yeah. So, we have actually done this with a whole bunch of different Nintendo games. But the thing is like the thing is in the real world, number one, you don't actually need to find every single last bug in your program.

15:26 You need to find the ones that are really likely to cause you a significant problem at some point. And number two, if you start early in your development process finding bugs quickly and reliably the moment that they're introduced, it's actually vastly less effort to fix them because you don't have to do some big complicated root cause analysis. You can just be like, "My tests were green before this commit and afterwards this weird issue, this weird timeout started happening. I guess it was that commit." You can look at that commit and scrutinize it or you can revert it. And you know that actually is

Guided search optimizes the vast state space

16:06 much much faster than going back to some old legacy system and trying to patch everything up. Yeah. Okay. I buy that. I'm going to have to challenge you on that though cuz you started with the example of if I make a major change and the tests say everything's fine, am I confident? One of the reasons I'm not confident is in a sufficiently large system, I haven't written all the tests. I haven't reviewed all the tests. I don't know how much of our footprint we're testing and how thorough we've been. And if you're So what you're saying makes sense when the code base is young, but every decent code base grows to be old and large and then you still got that big search space problem we're talking about.

16:50 That's true. Um, so I will say there's another like that there's two separate questions here, right? And and I think maybe I've confused my answers to the two. One question is I'm making a lot of changes. I have a very complicated system like how can I possibly keep the quality bar high assuming I have something that can find all the bugs quickly and that's the question that I just answered you. Um I think the second question which you are asking is also a very good question is this is a tremendously complex system.

17:26 How can you possibly find all the bugs quickly and reliably? And the answer to that question gets a little bit to the magic of antithesis because what we're doing is not just exploring the state space blindly, right? Doing that would, you know, would would take longer than the lifetime of the universe. It would take, you know, it would take a trillion years to find a winning route through Mario that way, right?

17:54 Yeah. Yeah. just randomly blindly trying trying infinite number of monkeys coming in 2027. That's right. But what you what you what you want to do is use guidance and use feedback from the system under test to optimize the search and notice when things have interesting things have happened, things that aren't necessarily bugs, but that are rare behavior or special behavior or unusual behavior. And so the test system can see that something interesting has happened and follow up opportunistically on that discovery. And that gives you a massive lift in the speed of finding issues.

18:36 And the way that we're able to do that is with this sort of magical hypervisor that we've developed which allows us to deterministically and perfectly recreate any past system state. So people generally think of the value of that hypervisor as like any issue we find is reproducible. Nothing is nothing is works on my machine. If we find it once we can repro it for you add infin item.

19:03 But the sort of real benefit of it is that we can do that while we're searching for tests or while we're searching for bugs. So for example, step back a second because uh I need you

Run unmodified Linux inside a deterministic hypervisor

19:14 to because we haven't actually touched on the whole hypervisor. You're the way you do this under underneath the spec I write is you're simulating a whole operating system. Yes. You need to explain that a bit first before we get into the magic of it. Yes. So, so basically what we're doing is we are taking your real system like you would deploy it to production and we're deploying it into a fake copy of production. And this fake copy all runs within a single you know a single guest operating system environment which is usually Linux and you know if you have multiple services or multiple nodes and you've got a database and a load balancer and a web server and whatever these all run in different containers interacting locally in that operating system right and if you have dependencies like AWS let's say those also run inside there,

20:14 right? We have a whole fake AWS that we can deploy in there with you. And oh, you know, we have lots and lots of other, you know, well-known cloud dependencies that we can just sort of put in there with you, right? And so then we have this sort of magical hypervisor which runs all of this just like you know like a normal VM would but the difference is that our hypervisor if it does the same thing twice it will like actually for real do the same thing twice. like all of the like very low-level decisions about when threads get scheduled or how long particular operations take or you know exactly how long a packet takes to get from node A to node B will reproduce 100% perfectly from run to run. I can I can kind of make a guess on how that's working, but I'm gonna make you spell it out for me because how are you doing that without diving into the networking stack on Linux and then the clock processing stack on Linux and all these

21:16 different subp parts of the kernel. Yeah, it's it's actually very tricky. Um the so so so the thing that you just described um is actually how Facebook once tried to do the same thing. They built a system called hermit which basically is like a deterministic version of Linux which tries to get this same property by modifying the kernel the networking stack etc etc the file system making them all deterministic.

21:48 Um, and that project I don't want to talk trash about, you know, but like basically it's it's abandoned, right? It's like not it's not being continued. Um, because because it's very very hard and because those things don't present stable interfaces, right? The kernel is constantly changing, you know, file systems are constantly changing. It was just a moving target and a very hard one and a very big one even for Facebook. That's a massive project right?

22:13 Yeah. Yeah. So, so we decided to completely bypass all of those issues by going in at a much deeper level. So, what we did instead was we run a completely unmodified Linux, completely unmodified user space. What we've done is we've gotten underneath all that and in the hypervisor we've implemented a deterministic machine model. So, okay, from the point of view of Linux, it's just running on a computer. This is just a very strange computer where operations always take the same amount of time. You know, if you if you do them again and so right and and you know and if you happen to ask for a random number at some particular moment, you will get the exact same random number the next time

Intercepting non-deterministic calls maintains near-native speed

22:59 you like go and do this. And so that means that we can test any software in this style without modifying it or changing it or imposing a bunch of requirements on it which makes it all like a much easier lift. So are you saying you've got a thing that will say okay so the CPU has returned this value and then the clock ticked and then the networking bus transferred this data and then the clock ticked.

23:27 That's right. That's right. And and the nice thing is we can do that because we've made the actual execution deterministic. We're not like recording or writing down anywhere that we did exactly this and and exactly that. Right? That would that's how there's other systems that work that way like RR if you're familiar with that. Um record replay. Um it's a system that basically records the result of every system call your program does and lets you replay them all again later which is very handy. Um but you know only works for a single process and but more problematically also it just generates this huge volume of data. I bet. Yeah.

24:04 Because you got to record every single thing the system ever did. But the nice thing about making it actually be deterministic is you don't have to record anything. The only thing you have to record is what were the starting conditions and then at what point did I reach in and like inject entropy or like change the future in some way, you know, and then everything else just evolves, you know, like like those 18th century guys thought the universe evolved, right? Like pure determinism governed by one equation. Like that's just that's all that happens and all that can happen. If you could know the position of every particle at the start, you would know who's having breakfast right now.

24:45 Exactly. Exactly. It that's how it just actually works. Um, and so it means you it's like tremendously more scalable because you don't have to store all this vast amount of data. I mean, this feels like a facicious question, but I do have to ask, does that actually work? It does. Yeah, it does. Um, and it wasn't it wasn't uh it wasn't like we weren't sure it would work, right? like you know when we when we tried it you know we had no idea if it would work and we had some dark moments um we had some really dark moments while building this right we had we had a few times where we thought yeah we've got it it works and then we would discover like one in every trillion CPU instructions you know something is a little different and that drove us insane for a while but we eventually got it so it's like no it it's it's really determined terministic um and and like this is not vaporware, right? Like we've got very large customers who are using this every single day and it works.

25:47 Have you ended up building basically like uh Intel chip with all the event buses in software? Uh no, because that would be too slow. So that that would be um that's another way that you could try and do this. the the the trouble and and that would actually give you some benefits over what we've done, right? If you did it that way, you could like a cycle accurate CPU simulator, you could find all kinds of like weird bugs that required like true multicore parallelism or like you know weird me atomic memory operations, stuff like that. Yeah. Um, we are not trying to find those bugs because 99.999% of developers can never even think about those bugs, right? Like we're trying to we're trying to find we're trying to find more more everyday type stuff.

26:43 And so our approach sidesteps a lot of that. Um what it does architecturally fundamentally is we say look most operations that run on a CPU can just run like you know normally at full speed on the host CPU because the result is deterministic. How long they take is not deterministic. And so we do have to completely fake the clock, right? And ensure that when I add two numbers on the CPU, that always takes the exact same number of clock cycles regardless of where things are in the memory hierarchy or whatever.

27:21 But right, but leaving that issue aside, it's deterministic and can execute at full speed. And then if we detect that you are trying to do something non-deterministic like read the timestamp or get a hardware random number or something like that, we can trap that instruction and return a deterministic answer. But what that means is 99% of your CPU instructions are just executing on the host CPU and it's very fast. Um and so that that means there's not much performance overhead at all to doing this which is which is I think really important to making it actually practical. Yeah, because we're going to talk about your optimizations, but you're still going to have a large search space. You can't afford for the CPU to run significantly slower than usual.

28:04 That's right. And there's even a there's even a neat trick here which is um we can actually run the CPU a bit faster than real life sometimes because if this if the guest CPU is idle, we can just advance time faster than real time to the next time it would do something. Right? So anytime that the CPU is halted, we just fast forward to when the next interrupt would occur. And so what that means is we can you can think of it as like running the host CPU always at 100%. And then if the guest CPU is running at less than 100% it ends up running faster than real time. You know an analogy here is let's go back to Mario, right? Like if you're trying to emulate a Nintendo game on a modern computer, you could actually run that emulator much faster than real time because the host CPU is much faster than the emulated CPU. And so we can do something very similar if your system is idle. And that's really very powerful because distributed systems, client

29:05 server systems are very frequently idle because they rely so much on waiting for network communication or waiting for events to occur.

Eliminate network side effects with fake cloud environments

29:13 Yeah. Yeah, I can totally see that. Um, tell me about then tell me about how you interrupt the networking stack and making that deterministic because it sounds like you've got a problem where like you've got to simulate the entire internet. If I access the internet and stuff that in Yeah. So, so there's there's two there's two levels on which I can answer this question here. One is um the actual networking stack itself, right? like in terms of I have put this data in this buffer and now I'm calling send on it.

29:48 We make that deterministic just the same way we make everything else your computer does deterministic like it you know the timers the timing everything is just going to be the same. Yeah. Um the the problem is exactly as you say. If my system goes and tries to call out to some thing on the internet that thing could return a different answer the next time I call out to it. Yeah. And so that's why we just ban that like and and that and that is like a limitation and a friction point for using this and it's why we had to do things like give you a fake AWS and give you you know a fake everything else that you need. Um so so what it means is is that you know you need to take your application and all of its dependencies you know your database server your you know whatever whatever things need to actually be able to run to provide the service and provide them all inside of the hypervisor. Um, we have some thought

30:50 that in the future we may be able to relax that limitation by recording just what happens at the network boundary so that we can go back and replay it exactly right imagine that you call out to some I don't know the chat GPT API right and you get some particular answer coming back we can just note the moment right like all the events inside the VM are already known because they're deterministic. So like if when we replay it, you'll make that call at the exact same moment in the simulation. And then we just need to record when the answer when we delivered the answer back to you and what the answer was. And that's a very small amount of data, relatively speaking, at least for a lot of applications. And so we may be able to relax that at some point. Um but that's not currently a feature we have.

31:42 Presumably, you'd have to save that for the next time you run with the same seed right? Yes, exactly. You'd have to you'd have to it would basically be a form of input right? Yeah. This is absolutely ringing my functional programming bells. The hard part is always side effects, right? That's exactly right. That's exactly right. And so, and so what we're trying to do is just say there are no side effects, period. Um, and you know that that that actually can get you quite far um if you've got the leg work to do an entire fake AWS.

32:15 Yeah, that's right. That's right. Are you doing like like things like the problem with AWS is you don't have the source code? Right. But are you are you doing like a fake Postgress or is it enough to just run Postgress in another hypervisor? Uh, you just run it in the same hypervisor. Okay. You just check it in there. It's in another container. Yeah, we have tons of customers who do this. Um, it's that that's very easy. Anything that's open source like like piece of cake, right?

32:42 Because it's just running on Linux, right? It's just it's just running. That's right. Like the the thing that's hard is is things like AWS. But it turns out that 99% of people just use AWS like you know like if you get that one you actually get a tremendous a tremendous amount of the market satisfied. Um there is a long tale of third party things. You know, the good news is here also we actually can leverage the open source community a bit because a lot of people have written emulators for these things to use for their own local testing, right? Like there's a pretty good Snowflake emulator out there. There's like a pretty good BigQuery emulator. Um there's like a there's like a lot of these local local emulators and so often those are a dropin replacement for the for the service and then we can just kind of go and we're we're off to the races. You'd be no worse off than your existing test suite.

33:33 That's right.

Leveraging determinism for time-travel simulation

3 Highlight · Time travel testing enables exponential search space speedups

33:34 Yeah. Okay. Okay. So, you're simulating an entire operating system. I see that you're overclocking it. Let's say my normal process runs at 10% CPU usage. You can run 10 times faster. That's still not enough. Take me through how you were talking about how you optimize the search space. Take me through that. Yes. Great. So the really key insight is that the determinism gives us the ability to travel through time, right? Because if I get to somewhere interesting in my program and then I make a mistake and I lose that thread, I can just be like, uh, let's go back. And I don't have to save anything. I don't have to snapshot anything. I don't have to I can just start the simulation again and do the same things that got me to the interesting place in the first place. Right.

34:31 If you had the same random se the same pseudo random seed, you get exactly the same series of operations. Yeah. So, so, so, so far that's not that exciting. Like, who cares? But the reason it's really exciting is that there's a lot of bugs which sort of look like I need really rare event A to happen and then I need really rare event B to happen and then the bug happens.

34:57 And so like let's say let's say that event A and event B are each one in a thousand right? Yeah. Well, the probability of getting A then B is one in a million. And that's a lot less likely. And if I have event C that also needs to happen, now we're one in a billion. Wow, that's really unlikely. I'm never going to see that in my test. I'll just always see it in production, right? Because I've got millions of users. Yeah.

35:22 So that's that's kind of why testing is hard in some sense. But the crazy thing is once I have a time machine, once I have a hypervisor, I can run until I make event A happen. And then if I notice that event A has happened, I can say this is interesting. I want to now just focus on worlds where event A has happened. I don't need to refind event A every single time. I can just lock it in, right? It's like if you play computer games, it's like save scumming, right?

35:57 It's like I can I can just save my state when I got the boss down to half health and now always reload from that point. And so it takes me a thousand trials to get event A to happen and now just another thousand to get B to happen instead of it taking a million trials if I always have to start from the start. Yeah. Yeah. Yeah. And then I can get to C after 3,000 instead of a billion. And you can see how the more such incremental intermediate steps I've got, the bigger a speed up I get. And in general it's like the nth root asmtoic speed up which is very significant.

36:31 Yeah. Yeah. And so the key then is can I

Guiding exploration through code coverage and logs

36:35 reliably recognize that something interesting has happened? Right. Yes. And what what is interesting? Right. You've anticipated my next question. Right. So so and the good news though is there's all kinds of clues. There's so many clues. Um you know one thing is we're running this all in our hypervisor and so we can see which lines of code are running. And so if some new line of code in some new function runs for the very first time, that's probably interesting. We should probably try following up on that a bit.

37:06 Hang on. How are you noticing that from down in the very close to CPU level? Well, okay. So, we actually can just see call stacks from the hypervisor. Um, and that is an option that we that we have in our back pocket. But the way that we practically speaking usually do this because it is a more delightful overall product experience is we actually have our customers instrument their code with callbacks that tell our platform that we've gotten to particular places and that's that's a better overall product experience because it means we can also report stuff about coverage to them. You know give them insights like hey this bug only happens when this particular line of code runs. We can associate to actual function names and so on. Yeah.

37:53 Um and we have good support for a large number of languages and it depending on your some languages it's literally entirely automatic and the customer doesn't even have to do anything. Um so so basically right so which lines of code have run and which new functions have run is a one very good clue right another one is like developers often put log messages when something interesting has happened right like okay and and and sometimes it's sometimes the sometimes the log message is like this should never happen or something like that right and so if our system sees the log message this should never happen yeah that might represent a test failure or it might Not, but it probably means you've gotten to something that the developer thought wouldn't happen.

38:38 Yeah. And so now probably we can like, you know, we can we can now focus our exploration on what happens next and can we get something really interesting to happen. Um there's other more speculative things that we have uh ideas that we have. You know we one thing that we can do for instance is you can look at any time the code branches like you know when we're going around a loop or something or when we're taking an if statement and you can see like what is like suppose there's a loop and sometimes you let's take an if statement it's an easier example okay suppose there's an if statement and you know we hit it a thousand times and 500 of those times, you know, we do the if and 500 of those times we do the else, right?

39:30 Yeah, that's probably a pretty boring if statement from the perspective of trying to find deeper behavior. But if we have a different if statement in the program and we hit it a thousand times and it always takes the if every single time we've reached it. Yeah. Well, now if we do get the else one time, that's a really interesting situation. like, you know, that's unexpected. And so our platform is smart enough to be like, "Oh, we got the else that time. Well, let's let's try and just start from there and let's try and just really hammer this else case and see what else we can make happen down that path."

40:12 Okay, that gives me a bunch of questions. Um, the first I think the first one is just to make sure we're understanding this. Surely you found a seed that gets me to this one in a thousand else case. Yep. Right. But now I want to fork off in a thousand new directions. Yep. Are you saying are are you you can't be storing the

4 Highlight · Bisecting backwards to find when a bug becomes inevitable

40:36 state of the machine from that seed and then introducing a new seed. Correct. But I don't see how you take your original seed and modify it to be slightly different. Uhhuh. Uhhuh. Yes. So what we are doing actually is periodically reaching in to the computer and receding everything. Oh snapshotting the state. Well, we don't even have to snapshot when we do this. We're just reaching in and we're changing we're just changing one register. We're just being like, hey, the next time anything in the whole computer asks for a random number, instead of giving it a one, give it a two and periodic. And that does get recorded right at instruction number 37 trillion. We decided that the next time they asked for a random number, it would be this. And so when we're trying to explore more and see what new things we could find from some given point, we just go back to the last time we did

41:36 that and we change the decision we made there. Ah, okay. Yeah. and and and and so and what's really cool about that by the way is it has a completely separate side benefit which is when we actually do find a bug we can then go back and and ask when did the bug become inevitable right this is this is kind of kind of crazy how how right we can we can we can go back to the previous time that we reached in and changed the future and we can try changing it to like a hundred different things and see if they all still hit the bug. And if they do, it means the bug was already baked in. And then we can go back to the next one before that and do the same thing.

42:20 Yeah. Yeah. And we can sort of bisect backwards and then we can find the exact moment when the bug went from really unlikely to really likely. And then we can do things like look at which lines of code were running then, you know, look at, you know, look at all all you know what what log messages were being printed then. And often that is actually enough to root cause the bug too. Yeah. Are are you then like how are you reconstructing that for user space because are you drawing me a

Controlling execution paths via pseudo-randomness

42:49 picture of how the code went through the evaluation path it actually took? Yeah. So basically everything so we've built a special shim that makes it so that every attempt by any software whether it's Linux whether it's the user's own software whether it's um you know whether it's our software which is injecting faults and and changing the scheduling and the order of stuff um anytime any of those things try to get a random number, they go to this special random source that we control.

43:30 Okay, does that make sense? Yeah. And so, and so by modifying what answer we give there, we can really quickly change the trajectory of what the system does, like what operations are getting played against it, what order the threads are running in, whatever. Um and so and so that's a way for us to like very quickly explore different possible futures from some point. Um there are ways that you know that a user can shoot themselves in the foot. Like if you if you at your program startup you know ask for a hundred random numbers and then cash all of those you know and then look them up later to to decide what decisions to make. you will have effectively defeated our like intelligent state space search, right? And so and so we tell users not to do that.

44:21 Um but but assuming that you're just doing the the sensible thing and and going and asking dev u random for a random number before you decide what to do, we will have total control over the actions of your software and can go back and change them. Yeah. But the so I can see how you can get to that point now, but how do you reconstruct that into something that's useful to me trying to debug the bug? Because it seems like you've got this history that says, okay, the operating system started up and then the software started up and then I need to know that here's the path it went through my code and hit that point. How do you reconstruct that report?

44:59 Yes. Okay, great. Um, right. So there's basically two kinds of histories here. That's that's that's that's I think the key. One is the history of like what did we do, right? And that's totally meaningless to you. Um you know, and it's and it's and it's by design very very very small, right? It's like at instruction 37 trillion, I changed this random number. At instruction 42 trillion, I like made this thing pause for a second. Like it's like doesn't mean a thing, right? Then there's the history of what did your system actually do and that's like a story that is mostly told in your existing log files which you probably know how to interpret. And so along while we're testing your system, we are streaming out all of the logs exactly the way that you in production you would stream them to data dog or Splunk or something. We're streaming them all out and and storing them in this massive

46:01 database. And when we hit a bug down one of these particular paths, we can then stitch back together the logs of your

Storing timeline logs for interactive debugging

46:10 system to form a unified view, like a unified history of like what it did down that path until it got to the bug. Okay. Yeah, I that makes me wonder why are you bothering doing that? Why not just ignore all the logging until you find a bug and then replicate it and get the logging there? Wouldn't that be less storage space? Uh, yes. And we're about to add that feature.

46:35 Okay. Yeah. Yeah. The the answer is because we're lazy. Oh, okay. Yeah. Um, well, the answer Well, number one, because we're lazy. Number two, there is actually some benefit to doing it, which will be lost for users who enable this optimization that's coming. um which is when you store all the logs across all the many worlds both the ones that saw the bugs and the ones that didn't. You can then host hawk go and analyze the logs as if they were production logs and go look for trouble that you hadn't defined as a test property ahead of time. So this goes all the way.

47:18 This goes all the way back to the start of our conversation which was you know how do you how do you pick good test properties? So this is yet another way you can do it. Um suppose that you've got your antithesis test all set up and you know we've run with you for a while and found a bunch of bugs and now it's all green. Hooray you're happy. And then one day you hit some horrible crash in production or some some other horrible thing and and and you're angry. You're like, "Man, Will, why didn't antithesis find this problem?"

47:46 And I ask you, "Well, okay, what does it look like when the bug happens?" And you say, "Well, you know, it looks like my system returns this particular error code." Um, and I'm like, "Okay, did we have a test property that says that your system should never return that error code?" We go, "Look, and it doesn't have that test property, right?" And well, that's too bad. Um, you know, we can we can add that test property now and we will now be able to find that bug in the future, which is good. Um, but it's still a bit a bit sad, right? I would like to know I'm I'm probably in the middle of a production emergency or you're you're in the production emergency, right? You'd like to know if your tests have ever seen this problem before. Well, because we've stored all the logs in this enormous database, you can just go back and query and say, hey, if I had this test property, would I have had any failures?

48:39 Oh, and then we can say why actually yes. It turns out that on June 17th we saw that exact failure. Um, now we can give you a very, very detailed log of what led up to that failure without you having to run any new tests or do anything. But we

5 Highlight · Interactive hypervisor debugging and counterfactual queries

48:58 can also give you a live reproduction of that issue because that's one other feature I haven't told you about yet is we've got this crazy hypervisor, but you can also use it interactively, not just in test mode. So once we've found a particular situation, you can ask us to replay up to that situation and then we just give you like a bash terminal inside the hypervisor at that moment frozen.

49:25 Yeah. And so now you can go poke around your system and be like, okay, well, you know, what is this endpoint returning? Like what's in my database here? What does this file have in it? and just sort of pour through it and debug it without having to worry about, you know, debugging in production and all that that entails. Yeah. So, is that are you saying that's like stepping me back five minutes before the bug crashed or can I actually like say, okay, I'm 5 minutes before the bug crashed and now I would like to simulate this thing happening on the network and step through it.

49:59 You can do you can do any of these things. Um the the default is we deliver you there at the nancond that the crash happened and and the reaction time and then you can you know you can poke well it actually it's like it's not that accurate yet. it's actually to within like a a few microsconds, but but you know, you can you can then poke around or whatever. But yeah, a very natural thing to then say is like, okay, well, hold on. I hang on. I want to go back a few seconds before the bug and I want to go see what this thing was doing then and I want to take a core dump of this process before the bug and you know, maybe I want to go kill this process and see does the bug still happen or like I'll try and restart the database and does that fix the bug? Does that unstick us? Like you can sort of ask all these counterfactual questions and you can do it without the fear that by doing that you are going to lose the repro lose the bug because you can always hit a button and just be right back there.

50:53 Yeah. Yeah. Yeah. So you can maybe answer the age-old question of is it better to regularly restart servers or not which I've argued a lot in the past. Yes. Yes. Um so yeah that that's uh that that's why we store all the data even though we don't have to.

Taming massive search spaces with heuristics

51:10 Okay. Yeah. So so you're doing that on a kind of logging level or you're re recreating the operating system at that exact point or both. Uh right. So the input data we save is exact instructions for getting back to any situation. The output data we save is everything your program did you know leading up to some point. And the the data model for that ladder thing is is very very efficient because if you think about it, all of the histories and all of the timelines that we're exploring have a natural tree structure to them, right?

51:46 Yeah. Yeah. Yeah. And so we can dduplicate output that has the same common prefix, right? If it's like I tested your system for 10 seconds and it did a whole bunch of stuff and output a whole bunch of stuff and then we got to an if statement and then we tried both branches, right? And we, you know, saw what happened down both paths. We can just store everything leading up to that if statement one time. Yeah. Okay. I'm getting uh visions of uh git trees in my head.

52:13 Yeah. Ve very very similar in some way. Um, we actually, uh, we actually ended up just recently, we're currently in the process of migrating to a custom database that we wrote that's designed to just store data in this format because nothing we could find commercially, you know, really really did the right thing here. I'm sure someone would sell you a blockchain for that, but yeah probably. Okay. Um the one thing I'm really not clear on is you've still got a huge heristic space, right? How do you if if you're tracking every if statement in my code base to say, "Okay, here are the ones where we've done this a thousand times, so we need to keep an eye on that." Are you still not getting a massive amount of search space that you can't deal with?

53:01 It's still very big. Um there's there's a there's a lot th this is where sort of the the the the dark art of testing comes in, right? Like there's a lot that you can do to make this search even more efficient. Like everything I've discussed so far has just been almost on like abstract like first principles, knowing nothing about computer programs or how people write them, just using like math, right?

53:31 Yeah, but then on top of that, there's a whole bag of tricks. Like, you know, I I can give you some examples. Um, a really easy way to find bugs in software is to construct asymmetric partitions between nodes. If you make it so that node A can talk to node B, but node B cannot respond, like that, that frequently finds bugs. Just empirically, it does.

53:57 And so, I totally believe that. And so we do it a lot, right? Like you know, you monsters. Yeah. Like you know there there's you know there's just like there's a number of or or like what so a thing that we do a lot is um pause particular threads, right? And and let other threads race ahead and see if we can create data races or concurrency bugs that way, right? Makes sense. And you know and that's that's a very realistic thing to happen in production.

54:25 like anytime you have a GC pause in a Java server that's you know other servers are maybe doing things. Um so so that's that's a thing that we do but so then there's a question like how long do you pause it for and it turns out that like there's actually a pretty good answer to that question. you basically want to use like kind of a fat tailed power law distribution. And the reasons for that are like sort of a long story that we could go into, but but it it's it's true. And exactly how you fit the parameters to that distribution vary from program to program. But like since we know that that's a pretty good distribution to use and we know some parameters that work pretty well for a lot of programs um we can sort of start there with any given new program and that just immediately um immediately makes us find more bugs faster than if we just started with a completely uninformed uniform random distribution.

55:23 Right. Yeah. And so, you know, part of the thesis here also is that as we test more and more and more people's software, we're just going to learn more and more and more such things and it's going to make the search more efficient. But I think you're right to dig into this because this is I mean, in some sense, this is the part of the story that's most unbelievable, right? Like the like the thing we're talking about is like the Turing halting problem. like you you know that there's no perfect solution here, you know, like like so so so this is we know from you know from the beginning that this is going to be very hard and I think a bit of our a bit of our belief is that it's very hard. So that means that that somebody should get really really good at it or or multiple people should get really really good at it but not every single developer should worry about like I'm going to pause this thread. What's the optimal distribution of pause lengths that I should use?

56:21 Yeah. Yeah. But do do you find then that like it seems to me then you've got a risk that I come to you with my software and you find a bug and really that's a Java bug. It's like that feels a bit unfair. Yeah. Um we we have done that before. Um, you know, I think that in such situations, people are usually still happy to know um because that Java bug could happen in production and you might be very sad if it did and perhaps my guess.

56:55 Yeah, it's still that's right. Perhaps you don't want to use that Java feature. Um, look, I will say another thing is a thing that we try to emphasize throughout our product is that we're not judging you like you know like like like everybody's got different quality goals and everybody's got different amounts of time for this stuff on their road map and everybody's got different you know and some people really do want to find every single last bug in their program because they're designing a pacemaker or a airplane or something.

57:26 Yeah. And then other people are, you know, they're a startup and they're making a pet food website and they're like, you know, it's okay if it's down occasionally. And and we want to support both kinds of developers, right? And so we we do try to offer good facilities for for you to say, "Thank you for finding that bug. I do not intend to fix this. Um, please please don't tell me about it anymore." Um, and you know, and and we will, you know, and we'll and we'll we'll let you know if we, you know, there's there's still things that could be interesting about that bug, even if you tell us that you're not going to fix it. For example, if we suddenly stop finding it, that might be interesting to you. Maybe it means you fixed it by accident or maybe it means your test got weaker somehow and we can't find it anymore.

58:11 Um, or the JVM finally fixed that bug and we happen to upgrade, right? Yeah, that's that's exactly right. And and so, you know, I I I'm very sensitive to the fact that um you know, we could easily by accident create something where every time you run our platform, it feels like a homework assignment. It's like, oh no, what have they what have they what have they done now? Like, you know, and we and we really don't want to feel like that. We want to feel like your cybernetic best friend, which is helping you do your job better. And whether your goal is to eliminate every single last bug, or your goal is to just only find the very very worst bugs and the rest of them whatever, or your goal is just to know about them so that when you hit one in production, you can then reproduce it and fix it really fast.

59:04 Like all of these are valid valid goals and we want to support you in each of them. Okay. Yeah. Yeah. There's a natural question that follows on from that, but I have to ask the cheeky one first. Do you find that there are certain languages that are better or worse at things? Like, does does Go always cause garbage collector bugs? Does Rust always cause asynchronous bugs?

Language ecosystems and bug detection tooling

59:28 Oh, I see what you're saying. Uh ah man, no comment. Do you want to go on record with answering this? Um yeah I I will say you know like I I don't know the thing I think the things that we have discovered are the things that everybody kind of already knows. For example very large Java projects have vast numbers of uncaught runtime exceptions. Like I I I assume you knew that from your past experiences and and I did too. But I do think I now have statistical confirmation of that. like you know they they they all do. Um you know Go is an interesting one. We do find lots and lots of concurrency bugs in Go, but it's not because Go programmers are bad. It's because Go has excellent tooling for finding concurrency bugs. So it's really easy in Go to compile your binary with T-San um the threader. Oh yeah,

1:00:28 it's a it's one of the um it's an LLVM uh sanitizer which basically detects data races in your program dynamically at runtime when they happen. Nice. You have to provoke the race, but if you provoke the race, you get a little print out that's like, hey, your program didn't crash this time, but there is a race here. These two threads access the same variable without any kind of mutex or whatever.

1:00:54 Oh, okay. Yeah. And and so it's actually tremendously helpful like you know we we have this like really good test property that we can use with Go programs as a result. Um you know so so that that that's cool. Um you know C programs do have lots of lots of memory safety issues. I you know yeah I can believe that. I absolutely can.

1:01:21 Uh yeah. So, so it's not clear if it you're not going to say certain languages are better or worse at some things because you might be more likely to detect it in some places. That's right. I will say certain languages have made it easy to find certain kinds of bugs, which I appreciate. I think that's a good thing. Um, and you know, when it comes to the language wars, I'm really just not a partisan. I've I feel like I've or maybe I just, you know, my my view is that they're all bad. Like it's it's just it's just you know what what's the right tool for this particular job.

1:01:54 Yeah. From a certain point of view, they're all lovably terrible. That's right. And they're all made by human beings who are trying their best. Yeah. Yeah. Absolutely. Okay. So, it's a good answer. It's not the controversial answer I need for clickbait, but it's a good answer. Um so, the question, the naturally following question I was going to ask you is who actually does use this? I mean what are people using it for? Do you get like operating system vendors contacting you?

1:02:24 So you can go on our website and see a long list of our customers. Um I would say that the initial success

Real-world adoption across data infra and gaming

1:02:31 we've stumbled into has been with uh three main categories who we've sort of surprisingly gotten a ton of. Um, one of them is any kind of database or data infrastructure or data streaming thing or you know distributed computational framework or whatever. You know, Confluent is a big user, MongoDB is a big user. Um, you know, various Kafka reimplementations are all using this to test their stuff. Um, there's a lot of that stuff. Um, and we've actually got a fair number of open source projects there too. We recently started testing CD um working together with the Linux Foundation on that one. That's been good. Um so a lot of that kind of stuff.

1:03:17 Then I'd say the second big category is uh blockchains, cryptocurrencies, which doesn't surprise me. Does not Yeah. Like it surprised me initially and then once I thought about it, I was like, "Oh, that makes sense, right? They're big distributed systems with billions of dollars at stake. You know, they probably they probably need some testing and not always the most rigorous rocket science implemented projects in the world. Dare I say I I will say it's very biodal. Um I' I've Oh yeah, I've seen a few that I've been really really impressed with. um really like they really take the engineering seriously and then I've seen a few that are a little a little bit flying by the seat of their pants. Um and you know but but they they want to do better which is why they're talking to us which is great.

1:04:11 Um and then you know the third big category is like sort of adjacent to that one. It's like fintech. You know we've gotten a bunch of fintech customers. Yeah. Yeah. You know where the cost of a bug can be catastrophic. That's exactly right. Um, the one that surprised me the most that we haven't gotten yet is is gaming companies. I figured that the game industry would be all over this because, you know, bugs for them are very costly as well and they spend so much money on QA right now and and it's relatively primitive.

1:04:42 Um, but for whatever reason, we do not yet have, you know, Valve or Epic or anybody like that as a customer. So, if any of them are listening, give me a call. Could you do that? Could you support that though? cuz would you then need to perhaps you already do start simulating a GPU? Right. So I was initially thinking that we would just test their engine code and especially test their net code which is very tricky stuff and uh and and quite hard to get right. We could also actually test the game logic. And as I mentioned like if you go to our website you'll see many many demos of using this to test Nintendo games. Um, and you know, I think like a complicated interactive computer game is one of the most complex software projects that that people make, you know, on a regular basis.

1:05:33 Yeah. Sort of exercises every part of computing at some point, right? Yeah, that's exactly right. Um, and I think you could get around the GPU thing. Look, I think we actually do want to start emulating GPUs, but in the meantime, you could get around it by just giving us a headless build of your game. Um, we then wouldn't be testing the rendering engine, but that's that's probably okay, you know. Okay. Um, so there's a question I really want to ask you. Let me just check I understand something. When you're testing Mario,

6 Highlight · Using Nintendo emulators to test hypervisor search algorithms

1:06:00 you are not emulating an old 8-bit Nintendo system. No, we are. You're Are you? Yeah. Ah, so you you've done a hypervisor for the NEZ. Yeah. Well, the good news is other people have already done that. So, you know, so, so the history here is actually kind of fun. Um, we were working on the hypervisor, but it wasn't done yet. And we wanted to parallelize our efforts. And so, we also wanted to start working on the bit of our platform that actually searches for the bugs, connects to the hypervisor, says, "No, no, go back to that point, change that random number, no, do this differently.

1:06:37 No, go around that loop one more." Right? Like, that's a very complicated component as well. And we wanted to test that, but we didn't have the hypervisor ready yet. And so we were like, well, is there anything else in the world which is deterministic and we can take cheap snapshots of all the memory and we can replay things perfectly and there's a large set of interesting programs that we can test it on. And then we were like, "Yes, it's called a Nintendo emulator." You know, and like the Nintendo emulators are by and large deterministic because the guys making TAS's, tool assisted speedruns, are making use of that property.

1:07:20 And then they have very cheap snapshotting because the Nintendo only has like 512 kilobytes of memory or something. And so you can just snapshot the whole thing. And and go back in a time machine to the 80s and say in the future people will mock you for how small your memory is. Yeah. Well, and and and and there's a giant diverse diverse playground of very different computer games which actually are incredibly challenging to test thoroughly and require different techniques to get into. And so this was just this unbelievable seam of like rich rich research problems, inspiration for different testing techniques. Um we we honestly learned an important lesson from every single new game we tried. Um really?

1:08:10 Yeah. Yeah. Yeah. I And I just assumed this was a very good marketing tactic, but actually it's serious engineering. Yeah. It started it started as engineering and then we were like, "Wait a minute, this is great marketing." And then so then we and so then we made the blog posts. But yeah, you can you can go and we we did, you know, we did a really my colleague Harrison did a really good post about um Zelda, which was one of

1:08:31 the most complex games that we beat and we found lots of bugs in that one. Um you know, I'm actually about to finish a post on Metroid that's going to go up in a week or two. Probably be up by the time this this airs. Um yeah, no, we there's some fun stuff. Okay, I'm going to link to those in the show notes. The reason I ask is I have to wonder. I'm thinking of like software that needs to be super reliable. Maybe this has happened. If NASA approached you and said, "We need someone to simulate the Mars rovers hardware." Could you do it? Would you do it?

1:09:05 Yeah. So, um, there's a guy named Toby Bell who gave a really good talk at Strange Loop, I think the last year that it that that conference ran, um, precisely about applying deterministic simulation techniques to um to a really advanced mission at JPL where they were trying to basically they were trying to build like a inorbit I forget if it was an interferometer or like some kind of like pretty complex camera. But the way it worked is like you had two satellites that had to like perfectly synchronize their movements. And they wanted to test in advance that the flight control software would work correctly in every imaginable situation. And obviously these things like need to communicate over a network with highly variable delays because the satellites are different distances from each other, different points.

1:09:59 And so they used very similar techniques actually to to test it. Um And we we have had uh a few emails with the guys at NASA who are designing the proposed new interplanetary internet. So they've got this project to develop a new networking protocol for highly delay tolerant networking. Um Oh, when you're sending messages to Mars.

1:10:25 Yeah, exactly. And and and that stuff needs better testing. And so we we've been talking to them and you know I would I'm a actually a big space fan so I would love to work with them. That is mind-blowing. I I was on a plane a few days ago and I was watching re-watching The Martian which is a terrific film and that's totally making me think. Yeah. Yeah. Yeah. Yeah. I'm going to have to go and watch that talk. I think I think we have to stop there because you've got me hungry for listening to NASA people talking about space.

1:10:55 Yeah. This this was really fun. You should totally check out that talk though. I totally will. And then I'll probably go and have a a Nintendo game nostalgia festival. So, thank you for giving me an evening. Yeah. Thank you. Thank you so much for having me. Well, that was brilliant. Cheers. Awesome. Have a good one. Thank you, Will. Briefly, while we're talking about The Martian, I'm going to have to recommend a different book by Andy Weir, Project Hail Mary. I really enjoyed it. Apparently, there's a film version coming out next year with Ryan Gosling. I hope they do it justice. So, if you want to wait for the film, other options for good things to read can be found in the show notes. Uh, Antithesis have a few posts about Nintendo games, so I've linked to those. I'm sure you can find the rest. I'm also sure you can find the like button. If you've liked this episode, please do take a moment to click it. If you're not already subscribed, please make sure you're subscribed for future episodes cuz we've got some interesting people lined up.

1:11:53 And with that, I'm going to say goodbye. I've been your host, Chris Jenkins. This has been Developer Voices with Will Wilson. Thanks for listening.