Inside Anthropic: How Claude Tag Is Changing Agentic Work

12,080 words · about 48 min to read

Claude Tag automates 65 percent of internal PRs

0:00 Claude Tag is your proactive teammate. what's really special about Claude Tag is it has all of the kind of connectors and tools and contexts that you're used to when you're using Claude Code, it's got this extra degree of proactivity and the persistence to see pieces of work through for longer periods of time. since we've started using Claude Tag internally, our product teams, have 65% of their PRs opened by Claude Tag. I think there comes a point where it feels like this product is really leaning into like, where we see model capabilities today, and it's the one that's really our daily driver at this point.

0:31 The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding. Join your hosts, Guy Podjarny and me, Simon Maple. Every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today. This is the AI Native Dev. We just wrapped up two amazing days at AI Native DevCon in London.

0:58 But the great thing is that we get to do it all over again in New York City this November. You're absolutely right. We're going to be back in the City that never sleeps on November 3rd and fourth for more amazing sessions, really engaging hands on workshops and much more. Yeah, all that great networking, partying, eating and drinking that you've come to expect from AI Native DevCon. We think we have one of the best hallway tracks from the business, and it's the perfect complement to our incredible speakers and presenters.

1:26 We'll both be in person and virtual, with live streamed access to all mainstage keynotes and talks. Sign up right now for our Super Blind Bird ticket for just $100. Only available for a limited time. We're really excited to be headed back to the Big Apple, and we hope to see you all that. Hello and welcome to another episode of the AI Native Dev. And it's wonderful and my pleasure to introduce a member of technical staff from Anthropic.

1:57 Welcome to the podcast. Hey Simon. So happy to be here today. Amazing. We're gonna have a great conversation about a gent coding in general, how Anthropic use Claude code and development in your environments, And also a massive congratulations for the very recent announcement and release of Claude Tag, which we'll go into in just a second. But tell us first a little bit about yourself, your role in Anthropic and what you do. Yeah, absolutely.

2:24 So I'm I'm a member of technical staff Anthropic and I sit in our applied AI team. So this is a team which sits between product research and go to market. And so we do a mixture of working directly with customers. I tend to work with startups and founders, and we also work on some internal projects with the product and research departments. Amazing. And we've got a lot to talk about today. But we do. Let's start at the very exciting news. Very recently, Claude Tag was announced.

2:52 You've obviously been using this internally for, for for many weeks now.

Proactivity and memory set Claude Tag apart

2:56 Tell us a little bit about Claude Tag. What is it and how do you use it. Yeah, absolutely. So Claude Tag is your proactive teammate. which you which means you write where you work in slack for now. And what's really special about about Claude Tag is it has all of the kind of connectors and tools and contexts that you're used to when you're using Claude Code, but it's got this extra degree of proactivity and the persistence to see pieces of work through for longer periods of time.

3:21 And like we said, it is able to collaborate with you and your teammates at the same time. Yeah, and it's amazing. And even internal internally in Tessl we use Claude code very, very heavily. And one of the things that we do, which sounds a little bit similar. So I'd love to kind of ask how it differs, is we have an integration of Claude and Claude code within slack. So we very often tag Claude to do certain things for us that that's been available for a little while.

3:50 What's the difference between that tag and Claude there and Claude tag? Yeah. So I think this is a great question because I think on the surface these things look really similar, and it's actually some of the more intricacies

1 Highlight · What differentiates Claude Tag from basic Slack bots

4:00 underneath that differentiate these two products a bit. But I would say, first of all, Claude Claude Tag is incredibly proactive. So it can kind of go and execute pieces of work for long periods of time and then come back and let you know when that thing is done. So it's really leveraging the ability of coding or agents these days to do tasks for very long periods of time. So that's one thing. And The second is like often you would kind of go and trigger your, your agents by asking them questions and kind of Claude in Slack would reply, Claude Tag will actually like sometimes come and find you and tell you that something need your attention.

4:33 So again, there's this element of proactivity. It also has memory and context which can kind of span your channels. So it can kind of understand what one whole team is working on. And because it's chatting to all of your teammates at the same time, that's something that builds up over time. So rather than you just kind of having these one off interactions, Claude Tag is able to kind of move out of the scope of kind of sessions and individuals interacting. With people. So it can almost like initiate the conversation. It absolutely can. Yeah.

4:59 Yeah. Amazing. So the way we tend to use the Claude invocation

5:04 in or tag within, I shouldn't call it tags. The actually. Yeah. Claude in slack. So the way we tend to use it is very often we'll start discussing a need or a feature or a bug, maybe in slack, and then we'll maybe at linear and say we need a, we need a ticket about this. And then we'll pretty soon after that just, you know, hit hit up Claude and say, can you just implement this, please? And it will return pretty quickly and say, okay, I'll use the context from this thread.

5:31 It obviously knows the code base as well. And then it will provide something like a pull request and say, can you have a look at this and see? And then we'll say, yeah, absolutely. This looks good. Merge this and we're good. So there's a couple of changes there. Then the first one is obviously the history there. It sounds like the history it will know about past interactions across the slack or beyond. Was that one of the first ones that you mentioned? Yeah, absolutely. So I think on that context point, the so we kind of we have this whole positioning system with Claude Tag, and you can really control that and tune it to how you want it.

6:03 But essentially it's kind of scope to a channel level. So if this is a channel where you're always talking about feature requests, it has a ton of context on historical feature requests and how like what the process is, for example, through skills for seeing those processes to end, to end like so there's a lot of history and context if configured as well. It can also have the ability to search other public channels at. So maybe you have like a customer support channel where there's a bit more context on where this issue arose.

6:28 And Claude Tag, given the permissions, could also access that information and surface it where maybe you might not even have that kind of visibility. So that's one thing, is just this like this context window or the memory scope that it has is larger. I think a second is you mentioned that. Do you go to Claude Code to implement the tickets afterwards after their daily linear? Yeah, I think that's typically the way we do. Yeah. So I think one one thing to surface here is you could achieve that whole end to end workflow in just one thread in slack.

6:55 So let's imagine in this case a piece of feedback comes through in the Claude Tag world. You could have it configured so that Claude Tag actually picks up the feedback first. And tags everybody they think should be responsible, like should be involved in the response to that. And as part of its skills and workflows, it could also then automatically raise the linear ticket because that's the workflow that you have configured and that's available just to kind of prompting in it, learning over time through doing that workflow.

7:21 And then the final thing is, which I think is the real difference is you no longer need to open Claude code to go and ask it to execute that. So again, it could just understand that the workflow it needs to do is kind of raise its start coding so it can spin up its own sandbox. again. Start running the code in your repository if you've got access to it, and then it can ping you when the PR is ready. It can also do things like verify the aspects of that meet the original requirements from whatever ticket was raised in the first place.

7:48 So I think what we see here overall is more proactivity end to end. If you do want it to be able to run that process from kind of ticket to probably asking you if it if you want it to execute the code first, if you want a human gate there and then kind of seeing that PR through prompting people to review it, etc., etc., it's much more proactive in that sense, and it has access to all the context in memory to do that job. Well, And it sounds as well that it's becoming more of the orchestrator as well. So whereas I would need to normally hit up linear and say raise a ticket for this, sounds like if I can just pull a Claude Tag in, it can actually do that behind the scenes.

8:22 And I'm more thinking about the functionality or the feature that I want to change versus the mechanism of the workflow of, okay, let's raise a ticket first. Now let's pull Claude in at that. At that stage. Yeah, absolutely. So I think again, it's really for individual teams. And to think about the degree to which like where it is in that workflow that they want to steer the agent. So you can be very clear about like these are the absolute gates where either you absolutely do not have permissions to do these things, or I always want you to ask for my input on these, these areas.

8:53 And I think what we've seen is because Claude Tag is able to build up this memory over time, it gets a really good feel for what those workflows look like, and it adjusts. So that's really cool. And I think just to throw in one last difference on that new workflow, what you'll see if you kind of went and implemented this with Claude Tag is just the ability to enable this cross-functional workflow is really cool. So for example, like you have the customer support ticket raised, you might kind of tag some of the account executives that are on that account just to see if they like for their visibility so that they know what's going on.

9:27 You can then kind of tag in loop in your engineers or products people for feedback on some of the product implementation. Maybe if tag comes back to you with a plan and then have your engineers looped in later when you need that code review to go through and the whole deployment to work. So I think what we see is this like multiplayer ability, which has been quite difficult to achieve so far with maybe just working through Claude Code or Co-op or something like that. So I think this is where we see some of those effects really multiplying.

9:52 Amazing. And there are going to be a lot of listeners who maybe, don't use

Chat environments emerge as collaborative developer hubs

9:57 a Claude or Claude tag in their in their slack environments today. So it's actually a pretty a pretty major change. So when we think about the process change, what are we going from and to. So let's assume people are agentic developers today. How does it change the flow whereby we're essentially living more in our chat environments, more in our slack versus potentially our more flows or ID?

10:24 How does that how does that workflow change? Yeah. So I think there's a couple of things. And just to just to frame it in terms of the impact that we've seen internally since we've started using Claude Tag internally, our product teams, like product engineering teams, have 65% of their peers are opened by Claude Tag. So that's kind of the scale of which like we obviously were there adopt Claude code. And now that's like often the the port of call for how we would kick off some of these workflows and how those things originate.

10:54 So yeah, that's how it frame it in terms of like where we're at today in terms of workflow. And yeah, I think this chat interface is definitely different. I think what it allows, what it allows us to achieve is when you're using Claude code. So let's say you're doing agentic development. You have these this concept of it's quite single player. So you're often kind of moving into your Claude code instance. You're potentially running code locally and you're kicking off individual sessions like we do have this parameter, which is a concept of a session.

11:26 And that's where you're kind of stating what you're trying to achieve, managing the context for that. And then you have some kind of like outcomes or goals or loops that you're running. What we have with Claude Tag is it all becomes a little bit more amorphous. So we're starting to like be less session focused and it's less single player focused. So when you're kicking off these agentic coding or just agentic processes with Claude Tag, what you're seeing is that it's so much more multiplayer from the get go.

11:51 So tag can like get the opinions from people that might be needed for, for a workflow. So rather than me as the developer having to go and ping like some product folks or some sales folks on something, I can get that embedded in the core tag flow. Also. Ultimately, all of this work is potentially happening in public, which means that those people have visibility on those workflows from the get go. They know exactly what kind of product specs the engineer is giving, and can always chip in if they want to, with extra context, etc.

12:18 and maybe their own Claude tags are going to like ping these things over to them. So there's this whole like collaborative nature which is which is really different. The other thing is it's a lot more asynchronous than what you'd achieve with Claude code. So I'd say that with Claude code, you're kind of it's great

2 Highlight · Transitioning to asynchronous, long-running agent tasks

12:34 for when you kind of want to be there for every single turn of the agent, like you want it to come back to you with exactly what it did, anything that needs follow up, updates, etc. and when you really want to steer that thing more closely, that's a great thing to do. With Claude Tag, we're seeing more of a pattern of these kind of long running asynchronous tasks, and that's really leaning into some of the model capabilities that have been unlocked with kind of recent iterations of models. And so what you see is like you kind of ping Claude Tag and it comes back to you maybe a couple of hours later having like built end to end, this feature that you talked about.

13:07 And so you're kind of able to just leave that agent to itself and it will come back to you when something's done, which is quite different, I think, to Claude Code, I mean, it's inspired by some features that we've released and called code,

13:16 but that would be how I think things have changed. And the way the industry is evolving. It's funny actually, about maybe a year ago or so, I was having a chat with one of the of product, I believe, at slack, and I kind of I'm used like is, is slack becoming the new idea of a developer. And it's kind of very interesting when we think about the evolution, you know, developers obviously loving their Ides, it becoming the most productive place that they're in.

13:43 Then all of a sudden when we started using, you know, the copilots of the world, where we started introducing a generic assistance, I would say, into the IDE, that gradually became more and more evolved with multi file changes and, you know, things like Cursor coming in and those types of things. But then as soon as Claude code really hit the terminal IDE, it changed people's way of working. And I see this as like almost the next shift where it really requires trust.

14:13 And I'd love to talk about trust for a little while. It really requires trust to actually step away from the IDE because you're not focused on code, which I think will be the future space whereby actually we get to that level of trust where we don't even need to look at the code. But today, I think there's this big shift of as soon as we're in the terminal IDE, we are we're actually relying much, much more on the tests and the validations versus looking at the code or the results from a code review.

14:42 As soon as that then gets extended beyond and into into slack or a chat environment, we're almost like abstracting away from the code one stage further, I think this kind of a tool, like a Claude Tag or anything from within slack, this wouldn't have been accepted a year ago, right? But we're so much more trusting of AI doing the right thing. And actually the results of AI generated code

Agent time horizons double every four months

15:08 and a code being reliable that we're able to use, agents from, from a space which is further obstructed away from a very code centric environment. Is slack and chat the new idea? This is a this is a great this is a great question. I guess it's hard to I don't think these are like like for like replacements. Like I think each has their, their place in their purpose.

15:33 But really resonate with this trend that you mentioned in terms of how our behavior, the way that we interact with agents is changing for a number of reasons. So I think one is that to this trust point, like one is that models are just getting better and better, like we are squarely on this exponential trend in terms of how long agents can run for to.

3 Highlight · Agent execution horizons double every four months

15:52 The METR chart always shows us that roughly every four months, the amount of time that agents are able to run for autonomously is like doubling. And so this is from a sheer capability perspective. We're allowed like we're able to trust these agents more because of the actual capability that's improving. I'd love to just double down on that way. You just said every four months the the, the time that an agent can do something more autonomously is doubling in terms of and is that the capability of that model, or is that almost because there's a human aspect to that as well?

16:23 Right. In terms of if it goes off and does something. Am I able to interact with it? Yeah. Is it pure capability of the model and the agent. That's that's so yeah. So this is this is an interesting one. And it's METR who produced this research and it covers a couple of different domains and essentially time horizon. So that's how long agents are able to kind of successfully complete a task of a certain length is a measure of capability. It's not perfect like there's other evals and benchmarks that we use to measure model capability, but it's one generally broad one that has really mirrored, like what it feels like to interact with these models.

16:58 So the longer that they're able to complete these tasks, often is mirroring the fact that they're doing more complicated things, like they're able to do these multi-step tasks, are able to verify the results of their work at multiple stages and come back and successfully complete these tasks. So this is one kind of like industry broad trend that we see. Yeah. And it's I mean, if you look at it, it's just like it's quite shocking

17:19 because every time you think that you're not going to keep up with that, that nice log chart on a straight line like you just do every time. And that's been happening for the best part of a decade. So this is this is really interesting. And I think when we talk about model capability, obviously, you know, the amount of time it runs for is one thing. But what we see over time is like a lot of behaviors, which previously we'd kind of put in the harness are getting embedded into the models. So to make that concrete models are a lot better at verifying their own work these days.

17:49 So both from like an instinctual perspective, like they do just check over their own work before coming back to you and saying it's complete, and where they have the tools to verify their own work, whether that's through like front end test or running tests or creating their own evals, they're getting a lot better at doing that as well. And so to your trust point, I think these things go hand in hand, like you give the models the tools to verify their own work, and they are better at kind of doing that themselves. And I think another pattern that we see in development today is that the behavior of the developer or the engineer is shifting much more towards how well can you define what success looks like in your case.

18:24 And that's not just for coding. That's really for everything. Like, do you have a really good sense of what success looks like, and if so, hand it to the model and it can loop over or have another agent review its work until they both kind of until the reviewer kind of believes that that thing is complete. So I think these are the two things. It's like on the on the agent and model side, they're just getting more capable and better at doing these things. And then on the human side, like we are getting, our behavior is shifting more to just like, can we define what good looks like?

18:54 And I think another thing to just like round that off is that having really like for a lot of us, having used these tools in anger for like the last year, we're getting a really good sense of like, where are these tools really useful? Where is it that they need a bit more supervision and we can kind of tune our behaviors and inputs more to like what makes those things work really well in tandem. So to finally, to your point on like it's like the new home or it's like the new IDE. I think it's a surface that allows your agents to be closer to where you're doing your work all the time anyway.

19:25 It allows you to tag them in in places like anywhere really. Like anytime you need more context or you want to delegate something, you're able to just loop the agent in. And sometimes that is a coding task. Sometimes it's like build this feature or this dashboard. Sometimes it's just like, what does this acronym stand for? Like, can you please just tell me what happened last week or something like that? So I think it just allows a bit more flexibility. You're not kind of switching context between working with my agents and working with my team. It's all a bit more integrated.

19:51 And it's it's really interesting when when you kind of like mentioned of defining what good looks like. Yeah. When we as humans, we know we always want to use the best tools, the best place for the best task at hand. And in the case of defining what good looks like when we focus on code. Yes, absolutely. We want to describe tests. We want to write some test cases and we want to we want to build them out. And a lot of the time we we lean into the IDE to actually build that out.

20:19 But when we want to collaboratively define what good looks like, we are not, we naturally find ourselves in a chat environment and it is the right place to define it. And so tagging something in at that stage,

Industry adoption unlocks massive codebase rewrites

20:32 giving it the context of what of what good looks like is super important. Let's, let's step a little bit back and look more broadly, because I think people who are the power users will look at something like Claude Tag and think, this is exactly what I need. I want to bring this in immediately. So here actually in the Tessl office today, we have a hackathon kicking off, and there's like 100, 150 folks out there, very, very broad sets of existing adoption.

20:58 Some people are more new to coding. Others have been using it obviously for for a for a long time. I'd love to ask you about the external adoption of Claude code from obviously the wider the broader industry from a purely a coding and development point of view, at what stage of maturity would you say is the industry at in terms of using Claude code? Obviously we'll talk about Anthropic adoption in the second, but externally, how are people mostly using Claude code today?

21:32 Absolutely. And I think we see this in all sorts of like shapes and flavors and forms from, being able to like for individual pieces of work, being able to complete that to like a higher degree of quality faster. That's something we see on the kind of individual level. And when we scale this to teams and across organizations, we've seen some pretty phenomenal things, like we've seen organizations, for example, like Stripe, do entire code base rewrites that would have taken weeks or months in like days or hours.

22:02 So this is the kind of scale we're talking about when we when we really deploy these things at scale and everyone's kind of on board these kind of ambitious projects that maybe you just, like, would always put off for like months or years because it's just where do you find the resource finally doable? And that's allowing people to focus on like other parts of like product and engineering work. And yeah, we really see this consistently like other teams, like shipping millions of lines, like a million line of lines of code in just a month or something like that.

22:28 So I think when everyone kind of like really leans in, that's the scale at which we're able to see development happening. And I think the other thing is you start to see more teams pursuing these kinds of moonshot projects that they just like, wouldn't have time, resources or capacity to otherwise. Like on the product side, something that we see is teams like basically prototyping a couple of different ideas or approaches to something, testing a bunch of them, either like with a tools or internally, and then just like going all in on the one that they see work best.

22:57 So there's a bit more room for like experimentation and I guess a bit more like bravery and boldness in the way that you're approaching product development. And of course the, the, the build versus by question then becomes super interesting because it, you know, tools like Claude will make it so much cheaper to build rapidly. You can take an idea or a prototype very, very quickly to, to an actual live working application.

23:24 And I guess the question then is if teams want to continue maintaining that over time. And I guess, you know, when we think about what's changing quickly in this industry, is it humans and people that are slowest here in terms of the adapting? Are people able to keep up with the with the rapid change and delivery that's happening in the AI coding space?

23:51 Yeah, it's a great question. I've been I've been on a bit of a tour around a couple of European cities recently, talking to some like founder communities. And one question I love to ask everyone in the room is like, who has FOMO that they're not using AI, like they're not using they're not AI, build enough in their day to day work in their life, etc. and it's always just like a full room of hands. And the Anthropic employees as well. We all have our hands up. Like who can keep up with the pace of this? Karpathy, famously in his tweet as well, never felt more out of touch.

24:18 And if Karpathy saying. That I just. Like ultimately the the the scale of things, the speed of things is like more than like one human like mind or person can keep up with like, I mean, it's more than a full time job at this point. I think, sometimes I even discover features I didn't know that we had. Because who can keep up with that? Yeah. But I think that there's, like, we've talked a lot about this, the speed and the pace at which these things are developing.

24:43 And there's a there's another story as well. Right. Which is like, well, how much is this translating into actual impact for people like as product developers or builders, like how are you mirroring that exponential and the value that you're able to deliver to your customers and in the, in the like impact on processes internally? And I think this is a much harder question, and this is a much harder problem to solve. Like we can we can have all this raw intelligence, but do we have the infrastructure to kind of bring that value to life.

25:10 And there's so much that that covers like it covers your harnesses and how you're managing. Maybe you're like context and your memories, how you're managing your tools, how you're giving access, giving these agents access to everything they need. And also in a secure way, like how are you permissions that it's something we thought a lot about when we were designing Claude tag that specific problem. And then there's there's everything else like there's all of the infrastructure. Like how are you going to host and deploy these models and how are you going to deal with with all of the inference essentially.

25:39 So I think that, yes, while it's amazing that there's lots of like shiny, amazing things happening, like those are the problems that like, everyone should be really laser focused on. And we've, you know, we always try and like develop tools to help people like really access that value because on paper it's there, we can see it. But in order to make sure that people really feel that, I think that's like a lot more hard work. And I think it's something that a lot of developers experience. Yeah, very, very interesting. And you mentioned memories, and I also love to chat a little bit about dreaming as well, which is something that you talked about AI Native DevCon.

26:10 Yeah. Our, our conference in London here, and I'd love to talk about that

Internal dogfooding drove Claude Code adoption

26:14 in just a second. But before we do, I talked a little bit about external communities and industry I'd love to talk about internally at Anthropic Now, and how Anthropic develops software itself. And then I'd love to talk a little bit about much. It uses Claude code generally in the org as well as coding. So so first of all.

4 Highlight · The humble origins and rapid adoption of Claude Code

26:35 Why don't we go back to day zero? Boris is playing around in his basement playing, you know, building this thing called Claude Code. Take us through that story. Yeah. So this is like, very much, like a side project that Boris was working on. And I think there's, first of all, like, culturally Anthropic. There's this real, there's this real experimental culture. People are always building their own tooling and experimenting with things.

27:03 And this was something that Boris was working on. And the funny thing is that originally, I think when he shared this in a slack post, it got like six reactions. Which always tells you that, like, you just like data isn't perfect. You can't there's not like perfect processes for understanding, like what, what good products look like, etc. but a couple of people like, saw this. I'm really excited by it and continue to work on it. And over a short period of time we saw like amazing adoption within the company.

27:30 Like half of the company using this like weekly. Another thing that's important here, and I think it's a really key principle when when you're developing products in this space, is that we saw a real take off in the adoption of this product. Once the models improved a little bit more to make it like really achievable, to kind of work on these coding tasks for long periods of time. So like early iterations felt a lot less agentic. They, you know, it just was more similar to kind of just like getting like chunks of code back through, whereas later it really could start to like access different kinds of tools, work really efficient over the code base and like stay on top of a lot of context and stay goal oriented.

28:10 So one of the big things that we always say to people in their developing is like, build for where the models are going to be in the future, don't build for where they are today because as we've said, these things move so quickly. Super interesting.

28:20 There's a couple of things I want to unpack here, of course. So let's jump into the dog feeding first of all. So I think this was this is a real like dog feeding success story within Anthropic. Right. It was it was it was documented how popular Claude code was internally and how it kind of how it was realized all of a sudden that actually, this is a this is a huge thing that if our engineers are using this so, so broadly and getting so much value out of this, this is something we absolutely need to productize, talk about talk about how when Anthropic knew this was super valuable and, you know, wanting to share this with the with the broader audience.

28:59 Yeah. So I think it comes back to this idea that I think like I think like half I think it's like half of the team at that time. We're just like using Claude Code every week, which is quite crazy, like for a, for a new product. I think that sense of internal PMF really made us realize that it was time to release his product more broadly. And it's really the same story with Claude Tag. Like. Like I said Prior to the release, 65% of our PRs are being raised by Claude Tag, and I think there comes a point where it feels like this product is really leaning into like, like where we see model capabilities today, and it's the one that's really our daily driver at this point.

29:34 And I think one of the big patterns there is like, what happened with Claude code is at first it was all of the engineers who were relying on this to ship a ton of code really quickly. And then we saw this pattern where all of the teams at Anthropic were like, totally going all in on Claude code. Like, there's this crazy story of someone on the marketing team whose day started with, like, googling what the terminal was and how to use it. And by the end of the day, they'd like automated one of their workflows, which took them 30 minutes, and now it took 30s like they were able to produce these, like, ads in that short period of time.

30:05 And so I think what we've seen is, like, everyone is seeing the power of these tools and is very is able to kind of creatively find ways to map that to their workflows in different, different domains that are not coding. Maybe that kind of problem of verification and context is a bit harder. Like you're not set up so well with like neat file systems, GitHub to kind of manage your version control and like the ability to kind of unit test things, like you have to be a bit more creative.

30:32 But I think that that's like where we're all headed, right? Like we're able to kind of set out like outcomes and success criteria or rubrics for like what a good document looks like or what a good briefing looks like. People are, you know, changing their behavior around how they kind of create and produce data and where they store it, so that agents are able to more easily access that. So, like, we have this huge culture Anthropic where we work really, really publicly in slack.

30:57 And so we do that on purpose because it means that our agents can kind of connect the dots in ways that no person could, like, possibly ever have the visibility over. So sometimes, like I was working on, I like to the extent where my I talked to Claude Tag in a public channel, like all of my work with Claude Tag, unless it's something that's really private, I do in a public channel. And I mean, sometimes people on my people, like in my company who I've never met or messaged me being like, I saw you were working on this thing. Like, I'd love to use it.

31:23 Like, please, can you tell me, like, is that shareable? And like, can we collaborate on this kind of thing? So I think just the ability to kind of connect an organization at that scale is only possible because of the kinds of tools that we have. Super interesting. And it's actually it resonates a lot because even our legal team, for example, you know, build apps using Claude code, add some skills, check our skills into the Tessl registry. And it's like, it's amazing how much that's empowering. Not just the engineering community, but the the whole organization, which we'll touch in a little bit more depth.

31:52 I'd love to ask the question about, you know, when there's so much of a dog

Agent identities enable scalable enterprise permissions

31:55 food and culture around tag around Claude Coder in the early days as well. How much is the product direction driven by your internal feedback and the internal dog feeding? Yeah, that's a really good point. I think this is yeah, this is really important. But I think there's a there's a couple of things to think about here because like quite often with I think product development in the era, like there's there's things that on the surface look like an amazing idea and feel like an amazing experience, like single player.

32:26 But when you really think about what it takes to scale that thing to the enterprise, it's a really different shape of problems. So like in terms of the interaction with Claude Tag, like, I think we all knew really early that this was something that was working really effectively. And we're also aware that, like at Anthropic, we're pretty liberal with the way that we like, or we're pretty generous with the way that we like, share information with each other. Obviously there's like some very strong guardrails there, like what is like strictly private information to a team.

32:52 But we kind of have all of that set out in our like slack workspaces, etc. like we have really good guardrails for the positioning. And what that means is that within those spaces where you know that it's like trusted places to share information, people can be really open, and that's what allows our agents to perform really well. So one of the design, one of the design principles that we had with Claude Tag is just like we've really carefully designed how you permission each channel. So each channel, the workspaces and the channels have their own permission scopes in terms of what tools they can access, what API keys they have for different services and connectors, and potentially what other channels, etc.

33:27 they can access. So I think like obviously on one level, we want to kind of share what cultural practices are allowing us to work really well with agents. So one of these is like working in public, but at the same time, make sure that our products come with the guardrails baked in so that you can reasonably achieve this behavior in a way that's not like that. You can actually scale to an enterprise like we think that that that works really well.

5 Highlight · Agent identities versus user permissions in Claude Tag

33:49 Another thing that we did to make this work more effectively is we came up with this concept of agent identities. So one really big difference between tag and you working with like Claude Code or Co-op is when you work with code or co-op, they kind of assume your own permissions, so they'll kind of work using kind of my API, like, yeah, my API keys or my permission systems and I grant access to all those things.

34:15 Whereas with Claude Tag, we actually give that agent its own permissions and its own keys, etc., so that it can go off and like work autonomously on these things. It's not working on behalf of one individual, it's working on behalf of the team. And it's much easier to kind of audit that. It's not like doing this as you it's doing this as itself. And so that's another key architectural change that we needed to do to enable this multiplayer behavior. So yeah, I think to round that point off, there's definitely a lot of the kind of product level feedback that all of the teams will chip in with, and we'll make sure that it really works for different kinds of use cases and different types of users.

34:51 But at the same time, we think a lot of the work goes into

Simplify harnesses and avoid opinionated memory abstractions

34:53 making sure that this is something that actually scales to enterprises, and people can really get value out of. Amazing. And let's talk a I know our audience as well as the industry. The way they improve, the way they get better is through understanding and hearing what not just what worked for us, also how we tripped over how we felt and how to how to get up and try and find another, another path. And I'm sure Anthropic, just like every organization have, of course, you know, areas that they that they tried and didn't get on with.

35:21 So I guess from an adoption point of view or from a ways of working with agentic coding, what were some of your or Anthropic greatest learnings, would you say, in the way you were using Claude code, the way you were using maybe even Claude tag as well? Yeah. Of course.

6 Highlight · Simplifying agent harnesses as models improve

35:36 So I think, yeah, there's a few, there's like some on the development side and some on the behavioral side. I think on the development side, one thing I said before, like we should always build for like what you think is going to be like where the models are headed, not where they are today. And one thing that we think about a lot on the Claude code side is every time that we have a new model, we've discussed kind of how those models themselves become more capable in certain dimensions. And what that means is that we like very regularly revisit what those harnesses look like.

36:05 And we're very happy to, like, delete stuff from that harness over time to make it more simple and more lightweight and just let the model do the heavy lifting. So over time, we see something like the harness actually gets smaller because like, we can trust the model more with certain capabilities and we just need what's there to like, give it the tool use and infrastructure that it needs. So that's one thing I think new model doesn't mean like chuck in like way more prompts and way more, more architecture. Sometimes it means like less is more.

36:32 I think another thing is like to to reference back to that talk that I did for you guys on dreaming. We especially working with startups, I get a lot of people asking me about memory and context infrastructure in particular. And, you know, it's not a one size fits all kind of solution. And I think that people come up with really innovative ways to structure their memory databases or memory structures, etc.. And the solution we have in our managed Agents API is like a very simple memory file system that just leans on agents abilities like read and write to to memory.

37:05 And something I touched on in that talk is that we tried a lot of different things in the past, like indexed memory stores or like tools that were very specific about how to read and write a memory. And I think what we learned over time was like, this was just this had like a number of problems. Like sometimes we were being too opinionated about how the agents should interact with memory, and they were better left alone, especially as they became more capable to just manage that. And they were great at just using file systems and the native like bash and grep tools.

37:33 So one thing we learned was like, we could actually just remove some of these abstractions and even our like being opinionated on the structure of those like memory structures. We realized over time that like indexing, it wasn't something that we thought was best practice across the board,

37:47 and we thought a simple file system was better. So obviously you have to learn these things by trying. And all of these things are very open areas of research and development, and I'm sure we'll find more best practices down the line. But these are a couple of examples where we've kind of tried a few things out and simplified our workflows. A bit. It's really interesting. I'm really curious about the kind of like the context piece because or not sorry, the context piece, but the, the, the the agentic model changes and whether it's the agent change or the model change, it really does affect what we actually need to provide it, whether that's context or memory, in order for it to perform the best it can.

38:23 And I think what's most interesting is we don't need to change our code or our context in order for us to need to rerun an email to see, is this actually still valuable in its current state, or because of a model change or because of an agent upgrade? You know, do I actually need to provide it with less context because the model or the agent has actually got better doing these things without the context? And as a result, am I just bloating context by adding this skill, or am I or do I need to change the skill for this model?

38:52 And I think it's that continuous evaluation of our environment that we need to do on a regular basis to say, what do I need to change? Is it the context? Is it my prompt? Is it my harness because of an agent change? Yeah. And that's like it sounds like that's something that's very commonplace then within Anthropic. Yeah. So I think when we're, when we're testing these new models, especially on the applied AI side, because we're working closely with, with customers.

39:20 Well, in our early, early stages of testing, like really look out for what are these changes in behavior that we need to prompt around and like, where are the areas that we can be a bit more relaxed about now because the the model is just better. So we'll always come out with some guidance on like, what are the best practices for working with this new model and help customers with those migrations as well? So yeah, there's a lot of helpful resources that will always publish around model releases to help people make sure that they can, like, move over really easily.

39:47 Hey everyone! Hope you're enjoying the episode so far. Our team is working really hard behind the scenes to bring you the best guests, so we can have the most informative conversations about agentic development, whether that's talking about the latest tools, the most efficient workflows, or defining best practices. But for whatever reason, many of you have yet to subscribe to the channel. If you're enjoying the podcast and want us to continue to bring you the very best content. Please do us a favor and hit that subscribe button.

40:15 It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.

Automating incident response with high-trust human gates

40:23 All right, back to the episode. Let's talk a little bit about you mentioned very briefly a while back about how the marketing team, for example, which is wonderful, and I think Claude Tag really makes this easier and empowers people because, you know, folks who are non-technical or rather outside of the engineering team are probably much more comfortable engaging and interacting with a coding environment when they're doing it through slack.

40:51 How has Anthropic used Claude code? Claude tag outside of the traditional engineering spaces? Yeah, in in just so many ways, honestly, I think every team, like, the whole company is running on the rails of Claude, like, ultimately, another, another like, fun stat is in the last year, the amount of work which we as a company are comfortable delegating to Claude across all teams has doubled.

41:18 So it's gone out from like 30 to like 60% in terms of like where we're able to rely on Claude. So yeah, the marketing one is fun. That one was like a, like a pipeline for like ad generation or like copy generation. We have a lot of our, like, incident response infrastructure also relies on Claude to some extent to like, triage things and like looping the right people and where it can, it can kind of like start to diagnose, like code problems, etc..

41:44 So all of these different solutions rely on slightly different configurations of your agents. So like for example, on sales, we on our sales teams, we have like weekly briefs that Claude kind of like runs over what everyone did that week and gives you all of the stats and dashboard updates so that everyone's ready for the meeting. And nobody had to, like, labor over those slides. So that's one thing that just like runs on a schedule. It's just like so clearly a time saver for everybody. And then there's other things that are more responsive, like these incident response things where Claude knows when to jump in and it knows how to like proactive to be so that we really see, like all flavors of these things.

42:17 Like I for example, on when I'm developing products, I'll have like interfaces where I can type feedback into prototypes and then just like Claude, code will just like work on those in the background. So everyone builds our own tooling in this sense. And I think a lot of this has gone into how we've thought about that development of Tag then, which is called tag, which is meant to be really accessible across the board for all teams. And for example, like that proactivity thing is something you can really dial up and down so you can have everything from like Claude only responds when it's tagged, or Claude creates a schedule on which it runs its tasks.

42:50 Or Claude like will proactively jump into threads here when it thinks that it's got relevant contacts to share. And what we've learned from, like previous products like Claude Code and also on our managed agents is like what works well in terms of this, like scheduling and like knowing how proactive to be. Because the worst thing in the world is a bot that responds to everything with kind of annoying context. Like we see this proactivity thing as a real spectrum, and we've kind of really tuned it over time to that kind of knows what's appropriate, where to jump in and when not to, and when to do things on a programmatic schedule.

43:22 And that's something that you can also steer as a team. Like if Claude does something that you think is not not aligned with your preferences, you can just tell it and it will update its memory and behave more similarly to what you want in the future. Yeah, it kind of reminds me of thinking back to a previous episode that we ran. something which Olivier Pomel, the CEO of Datadog, was, was talking about about how, you know, is us getting up, humans getting up at 3 a.m.

43:48 because of a of a critical issue, you know, how long is that actually going to last for? How much do we trust agents to go ahead and, you know, on an incident actually, you know, go ahead, make a fix that could be reversible. Hopefully it will be reversible. And then maybe in the morning we wake up, notice something had happened and choose if it's the right way of doing something, perhaps reverse it, do it a different way. But we can allow we can rely upon agents to do this. And thinking about what you were saying, I kind of like envision this space whereby we have observability data, which is maybe even going into, into, into slack providing information.

44:23 And then I can see something like tag looking at something like that. And then when it realizes something is perhaps a little bit of looking say, oh, should I raise the incident? And you're almost like, have you know, an agent like Claude Tag saying, aren't you? I'm going to raise an incident here, and then I'm going to do some make some changes, document what I'm doing, make my change, push. That is, you know, it feels like this is actually. Oh, I'd love to actually experiment with this now and actually see if that's an interesting path.

44:50 Is that something you do and. So yeah, the the incident response agent is definitely a pattern that we see working really well. And as somebody who used to be an engineer who was on call, I know the fear of the PagerDuty call coming through at three in the morning, and it's not a nice one. But I think this is a really interesting one, which, which plays on a couple of important design principles, like one is how much, how much?

45:16 First of all, like for this agent to do a good job, you need to give it good, good access to different data sources. So like a data warehouse and potentially like whatever logging and metrics you have and also potentially your repository so they can start to diagnose things. This is like a very good starter kit, I think, for that kind of agent. But the other thing that's really important, and actually something which feeds into how we've designed our managed agents product is where do you want to design those gates between like human agent interaction.

45:48 So it's really up to teams. And we totally understand that rolling these things out at scale needs this like journey of trust that you need to go on. So like maybe when you start doing this, you just like let Claude have a go. And you also like keep your traditional process and you just like check that on whatever, like success thresholds, it did what you wanted it to. And then over time you get more confident, like delegating more and more of that work over. Or maybe it just starts like diagnosing fixes and passing that over to engineering team, waking them up if they think it's critical enough to like potentially starting to raise a draft PR to like whatever other like gates of access you want.

46:25 So I think this is just like one of those where you can really see the value for like engineering teams of like this being less of a problem and helping you resolve incidents much faster, which is definitely what we've seen internally. And it just needs you to like, carefully think about like, yeah, where do you want Claude to ask for your your approval on these things? And that's something that is teams and selves should think about and configure. We can suggest like what we've seen work well but it's a yeah I think it's obviously a high trust. Yeah. It's funny as a previous laptop, I used to have a sticker that said AI works while I sleep.

46:56 And now I'm kind of like, there will be a, you know, will break through that trust barrier and it will be AI fixes production while I sleep and it's, it's it's a it's the right path. I think it's a it's, you know, in terms of diagnosing in terms of getting root causes. A, you know, agents will find that data more accurately quicker. I think, and the loss of the outage, you know, it's more expensive sometimes when you look at it from the, from the cost of an outage to have a, have a human, you know, diagnose, find the root cause, propose a fix, etc., etc..

47:31 So there's definitely an interesting balance there that we're going to see how it's going to be. It's going to be fascinating to see that that shift. Definitely, definitely. And I think in any case, I would much rather be woken up with a call that's like, hey, there was this incident. I think this is the PR that fixes it. Here's the test I ran to verify that's the case. Like this is the blast radius. I'd much rather that was why I wake up to than like, please, can you, like, look at this incident. With a time pressure or. Exactly, exactly. So even if you still have that human gate, I think that like that information handover looks quite different in the two cases.

48:00 So I'd happily, like, approve that PR at three in the morning. Absolutely. So you gave it a wonderful session, native Dev on London just a number of weeks ago, actually. And in fact, we have our Native DevCon in New York, which was announced

Dreaming enables automated agent memory optimization

48:13 that's going to be happening in November 2026. So take a look at that as well. And now you mentioned a concept called dreaming, which was super curious. First of all, why don't you tell us a little bit about dreaming what it is? Yeah of course. So dreaming is a research preview feature that we have on our managed agents offering. And for those who aren't familiar with managed agents, Claude Managed Agents, this is a product which essentially allows you to build and deploy agents much faster in production.

48:42 So we've we take on a lot of the everything from kind of managing the harness of your agent on Anthropic side to some of the infrastructure and the observability, etc.. And what we're really doing with this product is taking all the learnings that we've, we've got from building these agents over some period of time and building them into like, really concrete primitives, like agents and environments and sessions that allow you to quickly compose those agents and deploy them really fast. So that's the managed agents product.

49:08 And of course, given how important context has been as a concept in agentic development, it wouldn't be complete without a memory feature. So this allows agents to read and write to different memory stores as they learn things. And that's like very well accessed gated. So there's like huge organization level context which is really important. And that can only be read from to like agent scratch pads that allow them to drop context about the work that they're doing, which is just amazing

7 Highlight · Using dreaming for automated memory optimization

49:37 in terms of enabling their work. And like I said, people often ask about like, well, what is the best practices for structuring these memory systems? This really starts to run the risk as you run it over longer periods of time that there's like stale information in there. Some stuff has gone out of date, there's missing information, it's just confusingly written, etc.. And so what we designed and introduced is this concept called dreaming, which is kind of what it sounds like, I suppose, where essentially you are able to run these dreaming jobs at whatever cadence you like, where you input some of your memory stores and some of your session transcripts from managed agents.

50:15 So these are basically like basically the traces of how your agents have gone and carried out a couple of tasks and you give this all to another agent, and it basically reviews those transcripts and it reviews the memories and it looks for any kinds of discrepancies. So maybe it finds that some information is missing, that the agents would have performed better if they had that context or vice versa, that there's something misleading in there which is degrading performance, or it just finds a new way to kind of reorganize that information to make it easier to search and surface for the agents.

50:43 And it does this all in like a pretty, like a pretty extensive manner, like it gives you hypotheses for what to change, gives you attachments to the sessions where it thinks that the evidence is there, and then you can basically decide which of those changes to implement. And I think what's really important here is like this really opens the path towards continual learning, like this idea that you can run your agents on one day and then based on whatever could have been optimized, you can run them the next day and actually see that they get better.

51:12 And with dreaming, you can kind of hand over that kind of to, to a large degree, like allow that process to run in an automated fashion and then just to prove whatever you think is relevant. So it's something we're really excited about. And we've seen a bunch of customers just see much better performance improvements with their deployed agents when they run processes like this.

51:31 Amazing. And if folks in the audience you want to learn more, your your talk is actually online. So we'll make sure we link the audience to that. And you can see Lamis's session of course full. And yeah, just add another point there I guess. I mean it was such a delight to be there that day. And I think like we said about FOMO with features, it's nice to have the opportunity to speak about one of those, like more,

Practical daily workflows with Claude Tag

51:53 more complicated features potentially, that people have less opportunity to use. So yeah, hope folks that find that enjoyable. Absolutely. It was people love the love the session. We got amazing feedback about your session. So thank you very much for that. So why don't we we're running out of time. But we should we should wrap I'd love we always love giving practical advice and practical things that our listeners can do. So so what would you say is, is something that you would say for folks who are maybe, you know, trying to introduce maybe it's managed workflows, introducing agentic development further in their organizations.

52:29 What would you say as some day to day practices that people can can do from your experience that will that will massively unlock the next stage for folks? Yeah, of course. And I think just like thinking about, you know, potentially broad audience here, I would really encourage people to like go and set up Claude Tag and give it a try. You just need to get your slack admin to turn it on and do some of the configuration. But it really doesn't take long. And to give you a flavor of the things that I'm doing with this. So the first thing I set up was a daily briefing.

52:55 And because of the connectors and contacts that tag has access to, it's able to tell me about things that happened over the past 24 hours, especially working with international teams, like things that my teams who are in San Francisco did. It just like immediately surfaces at all to me, so I don't have to wake up to like a wall of like, emails and slack messages. But I just have a nice curated brief about what needs my attention. It has context on my ongoing workflows as well, so it can tell me, like this thing that I'm working on, maybe this talk, like maybe you want to review these documents before, before doing that, etc..

53:24 So Daily Brief is great. And then something else I have set up is it it knows like which channels are important to me and it pings me about anything urgent that needs my attention on a live basis. So anything it sees that like thinks is relevant to me, it will tell me about. And that's great because while working in public as a company is fantastic, it means there's a lot of things going on on slack, and I cannot keep on top of that. And then some other things like on a team level, what we have is like tag will do like weekly reports on various things.

53:54 So on the applied AI team, we have a weekly report of like his different things that the team learning saw this week, which is really nice, encourages people to kind of keep sharing that context and really scales that knowledge across the organization. So that's cool in terms of like best practices. And yeah, I think these I think these are a couple of good places to get started. And I think you'll very quickly get a feel for what the kind of capability of that is. You know, one final flourish you can do is like, ask a tag to build some custom software for you, and it can deploy it into, like, Claude Code artifacts, and then you can kind of share that as even maybe just a personal dashboard, or you can share it with your team and be like, hey, this is something to track x, y, z workflow that we have.

54:34 So yeah, these are some good places to get started. Amazing. And it's actually super timely because just this week I'm a big fan of things like getting things done and those types of workflows. And one of my biggest areas of trouble of doing those types of things is the reviews and the check ins. And so what I did was I created an app which I've which I've deployed, and it does very similar. It looks through my slack, it looks through my email, it looks through my Todoist to do lists, and it also looks through my granola notes as well. My meetings.

55:03 And it will add a whole bunch of to do's that I think that from granola. And it will, it'll essentially say, oh, here are some things that you should be aware of or should add to your calendar. It adds them in. It's kind of like a little bit like my EA as well. And I'll tell you what, that's really unlocking productivity. So I totally am on board with those reviews, weekly reviews, monthly reviews, and daily daily check ins and reflections. It's a real game changer from a productivity point of view. Yeah, no, definitely give it a go with.

55:29 I need to. I need to switch to tag and. Try how it goes. Some of the ones I really like as well are like just getting these agents to like, tell you what you did well that week. Like, that's a really nice thing sometimes that you don't have time to reflect on. Just like, tell me like three wins I had this week. Tell me a couple of reflections on things I could like, optimize, etc. but sometimes you don't have time to reflect on those things. And you know, in the world of AI, a lot is happening very quickly. It's nice to be able to take moments to reflect on what's. Going on. And what I also do is I actually also put my I've just recently had an annual.

55:57 So what I do is I put my annual feedback in. Yeah, as well as other things that I feel like my management team and things like that would want of me. And it also gives me feedback based on what I'm doing. Am I, am I, you know, taking my next steps in improvement? Am I am I doing what the team are needing from me and those types of things which are actually really valuable in terms of you're doing what you want and actually, you know, other people, their needs are also being satisfied by some of the stuff that you're doing.

56:25 So it I don't know. So all I need is an agent to just do my work as well, and then I can just let it go on. We can just, you know. We can we can go enjoy the sun. Yeah. Amazing. Lamis. This has been this is absolutely flown by. It's been an incredible discussion. I very, very much thank you. Not just for our Native Dev session, which was like I said, it was very well received but absolutely wonderful and fascinating conversation. So I really, really appreciate you joining us and thanks for all the insights as well as to how Anthropic are using Claude tag Claude code, things that you've learned.

56:55 It's been it's been wonderful. Thank you so much Simon. It's been an absolute blast as well. And thank you so much for having me. Amazing. Lamis. Thank you so, so much. I'm sure our audience loved that discussion. Tune in to the next episode. Bye for now. The AI Native Dev is brought to you by the package manager for skills and context. Your hosts are Guy Podjarny and me, Simon Maple. Our producer is Tom Dowler. The AI Native Dev is not just a podcast, it's a community.

57:21 And we host monthly meetups at the Tessl offices in central London. Visit Tessl IO forward slash community to learn more and I hope to see you there.