Ep. 133: Why is developer experience so complex?

Unlock the secrets to a thriving developer experience as we dissect the critical balance between productivity and fulfillment in the tech world. Prepare to have your perspective on developer metrics as we dive deep into the implications of AI advancements and economic shifts on the roles and recognition of software engineers. This episode is a treasure trove for anyone seeking to understand the nuanced landscape of the tech industry, where the intangible aspects of job satisfaction go hand-in-hand with tangible outputs.

We challenge the traditional yardsticks of developer output, advocating for a collaborative approach to productivity that values the system over the individual. You'll discover the art of using metrics responsibly in a way that empowers and informs developers rather than policing them. Drawing on our expertise, we leave you with actionable insights into fostering an agile environment where collective progress is king and where developers feel secure, autonomous, and integral to their organization's success.

This week's takeaways:

• There is no single metric to measure developer productivity effectively. Look at correlations across multiple qualitative and quantitative metrics.
• Focus on development productivity metrics for the overall system/team rather than individual developer productivity numbers.
• Make productivity metrics transparently available to developers, allowing them to self-regulate performance rather than being managed top-down.

Resource:  

Measuring Developer Productivity: Real-World Examples -
https://newsletter.pragmaticengineer.com/p/measuring-developer-productivity-bae

Tune in for a thought-provoking conversation that reshapes how we perceive and enhance the developer experience in an ever-evolving industry. Subscribe to learn more insights around leadership, agile, and DevOps transformation at scale.

Chapters

• 0:04 Measuring Developer Experience and Productivity
• 10:46 Measuring Developer Productivity and Experience
• 14:42 Focus on Development Productivity in Agile

Transcript

Peter: 0:04

Welcome to Definitely Maybe Agile, the podcast where Peter Maddison and David Sharrock discuss the complexities of adopting new ways of working at scale.

Dave: 0:12

Peter, I'm looking forward to this conversation. Good to see you again. Good to see you again, Dave. You had a topic in mind.

Peter: 0:19

We were going to talk about developer experience and all the different aspects of that, and there's many different avenues of this and there's many different avenues of this we could potentially explore, but I'm interested to see where this conversation goes. It's something that has been coming up both through the industry, press and the media and through our clients as well. There's a lot of focus on it and some of it driving from like a developer productivity angle, like developer productivity versus developer experience or an AI. What's the impact of things like co-pilot and the impact on organization as we start to see these AI tools? So how does that impact the role of software engineers and organizations and what does that look like? And how does that look like? So how does that change?

Dave: 1:05

developers experience.

Dave: 1:07

So, I mean, it's an interesting market that we're talking about here as well, because you know, we're going from a few years ago when developers could do no harm and they were. It could do no wrong, I should say and they it was a growth market. Everybody needed developers and now over the last couple of years, we've seen literally thousands of developers being shed from organizations. So when we start, whenever I think of developer experience, one of the primary things I think of is how comfortable do they feel where they're at? Do they feel valued? Is it clear that you know what their role is in an organization, how they're creating value, how their careers may be progressing is in an organization, how they're creating value, how their careers may be progressing? I think there's a safety element and a make me feel like I'm part of this organization element that comes in.

Peter: 1:52

To start with, I would say that and it's interesting when you're starting from that but that's looking at it from the developer's point of view into the organization.

Dave: 2:00

So you did say developer experience.

Peter: 2:02

Right yeah, developer experience. There's the view of the organization wanting to then measure this.

Dave: 2:10

Yeah, but already now we're kind of tripping over our feet a little bit.

Peter: 2:15

So what is a developer's experience in the organization? Why might we want to measure? This is an interesting topic. So how are developers experiencing it? It's going to vary based on the culture of the organization and a lot of different elements like how easy is it for the developer to get their job done Some of the immediate sort of pushback against the hey. Well, copilot will replace all the developers. Well, it helps them with a portion of their job and their role. But there are a lot of other activities, especially in a large organization, that developers are involved in, other than just writing code. There is also this piece, which is well, do we want them to spend more time writing code or less time writing code?

Dave: 3:03

So let me pull you back before we dive off into what does the developer do and how do they contribute to the organization. You made a comment which has been sort of zipping around my head, which is about measuring productivity I think you were going to talk about, but certainly a little bit more about developers and what good do they offer the company. How do we know we're getting what we need to from the developers? Something along those lines what's being measured or what's not being measured? Can you talk a bit more about that?

Peter: 3:31

well, it's a, it's a question that is being asked. I mean, there's a couple of things, there's a couple of forces which are causing the sort of organization to share developers. One of them is the macro economic situation rising interest rates. Money's not free. We probably overstaffed previously and so they've cut down on the size of it, but the number of developers that they have to be able to do these things. So there's that side of it. There's also then there's the impact of AI coming in, and I don't know how much that's actually impacted that.

Peter: 4:06

I think the first one of those things, the sort of the macroeconomic piece, is the primary reason at this point that the AI piece I don't think has caused organizations necessarily to immediately say, oh, that looks like it can do the job of all these developers, I'll fire everyone. I don't think that's quite how that's gone down myself, but I think there is a desire, an understandable desire, to understand well, how does this impact the developer footprint? Do I have the right developers? How can I use these tools to get more from my developers, to help my developers improve, as improved and on the positive side, I mean where we look at it like, how do I bring these in in a way that they they improve the outcomes that I'm getting from my, from the development teams that I have. Uh, what couldn't? How can I get more productivity out of them? As in and productivity is a bit of a loaded my goodness, I think you've been.

Dave: 4:58

You've been hanging around managers way too long and leaders like here's what here's what I'm hearing you say, which is we need to see productivity grow from our development, from everywhere. Of course, everything is about growing productivity over time. And how do you know your developers are improving their outcomes, right, and and I, but what worries me about this? So when I'm thinking about talking to developers and developer experience, i'm'm thinking in the Dan Pink mastery and autonomy perspective Mastery, autonomy, purpose, exactly, but I'm thinking specifically about mastery and autonomy, as in I want my developers to be in an environment where they can see a path towards mastery, so they're getting the feedback.

Dave: 5:38

When I think about developer experience, I'm thinking are they getting the support they need? Are they getting the direction? Are they getting the tools and the environment so they can become stronger and stronger, can master their craft and the autonomy to be able to go out and do those things? And what I find interesting is the information that you need to be able to provide that to your developer community is very, very, very similar to the information that managers might use to start thinking about is my team X as productive as team Y, and things like this. The data is often the same.

Peter: 6:11

It's how it's used Right and there's a lot into that as well is that you can't look point in time. You have to be looking at the system trends over time. You need to be looking at common cause versus special cause, statistical variation, all sorts of fun things you can get into there. The one of the interesting pieces, let's say so. There's a lot of talk about this. So dora, as uh, is kind of the common sort of from the accelerate book and, like a lot of people, stop at the sort of we've talked about this before. The four main metrics which they showed are correlated to high performance, forming organizations, which really give you a view of your technical capabilities, their outcome metrics. If you look at these, the outcome is that your technical capability is improving. But they're not the only thing that you should measure.

Peter: 6:58

In fact, not long after Dora, a couple of years, they came with Space, which extends Dora in a lot of ways, and Space Metrics adds some of the pieces you're talking about. So you're talking about what you want to see from the developers. I'm talking about it from the organization side, about how do we know that we have those things? So if we can get agreement that, yes, we want the developers to have a path to master. We want them to have autonomy. And how do we know that we've created that in our organization? Like, how do we know we're going in the right direction to create that developer experience in the organization and so things like space, and give the sort of the additional pieces into that, or guidance around what some metrics you might want to measure, around that which it marries up the sort of the qualitative and quantitative sides of measurement to start to understand what is the developer's experience.

Dave: 7:55

I think we're sort of circling around that whole productivity question again, kind of coming back to it from a different perspective or with a little bit more information. And it is absolutely correct that organizations are obsessed with productivity. It's a competitive advantage, it's something that you know. There are lots of different perspectives on where productivity has gone in the last few decades, but basically I mean, if you look at something like AI and you look at the cutting edge things on the technology, the reality is that productivity should be growing a lot. Right, we should be seeing these tools augment what people are doing and being able to improve productivity in that area. So productivity growth is on everybody's agenda. Again, one of the things that really bugs me about this is and we've talked about this the power of things like bringing teams together.

Dave: 8:42

There's an intangible element to productivity growth and it's really easy to look at it from hey, you've got access to co-pilot, we're doing this, we're doing the other, we're, and all of that's going to speed you up. But this is just a modern way of saying. We should teach our developers how to type quickly so that we get more code and then the problem, of course, that more doesn't mean better in the development process, Absolutely yeah, so I mean.

Peter: 9:08

The way I typically give the example is that the developer who spends seven hours thinking about a problem to write one line of code that makes you a billion dollars is better than the one who spends eight hours creating a thousand lines of code that don't do anything for you. So you can't measure lines of code, as you've got to look at the outcomes of like is this helping me get to the solutions that generate the outcomes I want faster? Am I? Am I enabling my doctors to be able to solve those solutions? And then things like code are incredibly valuable if they help them with doing that. So and then. So that's where we start to look at it from that perspective.

Dave: 9:44

Yeah, and if we come back to the question, what would you pick up from a developer experience? What is it that you think are the top three things that need to be on the table for a conversation there?

Peter: 9:55

I think that it varies by organization and I'm not the only one to see that there's lots of information out there. But finding what way you're like, I think it is important to measure it. I think it's very, very important to measure it. I think it is important that you don't measure it based purely on activity, that you base it on understanding the correlation of lots of different metrics and not just one. There's no one metric that will tell you everything about developer productivity. There isn't one. It just doesn't exist.

Peter: 10:28

So you've got to look at lots of different elements and some of those elements need to be qualitative. You need to go talk to the people and understand, like, how are they feeling? Because it's like telemetry and other pieces, just because it looks like things are going better based on the numbers you're getting from the system. If you don't go and talk to the people and understand like, well, what's getting in the way of you improving? Like, what would help you the way of you improving, what would help you get better? What else might you be able If I could remove barriers for you? How could we improve further?

Peter: 10:59

So if you don't go and ask those questions, you're not really going to be looking at developer experience but there are some great articles out there which talk to. Maybe we can link them in the show around, like talking about the different types of metrics that high-performing organizations measure in the developer experience space and looking at how that might, how those get applied in different situations, because it might not even be the same across all of your organization, different parts of your organization. You may want to measure different things for that part of the organization due to the technologies they're using, the types of systems they're dealing with, the client interactions that they're working with. All of these types of things.

Dave: 11:41

What you're describing there. I keep coming back to the idea that there's these metrics and data and information. If it's made available to the development teams themselves, you don't need to use it to manage. And the reason I say that is developers. They're very numerate, they're going to look at the data, they're going to draw conclusions and comparisons and challenge one another without any sort of prodding or oversight or anything from a management perspective, and yet it's the same information. So it's a difficult. That nuance is really important, though.

Dave: 12:12

One of the experiences I remember many years ago now probably 20 plus years ago working with an organization that had some of the best metrics I've ever seen from a developer perspective, like they had and I I mean, I know these numbers on their own are garbage numbers but they had things like lines of code and how many defects did the reviewers find as a percentage of the number of lines of code they were reviewing, and so on.

Dave: 12:34

So they had level of detail like loads of information about really minute things, which again individually rubbish. But as you put a lot of these things together, you start getting a picture of how those developers were working. What was really powerful is that was made transparently visible to the developers, and the managers were not allowed to use that data, and so what that meant was the developers held one another accountable to a very high standard that mastery and autonomy piece because they felt safe, that their managers weren't going to come in and have a conversation with them about, hey, you know what? The number of lines of code you're writing is not nearly as many as so-and-so right All of these things that can be abused if they're not done correctly and that's the bit that I'm hearing. It's a really difficult piece to get to right. We need to understand our system, our organization and how productive that is and what's going on there. But how do you do that without getting into the individual performances conversation?

Peter: 13:34

Yes, yeah, exactly a. It's not measuring the individual, it's measuring the system and, uh, and making that transparent. I agree, I think. I think that's a good way of looking at it, to make sure that, uh, the it doesn't become something that's going to be like weaponized essentially, which is always the danger when you start to measure things. That, and always the fear as well. So it's the other good reason to make it visible to everybody so that, uh, it takes away some of that fear. Awesome. So, um, do you, do you have any particular points you want to help?

Dave: 14:05

us. I was trying to get you to summarize with the top three. I think this is very definitely close to your heart, so maybe just pull out.

Peter: 14:12

Pull out a short two or three things that we should take away from our conversation all right, so so my my top sort of three things that I I typically uh want to talk about this there is no one single metric that allows you to measure uh developer productivity. You're far better looking at um your, your development productivity versus your developer productivity. You want to look at the system and not the individual, so that counts as two or three.

Dave: 14:40

It's a good place to stop. I think that's a great phrase to kind of close out with is to focus on the development productivity, not the developer productivity.

Peter: 14:49

Awesome. Well, thank you. As always, don't forget to hit subscribe and yeah, so you can get to us at feedback at definitelymaybeagilecom. Until next time. Until next time, peter, thanks. You've been listening to Definitely Maybe Agile, the podcast where your hosts, peter Madison and David Sharrock, focus on the art and science of digital, agile and DevOps at scale.