Measuring Developer Productivity to Maximizing Software Engineering Success

Vishal Pallerla•July 18, 2023

Measuring developer productivity is a complex task and involves a variety of factors and metrics. These metrics help evaluate the efficiency, collaboration and effectiveness of an engineering team. Understanding these metrics sets the groundwork to build a productive and successful team.

Measuring Developer Productivity: Key Metrics and Approaches #

1. Code Quality #

Code quality is an essential aspect of developer productivity. High-quality code is easier to maintain, debug, and extend. Some metrics to assess code quality include:

Code review feedback: Analyze the feedback received during code reviews to identify areas of improvement and track progress over time.
Static code analysis: Use tools like SonarQube, ESLint, or Pylint to automatically analyze code for potential issues, such as code smells, bugs, and security vulnerabilities.
Test coverage: Measure the percentage of code covered by automated tests, which can help ensure that code is reliable and maintainable.

Learn how Google measures Developer Productivity

2. Velocity #

Velocity measures the amount of work completed by a team during a specific period, usually a sprint or an iteration. It can be measured using:

Story points: Assign story points to tasks based on their complexity and effort required. Track the total number of story points completed in each sprint to measure the team's velocity.
Task completion: Count the number of tasks completed during a sprint and compare it to the total number of tasks assigned.

Image from Atlassian

3. Cycle Time #

Cycle time is the time it takes for a task to go from being started to being completed. It can help identify bottlenecks and inefficiencies in the development process. To measure cycle time, track the following:

Time spent in each stage: Measure the time a task spends in each stage of the development process, such as development, code review, testing, and deployment.
Average cycle time: Calculate the average time it takes for tasks to be completed.

So, to measure cycle time, you can track the time from when a developer starts working on a branch until that branch is released to production. For instance, LinearB measures cycle time by breaking it down into four phases: coding time, pull request pick time, review time, and deployment time. The total cycle time is the sum of the time spent in each phase.

Image from LinearB

Let's demonstrate cycle time measurement with an example:

Suppose a developer starts working on a new feature on July 1st. They complete the coding phase on July 3rd, and the pull request is picked up for review on July 4th. The review process takes two days, and the feature is approved on July 6th. Finally, the feature is deployed to production on July 7th.

To calculate the cycle time, we need to determine the time spent in each phase:

1. Coding time: July 1st - July 3rd = 3 days

2. Pull request pick time: July 3rd - July 4th = 1 day

3. Review time: July 4th - July 6th = 2 days

4. Deployment time: July 6th - July 7th = 1 day

Total cycle time = 3 days (coding) + 1 day (pull request pick) + 2 days (review) + 1 day (deployment) = 7 days

In this example, the cycle time for the new feature is 7 days. Next, you should examine your cycle time from various angles, such as from the perspective of your organization, your team, your iteration, or even your branch.

You can automate the calculation of cycle time using tools like LinearB or Keypup to get a more accurate and consistent measurement.

4. Lead Time #

Lead Time measures the time elapsed between the identification of a requirement and its fulfillment. It starts when a customer makes a request and ends when they receive the completed feature or product. In other words, Lead Time includes the time it takes for a feature or change to go from the first commit to the pull request running in production.

The main difference between Cycle Time and Lead Time is that Lead Time takes into account the waiting time before the development team starts working on the request, while Cycle Time only considers the time spent actively working on the request.

Lead Time = Cycle Time + Wait Time

It includes the time spent waiting in the backlog and the actual development time.

For example, if a customer requests a new feature on July 1st, the development team starts working on the feature on July 4th, and it is deployed to production on July 12th, the Lead Time would be 11 days (from July 1st to July 12th), while the Cycle Time would be 8 days (from July 4th to July 12th).

5. Team Collaboration #

Effective collaboration is crucial for developer productivity. Some metrics to assess team collaboration include:

Code review participation: Monitor the number of code reviews each team member participates in and the quality of their feedback.
Average time for code review completion: Measure the time it takes for code reviews to be completed.
Time spent in code review queue before closure: Track the duration tickets spend in the code review queue until they are closed.
Communication: Observe the team's communication channels, such as chat platforms and meetings, to ensure that information is shared effectively and efficiently.

6. Continuous Improvement #

Regularly review and analyze the metrics mentioned above to identify areas of improvement and implement changes to enhance developer productivity. Encourage a culture of continuous learning and improvement within the team.

Keep in mind that measuring developer productivity is not an exact science, and it's essential to consider the context and specific goals of your engineering team. Use these metrics as a starting point and adapt them to your team's unique needs and objectives.

In addition to understanding the significance of these various metrics, you may be wondering how to effectively calculate and utilize them in your software development process. We recommend referring to our detailed guide, How to Calculate Developer Productivity Metrics Using MergeStat and DevZero. This comprehensive tutorial will assist you in setting up a DevZero account, creating a MergeStat template, running few queries related to the metrics we have discussed.

Combining Metrics for a Holistic View #

It is important to note that no single metric can fully capture the productivity of an engineering team. Instead, use a combination of these metrics and tailor them to your organization's specific processes and goals. For example, consider how different emerging development practices, like the usage of feature flags, influence common metrics. Additionally, focus on measuring outcomes rather than individual output, and assess the productivity of the entire team rather than individual developers.

Beyond Tracking Numbers: Identifying Areas for Improvement #

Keep in mind that measuring developer productivity should not be solely about tracking numbers. It should also involve identifying areas for improvement, optimizing workflows, and enhancing the overall engineering process. Using feature flags, as discussed in this section, can enhance workflows and efficiency. By doing so, you can help your engineering team stay consistent, efficient, and productive.

Maximizing Software Engineering Success: Balancing Efficiency and Effectiveness #

While efficiency in software engineering often translates to higher productivity, true software engineering success weighs heavily on the effectiveness of the team and its strategies.

Software engineering is complex and distinct from other fields such as manufacturing. Maximizing work in progress leads to increased wait times and interruptions as every pull request in software development is unique. Optimizing PR-based workflows requires asynchronous code reviews and strong collaboration, much like the measures implemented in productivity metrics.

Working together on a single story can be more effective and reduce lifetime costs.
Pair programming or mob programming can lead to faster, higher-quality code.
Proper resource allocation is crucial for engineering leaders.
Focus on smaller batches of work to achieve thorough and effective progress.

Reimagining DORA Metrics and Leveraging Feature Flags #

In the past, deployment and release often meant the same thing - shipping a change to customers. Under this model, DORA metrics were great for identifying teams that were adept at shipping high-quality software without disruptions.

However, the world and its needs have since changed. Feature flags have offered a new paradigm for deployments and releases, separating them into two distinct events. In essence, feature flagging allows you to deploy code but release it to the end users at your discretion.

How Feature Flags Change DORA Metrics #

As Ariel Perez, VP of Engineering at Split.io shares in Dev Interrupted podcast, this decoupling changes the granule of the core DORA metrics:

Deployment Frequency morphs into Change Frequency #

Lead Time decreases dramatically, as pending changes can be merged behind a feature flag.

In a world with feature flags, it's not just about how often you're deploying, it's also how often you're releasing.

Feature Flags also have a consequential impact on Mean Time to Recover (MTTR). For the elite teams, MTTR becomes limited by how quickly a fix can be shipped. But here, feature flags allow recovery by simply turning off faulty flags, turning MTTR almost instantaneous on best platforms.

Simply put, it seems DORA metrics need to be reimagined in a world with feature flags.

The Importance of Measuring Value #

In the sea of advancements, one must not lose sight of the importance of value in what we deliver. It's believed that about 70% of everything that goes into production has little, no, or negative value. Therefore, to learn and to improve, we need to measure the impact of every single feature that is being shipped.

Causal Analysis #

Causal analysis involves feeding telemetry into a feature flagging platform and determining whether a particular feature flag is having its intended or unintended impact. With data and smart analysis, faster decisions, capable deployment, and efficient learning are not far off.

From Outputs to Outcomes #

In a world driven by feature flags and advanced metrics, the key to achieving desired outcomes is to focus on shipping promptly and improving effectiveness through collaboration. Strive to be outcome-driven and prioritize customer value over mere output.

Balance fast shipping with quality delivery, consider long-term costs and maintenance before rolling out features and work in synergy to build better things. In essence, making the shift to an outcome-driven organization while maintaining an ability to ship quality features quickly is the way forward.

Conclusion #

Measuring developer productivity and maximizing software engineering success go hand-in-hand for the successful operation of an engineering team. By using a combination of metrics and focusing on both efficiency and effectiveness, valuable insights can be gained into team performance, and suitable areas for improvement can be identified.

Remember, our goal is more than just tracking numbers. Main focus should be to optimize workflows, balance effectiveness with efficiency, and enhance the overall engineering process, ensuring a more effective and productive team.

In conclusion, keep in mind that code isn't just about making things work. It's important to measure and improve how well the team works, adjust workflows, see the bigger picture, and use tools like feature flags to balance speed and quality. Ultimately, what matters isn't solely about writing code; it's about generating code that truly adds value.

Vishal Pallerla

Developer Advocate, DevZero