DevEx Framework: A 3D Metric Approach to Developer Effectiveness

Using surveys and qualitative data analysis, researchers have established a direct link between a developer’s state of happiness, developer productivity, and the organization’s engineering effectiveness. 

Another report, by Mckinsey, links improved developer velocity with key business performance indicators, such as the ability to innovate faster, higher customer satisfaction, improved brand perception, better talent retention, and the ability to outperform the market by 2-5 times. 

Leadership at most of the organization are cognizant of this relationship i.e., the influence of improved developer experience on value creation. In fact, companies are already embracing frameworks like DORA, SPACE, and OKR to improve the developer experience and outperform their competitors.
There is a multitude of other reasons to advocate for investing in developer experience (DX, aka DevEx), but in this insight, we shall discuss a new framework that recently created a buzz among engineering managers — The DevEx Framework. This first surfaced as a research paper, authored by the creators of the SPACE framework. 

But the proposed DevEx framework is no silver bullet. It just suggests organizations and their respective leaders about what they should focus on measuring, and how to optimize developer experience.

What are the 3 Dimensions of the DevEx Framework?

The authors of the DevEx framework have segmented the factors influencing developer experience into 3 dimensions:

1. Feedback Loops

This refers to how fast developers receive inputs/feedback in response to their actions. 

This also encompasses the quality of such feedback. 

Instances, where feedback plays a crucial role, include code reviews, CI/CD pipeline build status updates after a PR, test reports and bug/issue status reports, usability, security, performance, and user feedback reports. 

Faster, actionable, and continuous feedback can help developers work effectively with minimal friction.

It will also help the leadership to identify areas where tool upgradation or adoption, process intervention/optimization, and streamlined procedure establishments for the hand-off of responsibilities/deliverables can be established with an intent to improve the developer experience. 

This could safeguard developers from frustration, and delays. 

2. Cognitive Load

It refers to the mental effort, and processing capacity required by developers to code effectively. 

Developers often need to develop complex software systems. Their working memory and problem-solving skills need to be at their best to yield the best outcome. 

And for that, it is essential that they do not get uninterrupted, or else their attention/focus will suffer, reflecting poorly on their productivity. 

Engineering managers & leaders must endeavor to minimize interruptions and streamline things for delivering the best developer experience. 

It means, establishing processes to minimize complex code structures, ensure comprehensive documentation of the code, safeguard them from timeline and delivery pressures, and reduce the context-switching instances that mostly result from multitasking.

3. Flow State 

From a developer's perspective, flow state refers to a joyful mental state of deep concentration, heightened focus, and optimal performance while engaging in coding or software development activities. 

It is often described as being "in the zone", and represents a highly productive and enjoyable state of work. 

When in the zone, developers become fully absorbed in their tasks. Developers in the flow state exhibit an intense and unwavering concentration on their coding challenges. They are fully immersed in coding, blocking out distractions and maintaining a laser-like focus on the problems they are solving.

Frequent engagement in the flow state is conducive to improved productivity, innovation, and individual growth. 

Research has demonstrated that developers who derive satisfaction, and those who are happy with their work are more likely to produce higher-quality code/products. Therefore, it is crucial for teams and organizations to create an environment that fosters a flow state, promoting developer well-being and happiness.

In general, measuring and evaluating metrics within each of the aforementioned three dimensions would unsurfaced the various sources of friction encountered by developers. Accordingly, leadership & the team can respond proactively to re-align & re-prioritize efforts to maximize the organization’s desired outcomes.  

Although developer experience is a multifaceted and nuanced concept, teams and organizations can make significant strides towards improving developer experience by employing the DevEx framework and focusing on these three key areas.

What to Measure for Improving Developer Experience?

Organizations should be focused on measuring perceptual and workflow metrics, and also the north-star metrics i.e., the KPIs. Identify where friction exists by measuring topics across the defined 3 dimensions of the DevEx framework. 

For instance,

Signals to Measure the Effectiveness of Feedback Loops

A shorter MTTR could mean that bugs & issues are being resolved quickly, thus indicating an efficient feedback loop in place. 

Similarly, quick code review turnaround time also signifies streamlined feedback loops for developers to iteratively improve code quality. Hence, boosting the developer’s self-confidence, and overall experience. 

Lower bug reopen rates, higher deployment frequency, and shorter cycle time are some other engineering metrics that could reflect the robustness of feedback loops in your organization/project.

While these are mostly workflow and technical metrics, engineering teams must also put effort towards measuring perceptual metrics such as 

• How happy a developer is with the collaboration tool to exchange ideas and discuss aspects of the received feedback?

• Developer’s satisfaction with the testing methodologies, processes, and results reporting.

• Timeline for a CI/CD pipeline to generate results for the PR.

And so on

Here are some more DevEx metrics examples to spot areas of improvement:

DevEx Metrics to Know the Cognitive Load on Developers

To gain valuable insights into the mental effort required by a developer to deliver the workload, you can measure task complexity by tracking number of dependencies, scope of work, level of technical challenge etc. 

You can also look at time spent by developers on context switching while multitasking, defect rate, time spent debugging, bug reopen rate, and time spent dealing with meetings, emails, and ad hoc requests.

What DevEx Metrics Can Signal a Healthy or Broken Flow State?

Any metric that could help you measure developer interruption or their motivation levels should be measured to evaluate the flow state. For example, context switching, change requests, burnouts, decision autonomy in regard to software architecture/feature implementation, technology choices, skill development, project & personal feedback, recognition, diversity of challenges, and networking opportunities can all be tracked to assess the developer flow state.

The question that arises is how you ensure the accuracy and efficacy of perceptual DevEx metrics. In fact, how do you measure them? 

How to Measure the DevEx Metrics?

The authors of this research paper recommend starting with surveys to start measuring perceptual & workflow metrics. Surveys could be tricky, as the same question can be interpreted in multiple ways by different developers. If you’re designing surveys, ensure that it is based on well-defined constructs and validate their consistent interpretation across teams and developer factions. If possible, utilize both periodic and transactional surveys to gain a deeper understanding of the overall developer experience.

Also, while analyzing the survey results, 

• Ensure that the results are seen at all three levels to get a clear picture i.e., organizational, team, and individual levels. 

• Compare the results against industry benchmarks to rule out perceived biases.

DevEx Framework: Build Productivity and Success for Developers

Unhappy developers have low cognitive performances i.e., lower level of focus, deteriorating performance, and suffer indecisiveness. They also have lower motivation, and a high tendency to quit or withdraw from work. Other consequences of unhappy developers include lower productivity, decreased process alignment, deviation from the best coding practices, low code quality, broken flow state, and decreased communication and collaboration. While happy developers exhibit and yield contrastingly positive results. High cognitive performance, high motivation, great flow state, joy of coding, peace of mind, awesome problem-solving skills, higher creativity, sense of accomplishment & proud, high adherence to processes and coding best practices, increased collaboration, effective communication, comprehensive documentation, the expedited delivery of high-quality code and artifacts are some of the positive consequences that are associated with a delighting developer experience.

It makes sense for engineering managers to invest time and resources into improving their overall developer experience. 

And it all starts with measuring and tracking DevEx metrics. 

As underlined in the research paper, surveys are very effective to measure the DevEx metrics. Besides, make use of engineering analytics platforms, such as Hatica that integrate well with major developers, issue management, code repository, and project management tools. Hatica pulls data in real-time, and presents it in the form of interactive dashboards to help you generate reports, and enables you to devise actionable strategies to mitigate any identified challenges.

Subscribe to the Hatica blog to improve your engineering processes and to keep building amazing developer experiences!