State of Software Development - An Analysis
It's more important than ever to have a pulse on your market and customer needs. Your employees play a critical role in delivering value. As the leading predictive forecasting and risk management solution for software development, we are uniquely positioned to examine how teams are tracking towards software delivery. Through the customer data in our platform, we are first to know when work behaviors change, how it affects throughput, and ultimately, your ability to meet customer goals. We asked our data science team to investigate the trends across our customer base over the past 3 months to understand the impact of COVID-19 on product and engineering and want to share those results.
Now that pandemic restrictions are slowly being lifted, leaders like you must decide how to move forward; the data in this series is intended to help you understand how COVID-19 has impacted business so far, how the transition to remote work affects organizations, and what we can learn in order to move forward with the best outcomes in mind. Our analysis includes:
- How are product and engineering teams adapting?
- How have work patterns changed?
- What tools, processes, and methods make the biggest impact?
- Your old ways of measuring productivity may no longer be valid. How should you measure productivity and engagement given the current conditions?
- What should you consider moving forward?
Context on Allstacks’ Approach to Software Development
Product and engineering offer critical contributions to an organization’s mission. Our software development formula (shown below) explains how these functions come together in the real world to deliver software to the market. As we explain the aggregate data from our study, we will focus on people, process, and outcomes, which are the moving parts of this formula. We focus on what we can control and how to account for the unknowns.
In this section we cover:
- Did productivity change?
- How are people affected?
- When are people working?
- What are they dealing with?
- Are activities taking longer?
- How should you support your team?
Part I - People
As you may know, your people are the most precious resource that your organization has to drive action. How they do their work, what they do their work with, and how you can help them optimize for best results are key questions.
Allstacks’ Founder and CEO Hersh Tapadia discussed the data on how developers were affected by the work from home transition and offer suggestions on how you can support your team.
Organizations have experienced unprecedented challenges as their work and personal life blend together in one environment and due to the sudden need to work remotely. A recent software development trends report by CodingSans revealed that as of February of this year, 60% of projects were handled in person; needless to say, those organizations whose strategy relied on face-to-face interaction were tasked with quickly adapting. We’d like to share the data behind the transition.
Results from the Market Data
Based on our research and in working alongside our customers, most organizations switched from being in the office to at-home, remote work the first week of March.
Insight: Higher Organizational Performance Despite Initial Dip in Throughput
There was a dip in throughput in the first 2 weeks of March as individuals figured out how to exist in the new world where everything takes place at home. Organizations were able to quickly regroup, which see can see in the rebound during the week of March 16 and beyond; software development activity went back to being very stable. The data shows that more work was focused on stories and denotes higher organizational performance overall, despite the initial dip.
Hypothesis: Throughput decreased as a result of the sudden need to change processes, planning, and communication methods. Organizations had to establish the fundamentals first, such as getting individuals the right equipment and access (i.e. VPNs), to do their jobs effectively and securely. Once software teams settled into working from home, they focused on stories, versus another type of work, which indicates that they executed on planned work versus addressing unexpected, new work.
Insight: 25% Increase in weekend work
Work patterns remained steady day-by-day with the exceptions of Monday, Saturday and Sunday. Work activity went up by 10% on Monday. We also see a dramatic increase in work activity during the weekend with a 25% increase in work on Saturday and an 8% increase on Sundays.
Hypothesis: Developers are juggling many responsibilities and blurring the lines between work and home. People had to adapt to a new environment where they are simultaneously prioritizing work and home, including taking care of their children, tending to homeschool, and ensuring the well-being of their elderly family members. The increase in work on Mondays is likely attributed to the fact that there are fewer status meetings scheduled, freeing up time for employees to perform the work. It’s common for organizations to schedule meetings early on in the week to ensure alignment for the rest of the workweek.
Considerations: When employees don’t have clear boundaries between home and work, it can easily lead to burnout. As a leader, you can encourage your team(s) to rest, take care of their mental health, and better separate their work and home life to prevent individuals from feeling like they are always working. While flexible work schedules offer many benefits, unclear work schedules can result in unpredictable outcomes. Keep an eye on work patterns and potential issues to prevent surprises.
Insight: The work day is starting earlier for remote employees with quieter afternoon hours.
When we look at the work activity on an hourly basis, we can clearly see that people are starting their day earlier, with most individuals beginning at 7am; the data also indicates that more work is performed during lunchtime and that workdays are ending a bit earlier. Afternoons are now quieter, and there’s an uptick in activity around 7pm.
Hypothesis: We see a drastic increase in morning work during what would normally be commute time. Individuals have reshaped their days to create routines that best fit their ‘work, live, and play" lifestyle in the same environment. As previously mentioned, people had to figure out the best times to take care of their family responsibilities along with their work duties. Oftentimes, this means parents are taking turns juggling childcare. Without commute times, people are able to get straight to work which accounts for earlier start times. Additionally, without the constraints of a traditional 9-5 office setup, individuals settled into a more natural work pattern for their chronotype, also known as an “internal clock.” They were able to create their ideal daily routine based on their biological predisposition.
Considerations: Encourage your team to work when they can be most productive. As a leader, focus on how your team is tracking towards outcomes versus worrying about when the work is getting done. It’s your job to find out how you can help them deliver work in a consistent, predictable way.
Insight: Work Patterns and Collaboration Remain Steady
The data shows normal fluctuations in work patterns. Cycle times remained steady across the board with a few variants; there were slight upticks in March and April following a stable February. Pull request (PR) cycle times pre and post COVID-19 had no material difference from the time a PR ticket gets opened to when the pull request gets merged into the codebase. Individuals and teams were able to meet commitments.
Hypothesis: Through the transition, people have been able to remain collaborative and drive their outcomes forward.
How to support your team through the transition
At the end of the day, if you make a concerted effort to take care of your people, the work they do will yield better results. Here are a few additional strategies and tips to summarize the information we’ve covered in “Part I - People.”
- Encourage the team to establish work-life separation
- Focus on outcomes, not specific activities
- Encourage the team to settle into their natural work patterns
- Lean into your processes to help support communication
Part II - Process
We previously covered how people were working before COVID-19 versus after the transition and the impact of those changes. This portion of the series will focus on the process, including what necessary changes took place during the transition and best practices moving forward in the remote work landscape.
We also have a special guest, VP of Product Darrell Hyde of ClearDATA, join us to discuss how they maintained consistent software development productivity despite the pandemic’s chaos. He offers insights on building a remote, agile culture and how his team focuses on delivering customer value.
In this section we cover:
- How were processes affected?
- Did productivity change?
- Did throughput change?
- How is coding behavior shifting?
- Are tools enabling collaboration?
- How can you support your team?
Allstacks’ Founder and CEO Hersh Tapadia shared the results of our study and takeaways for you to leverage with your own development team(s).
Insights: Cycle Time Remained Steady
Cycle time remained fairly steady and trended downwards from January to May; this is encouraging because it means individuals and teams were able to work predictably and make progress towards goals.
Insight: More Process Visibility & Consistency
Our study shows that organizations are leaning into processes and being more diligent in making updates within their tracking tools; in this case, we see developers documenting each “transition,” as work status progresses from one stage to next in the lifecycle. The adherence to process updates adds transparency and clarity to overall project status.
Hypothesis: Prior to the remote work environment, it was common for developers to start working on tasks and forget to update their issues in their tracking tools until the end, once the work was completed. With fewer in-person meetings and face-to-face interaction during the workday, employees are more likely to track work on a granular level.
Insight: Increased Developer Collaboration in Delivery Tools
Developers have increased collaboration within their issue tracking tools, like Jira, and are adding more comments within the cards. Individuals and teams are responsible for tracking what they are working on and how it’s going. More comments and discussions within the thread of the issue itself offer better context and clarity, versus using chat or posting comments elsewhere to get clarification.
Hypothesis: Reducing face-to-face communication means more communication needs to take place in the context of the specific issue or work task. It’s much easier to keep all of the information they need in one place, so anyone can follow the thread. Adhering to this process keeps everyone aligned.
Insight: Commit Frequency is Steady but Commits are Smaller
Commit frequency has remained consistent but commit size has changed. See below.
The number of large commits has decreased while the number of small commits has increased. As previously mentioned, the frequency of commits has remained the same, so the increase in small commits is overall a positive change. Small commits are easier to understand when reviewing a pull request and are easier to manage. Big commits are harder to revert; you may only want to revert a portion of the changes versus the entire commit.
Hypothesis: The increase in small commits is likely due to the fact that there are more distractions working remotely.
Insight: Remote Work Schedules Fluctuate vs. In-Office Work Schedules
As discussed in the first part of this series in “People,” work schedules fluctuate and are more volatile compared to in-office work patterns. As a result, cycle time from first commit to opened PR has also has become sporadic.
Hypothesis: This behavior is likely due to the increase in smaller commits done throughout the day.
Insight: More Developer Collaboration During Code Review Than Comments in the Code
The data points to an increase in the number of developer comments in the merged PRs, similar to how we saw an increase in comments at the issue and story level within tracking tools. There’s a lot more discussion in code review and commentary in the code itself, which demonstrates teams’ ability to collaborate efficiently where it makes the most sense for them.
Strategies to support your development team and their processes:
- Find ways to replace face-to-face interactions using your tools, keeping in mind that each individual and/or team has their own work schedule.
- Encourage your team to be detail-oriented to avoid any gaps in communication by using all of the capabilities your tracking tools have to offer. Keeping discussions within the issue threads creates better documentation and keeps teams informed and aligned.
- Focus on areas where outcomes are starting to slip; use this time as an opportunity to see what’s working, what’s not working, and how processes can be improved
Q&A with ClearData VP of Product: Building Better Processes
Adam Dahlgren, Allstacks’ VP of Product goes behind the scenes with Darrell Hyde, the VP of Product at ClearDATA, to dig into what makes their organization's product and engineering functions continuously successful, even as external factors ebb and flow.
A summary of this discussion is below:
Question: What processes and tools have made the biggest impact on the success of your team?
- “We haven’t introduced any new tools as part of going remote across the board. So all the usual names still apply—Allstacks, Zoom, Slack, Confluence, Jira, Gitlab—these platforms keep us moving and collaborating. Reliability and business continuity with tools and services are more critical than ever.”
- “Our quarterly roadmap planning sessions have actually gotten even more focused and productive than before [COVID]. Focus on what’s important to your customers.”
- “If you assume that no process has to relies on two people being in the same room, you don’t build a process around those types of dependencies.”
- “I spend a lot more time looking at team Scrum boards to see where things are going; ultimately, I am looking for impediments. I am a believer that everyone who isn’t’ a developer on a Scrum team, all the way up the ladder to my boss, should be looking at the data to clear impediments.”
Question: Did you make any changes in your operations as a result of COVID-19?
- “More so than ever, we’re very reliant on our data. It helps me feel in control and it's very important to us. When we commit to meeting a feature within a given period of time, that we’re able to do so to fulfill customer needs. I can make sure we’re on track or we can make course corrections early. Forecasting and velocity and data are that much more important. Our customers need us to be at our best to support them.”
- “Now that everyone is remote, it’s imperative to have boundaries of when we start and stop work to find balance.”
Question: In terms of company culture, are there any specific best practices that you can offer others who are cultivating the positive and productive company culture that you’ve achieved?
- “Find ways to replicate the social experience you’d otherwise be missing. For example, I use the same Zoom bridge, which allows us to create ‘bump into you’ time with others in between meetings.”
- "Our CEO has been doing weekly fireside chats to share what we’re hearing from customers. We have regular all-hands meetings. Our mission and focus are to help our customers in healthcare that have a direct, front-line impact during the pandemic.”
Part II - Conclusion
No two companies are the same, so there will not be a one-size-fits-all approach to developing processes that make your team successful. However, we hope that these insights will help you refine your processes as you determine how your team will work for the remainder of 2020. L
Part III - Outcomes
You may be asking yourself if it’s sustainable to run your business this way. The impact of throughput, cycle times, work patterns and more have been summarized throughout this study. This final section discusses how code was released in past months and if delivery was affected by the WFH transition.
In this section we cover:
- How were outcomes affected?
- Is work consistent or more volatile?
- Are people able to focus on planned work?
- Did teams effectively release code?
- How is delivery changing?
- How should you support your team?
Allstacks’ Founder and CEO Hersh Tapadia shared his final thoughts and data from our COVID impact study so you can plan out the remainder of the year.
Insight: Increased Bouncebacks as Developers Adjusted
Allstacks tracks alerts and risks to find anomalies in how your organization typically works. For this report, we studied our population sets to find out how the new circumstances have affected work in progress. Are stories and issues getting stuck? Are they bouncing back to in progress? Are these behaviors any different than what you were experiencing pre-WFH?
Initially, the data shows a decline in the number of bouncebacks, but they increased as we got later into the WFH cycle. There was a corresponding increase in the number of items that got stuck. At the onset of COVID-19, from March 2 to April 6, teams focused on newer items over existing items; eventually, teams saw the need to revisit old work, and there was a shift back to address the backlog. From May onward, balance was restored, as we see work normalize with developers fluctuating back and forth between new and existing backlog work.
Hypothesis: The volatility in engineering bouncebacks from March to April can be explained by the need to revisit requirements for old work; as you can imagine, after a certain period of time, developers need a refresher on what the work item was about and what exactly needs to be done.
Insight: Fewer Distractions Contributed to Higher Focus on Strategic, Planned Work
Pre-work from home there was a lot of variability in working towards unplanned work versus planned work (this is also known as epic-focus or roadmap focus). Unplanned work can come in the form of one-off features or an escaped defect, for instance. Since the transition (March onward), the data shows more consistent focus on planned work, as 40-50% work is planned.
Hypothesis: Progress towards longer-arched tasks can be explained by fewer interruptions throughout the day, where managers are no longer able to “steal time” from developers in between meetings and tasks.
Insight: WFH Had No Impact on Developer Throughput
The data indicates there’s been steady throughput for developers merging code into the codebase. The WFH shift had no change on developers’ ability to push code. Based on the information gathered in part II of this study, organizations have seen success leaning into the tools and processes their organizations already had in place.
Insight: After Initial Slip in Delivery Dates, Normal Delivery Timelines Returned
At the beginning of the COVID-19 transition, there were spikes in project delivery slippage, but delivery returned to normal around April 27; the typical range for slippage is 5-10 days. This implies that the study participants were able to deliver on their respective “normal” outcomes.
It’s great to see that organizations have been able to continue to track towards their company initiatives without significant disruption. Here are a few more suggestions on how to maximize your team outcomes.
Strategies to Support your Desired Outcomes
- Set aside a no-meeting day to help developers focus on planned work without any interruptions.
- Now is a good time to reprioritize the backlog and roadmap rather than resurfacing old tasks without review. Do you really need to revisit those tasks knowing what you know now? Taking a few minutes to assess what you need to move forward with and what can get left behind is invaluable.
- Lean into ceremonies to generate good requirements and strong planning sessions early on to avoid too many course corrections or inefficiencies later on.
- Oftentimes, you schedule a meeting to make a decision, but what happens in between that time? That gap likely means that your team has paused work in the interim; try to reduce this stop-and-go behavior.
A special thanks to all our customers who participated in the study! Stay tuned for updates on “The State of Software Development” later in 2020.
Want more people-centered engineering and product content?
Subscribe to our newsletter to get a monthly email with a collection of our latest and best content.