Estimates in the uncertain world of software development
Challenges and strategies for better estimations
Estimating software development projects is a pretty tough task. Even if we try to anticipate all possible risks and uncertainties, it’s still really hard to meet the initial estimate.
From my experience, the estimates in all the projects I’ve worked on were around 50% accurate. According to studies the accuracy varies from 25% to 75%. Those rare times when “teams” actually managed to meet the estimate, it was usually because this was one person who handled everything — design, architecture, coding, and testing. Pretty rare in our current reality, though. The main issue with estimation is that there are so many dependencies, and each dependency has its own level of uncertainty, making the whole thing unpredictable. Estimation is like trying to bring some predictability into a world that’s basically unpredictable. So, it’s safe to say that meeting the initial estimate is pretty much impossible.
“The only thing we know about the future is that it will be different.” — Peter Drucker
In our R&D team a big part of the work involves doing some research. Sometimes it takes up to 50% of the total time we spend on development. The amount of these research tasks in certain quarters can be like 50% or even more of all the tasks we have. So, basically, we often deal with a lot of uncertainty, but we still manage to stick to the initial estimates. There are some observations which I would like to share with you.
There are so many conditions to consider that it’s basically impossible to estimate tasks in general. This means that any task will always take different time (usually longer) to finish than we expect. That’s the only thing we can be absolutely certain about. Although, sometimes we do get lucky and can make an educated guess, but that’s more of a rare occurrence rather than something that happens all the time.
In general, it’s not possible to fit in the estimate, but there are many special cases.
Why is it impossible to estimate accurately?
We are optimistic by nature.
Developers are super optimistic, no matter their experience or whether they’ve included all the extra stuff like testing, communication, and research in the estimate. We just tend to believe everything will go smoothly. Plus, we’re under pressure from the business to get things done faster and cheaper, which makes sense in its own way.
So, we often end up underestimating the time by half, simply because we’re so confident in our abilities. As we dive into the project, the estimates become more realistic, but the optimism never goes away.
Another cause is…
This is simply impossible.
I want to give an example from mathematics and astronomy that, in my opinion, illustrates the problem well.
In astronomy, there is a problem known as the three-body problem. Its goal is to determine the relative positions of space objects at a specific moment in time, taking into account the mutual gravitational force between them. This problem is well-solved in the case of two bodies, but becomes unsolvable in the general case when a third object is added. In simpler terms, it is impossible to predict the positions of three objects relative to each other at a future moment in time. However, there have been numerous special cases where solutions were found. I believe this problem shares many similarities with estimations. In contrast to the tasks in astronomy, we encounter different forces between objects and we have more than just three bodies to consider.
Let’s consider a small example. We have a task to develop a web chat application. To make the example closer to reality, let’s assume that the server and frontend are developed by different teams. Additionally, we have localizations with translations and various geo-settings. If we try to express this mathematically, it would result in a formula where each command and dependency represents an unknown variable. In the end, it would look something like this:
Where A is the backend team, B is the frontend team, C is the localization and translation team, and D is the geo-settings team. Each team has its own variables representing their internal things, which makes the formula more complex:
Also, additional weights could be added to express the importance of these internal things.
And so on. Another factor is the communication between teams, which is also unknown. As a result, there is a large formula with a lot of unknowns. Therefore, it is not surprising that the estimates are not accurate.
So, if everything is so unpredictable and difficult, why do we engage in estimation? Why don’t we just put some random numbers if, as a result of all our efforts in estimation, their quality is far from expected and, more precisely, it is not possible to give a correct estimate in principle? Let’s see what exactly we want to achieve in the estimation process.
What is the actual task of estimation?
Given all of the above, it is clear that solving the problem of estimation is not just about using techniques. Understanding the concepts behind them, specifically what we want to achieve, is important. If you think the goal is to guess the number of hours, then that is a superficial understanding of the problem.
The unknowns in the project are the dependencies and risks. I have concluded that when it comes to estimation, a more important task than determining the number of hours is actually identifying the quantity of these unknowns during the analysis phase. This greatly impacts the quality of the estimation. We can better understand the importance and value of the unknown if we know it exists. However, our problem often lies in the fact that the quantity of unknowns remains unclear until later.
Therefore, the main problem and task in estimation is to determine the list of these unknowns.
Now let’s move on to specific recommendations.
How to give better estimates?
All the methods we use for task estimation, I have divided into three groups and called them visualization methods. Imagine a task from those you have. Try to imagine it already completed and functioning. If you have all the details in your head and a clear understanding of this feature from A to Z, then congratulations: you have a good foundation for getting an accurate estimate. But if the feature, to some extent, works but there are moments that are unclear or covered in fog because your team is not dealing with them, then there is room for improvement.
The first group of actions that will help in estimation is visualization of value.
Visualization of value.
This requires a good understanding of why what we want to do is important and beneficial, and how it will contribute to creating business value. It’s important to avoid uncertain or vague statements, like saying “maybe” or “possibly will work”. Every task should have a clear reason for being done. If there’s no clear reason or the reason isn’t well-defined, there’s a high chance of wasting time for everyone involved.
When you visualize, it’s important to ask questions that help you understand the value of the planned work and how it will be produced. But think about the balance. If everyone starts clarifying the value on a company-wide scale, we’ll end up getting stuck in discussions and explanations. However, understanding how the value is created for the company, starting from what we’re doing, helps us see dependencies that we might have missed otherwise.
For example, ask questions such as:
- Why is the feature important?
- What will happen to the business if the feature is not implemented?
- How does the feature specifically contribute to the business value?
- Which parts of work could be eliminated while maintaining the same value?
Therefore, as the person submitting the task, you should be ready to answer questions that challenge the reasoning behind your goals. Asking questions is helpful in general. We might feel confident about a solution simply because it’s ours, even though it could be wrong. That’s why questions are beneficial, and it’s good to have a lot of them. If the task submitter can’t give clear answers to the questions, it becomes a risk for estimation. Maybe everything will change as we go along?
You can follow steps such as the ones below:
- Clarify the business cases by drawing flowcharts of business usage scenarios, indicating the roles and outcomes for each important component of the feature.
- Share the schemas with every team involved in the implementation.
- Discuss with each team their expected business scenarios, including the inputs they will receive and the outputs they are expected to provide, on a business level.
- Share a brief summary of the outcomes from these clarifications with the group of stakeholders via email or different written form.
Visualization of the process.
The next set of techniques is about visualizing how the solution works. We try to imagine what the solution will look like when it’s finished. This helps us understand the processes involved and how they connect with other systems and teams. This allows us to break down a big task into smaller ones. It’s not about solving the task right now, but understanding the main things that need to happen. For example, we may not know the details of a specific report yet, but we know we need to check data availability, make sure the report can be delivered, and gather samples. Each task may have different expected results. During visualization, we don’t solve everything, but we identify them, ask questions, and imagine how the final product or work should look. We may also set milestones to track progress at different stages.
To visualize the process, the following questions can be helpful. These questions should be discussed from a functionality standpoint, rather than focusing on technical implementation:
- What does the finished and working feature exactly look like?
- How should information flow between different components and what information should be included?
- What is the expected behavior in case of a malfunctioning feature?
There are several procedures that can help answer these and similar questions:
- Conduct backlog grooming meetings with representatives from developer teams.
- Decompose the entire work into independent tasks.
- Plan the implementation of tasks within development teams by discussing the specific details of how the expected functionality will be implemented.
Visualization of progress.
And so we smoothly approach another important group: visualization of progress and work.
To see our own results is important for an individual. But if the result is delayed because the task is big, then the only way to observe the progress is by looking at where we were and where we are now on a smaller scale. This creates a feeling of movement. Without glorifying the process over the outcome, we track the progress of each task during regular meetings. Each team member is asked to individually assess how much effort has already been spent and how much is still required to complete the current work. With this exercise, everyone in the team gets a sense of progress and understanding of where we are as a team.
As an additional benefit of all we do in previous visualization techniques, we gain some understanding of the approximate main steps of working on a solution. These steps are given dates in our internal implementation plan. Stakeholders may have their big, long timeline, but we may have a more detailed timeline with shorter periods. Literally, we internally commit that certain things should be ready at a certain date. We can set checkpoints. If they are not ready by the specified date, we can already react to this and understand the reason, and take mitigating action if possible.
To visualize the progress, we should take several steps while implementing something. This is not just limited to activities performed before starting the implementation. The following measures can help achieve good progress visibility:
- Plan and schedule internal checkpoints to track the actual progress of the implementation.
- Communicate any doubts about estimates as soon as internal checkpoints are overdue.
- Avoid committing to estimates under pressure from senior personnel if they lack justification in the planned work.
Conclusion
In the article, I discussed a very broad topic that has many different implementations. Just like the aforementioned three-body problem, there are numerous special cases where the problem can be solved, but there is no generic solution. Therefore, it is completely normal for some people to have no issues with estimations while others struggle greatly with this task. On the internet, you can find many thoughts on how to improve estimations, although most of them simply highlight the complications that come with this challenging task. If you would like to learn more about this topic, I can provide you with some initial articles that can serve as your starting point on this long journey:
