How to Estimate Work
Source: https://vadimkravcenko.com/shorts/project-estimates/
Software Development Estimations
First, I planned on calling this article âMastering the Art of Estimations: A Definitive Guide for Developersâ, but then who am I to tell you how to estimate projects? I can only give you some pointers and describe some things that worked well for me over the years. So thatâs exactly what I will do â give you some rules of thumb to make your life easier.
Ah, software estimation - the bane of many developersâ existence and the scourge of project managers everywhere. A constant battle of âtell me how much itâs going to take youâ and âgive me a clear description of the task first.â I'm here to bring some much-needed reality of the complexities of real-world development. Iâve had my share of estimations on both sides of the fence. Iâve seen it all. And I'm here to give you the harsh truths about software estimation, so buckle up.
And to you junior developers out there, pay close attention - this stuff will be the difference between a successful career and a lifetime of missed deadlines and frustrated coworkers.
Before we dive, a story from my personal experience. This was back in 2018 or 2019 when my business partner approached me and asked, âHow much time do you think you need to build a mobile bank?â. At this point, I almost spilled my coffee, but I took a moment to gather my thoughts before I answered. âWhat do you mean a bank? Iâve never built a banking app. How should I know?â He said, âWell, just give me a ballpark based on your gut feeling; I need some timeline to sell itâ. At this moment, I wanted to list him all the ways it could go wrong if we estimate based on such an absurd requirement. Still, I thought it might be a good challenge to see how good my gut feeling was.
I told him it would take me at least a year with a small team of 6-9 people. âOf course, we can refine the estimate as soon as you give me more details, but this should give you a timeline you can work with.â He said, âOkay.â
Eventually, we did build a mobile banking app â we spent several months on the initial designs, which were perpetually getting refined during the development phase. The app coding took ten months, a team of 7 people, and much scope-cutting to get to that first AppStore release. Even with the shortcuts we took (e.g., building with someone who already has a banking license instead of getting our own), we still barely made it, and we had a long list of features after that initial release that we needed to deliver.
So we can say my estimate was right, or better yet â the team made the initial product fit the estimate by cutting scope.
đĄ So we can say my estimate was right, or better yet â the team made the initial product fit the estimate by cutting scope.
Iâve also been wrong with my estimates many, many times before. I remember grossly overestimating the development of some components for custom CMS. My team and I communicated that delivering the feature would take 3-4 weeks, but we were done in a week and a half. Youâd think this is a good thing, and everybody was happy, but no, in the agency business, this makes the client believe that youâre inflating features to get more money, leading to you losing their trust in your expertise.
I also remember underestimating projects, which is quite dangerous if youâre an agency and doing fixed-price projects (which I try to do less and less). There was a project that I was pretty confident in implementing in the agreed time. Still, after we started digging, we found out the feature required us to do a major upgrade of some packages, which resulted in a cascading complexity in other places that needed fixes. It took us twice the time agreed upon to finish the project, but hey, we upgraded the underlying software, learned a good lesson, and made the client happy.
So letâs get down to the dirty business of estimations.
Why Do Estimations at All?â
After reading my stories, you might wonder, "Why bother with estimations? Aren't they just a pain in the ass?" Sure, they can be. But estimations serve a crucial purpose in the realm of software development. Without them, no software project would be green-lighted. Even if itâs an R&D Project â someone at the top needs to see if the company can invest that much money and for how long, which requires some numeric value attached to a project. We'd be living in a chaotic world without estimates â with uncertainty, over-commitment, and missed deadlines.
Here are some reasons why we need estimations in the world of software development:
- Resource Allocation:Â Estimations help prevent under- or over-commitment of resources. You don't want to leave your team stranded in a sea of unfinished tasks, nor do you want them twiddling their thumbs because you thought it'd take six months to build a simple feature. Get your estimations right, and you'll have a well-oiled machine of a development team.
- Collaboration:Â Estimations create a shared understanding of the project size among team members and stakeholders. When everyone's on the same page about what needs to be done and when it's due, collaboration is more effective, and your team can focus on what they do best - writing kickass code.
- Sanity Check:Â Estimations help you, as an engineering manager or startup founder, to maintain a semblance of sanity in the face of constant deadlines and deliverables. By knowing how long tasks should take, you can make informed decisions about priorities, allocate resources effectively, and maybe even get some sleep at night. A proper Gantt chart of deliverables can alleviate stress on the weekend.
So we agree that Estimates are a necessary evil. Letâs explore the good, the bad, and the ugly, so you can learn from the mistakes of those who came before you and emerge a better, wiser developer. I hope.
Estimations in Software Projectsâ
Estimations in software projects are about as sure as your favorite bar's closing time during a pandemic. We must acknowledge the inherent uncertainty and work to mitigate it. In my example of the banking app estimates - itâs a gut feeling imagining a team working on delivering and how I would go about building it, which modules first, which second, and what roadblocks I can imagine. Of course, you canât account for thousands of unknowns, so the estimates should be on the safe side first and get shorter over time, not the other way around.
Itâs much better to tell someone, âOkay, after a week of exploration, we can say that the timeline is 12 months instead of the initial 24 months suggested.â rather then "Ooops, the scope got huge and it's now 48 months instead of 24"
đ¨ Itâs much better to tell someone, âOkay, after a week of exploration, we can say that the timeline is 12 months instead of the initial 24 months that we estimated.â rather then "Ooops, the scope got much bigger and it's now 48 months instead of 24."
Here are some rules of thumb to follow when estimating in the real world:
No two projects are the same:Â Sure, there may be similarities, but there's always a twist that'll throw a wrench in your well-laid plans. The key is to learn from past experiences and apply that knowledge to future projects. If youâve built a chat system before, it might give you some insights into the next one you need to implement. Have experience integrating the Stripe payment gateway? Well, you might transfer that experience onto the subsequent Braintree integration that is requested.
Uncertainty is your constant companion:Â Requirements change, stakeholders change their minds, and unforeseen challenges pop up like a bad case of acne. The best you can do is remain calm and adaptable and ensure your estimations have some buffer for the âhuman factor.â I usually add 20-25% on top of the estimates for things like people getting sick, third-party vendors being slow to respond, bureaucracy, and misunderstandings.
The cone of uncertainty in software projects (McConnell 2008)
Communication is king:Â Many estimation issues stem from poor communication between developers, managers, and other stakeholders. Keep the lines of communication open and clear, and you'll save yourself a world of pain. Remember â itâs better to over-communicate than to expect people to communicate with you first. Need something from someone until a deadline? Annoy them with friendly reminders. Afraid someone would misunderstand your needs? Highlight and emphasize each requirement several times.
Most of these lessons came from my experience and missed deadlines, which is always an annoying event.
Wrong Ways to Do Estimationsâ
Itâs better to learn from the mistakes of others. Here are some misguided approaches to software estimation that you should avoid like a tequila shot on an empty stomach:
The blind guess:Â Plucking numbers out of thin air is a surefire way to disappoint everyone involved. Base your estimations on data, past experiences, and input from your team, not wishful thinking. We have this running joke in the company that every time someone comes to me and asks for an estimate, I always say six months, regardless of the project, which always triggers a follow-up discussion as thatâs either too low or too long.
The "one size fits all" approach:Â Assuming that every task will take the same amount of time is a recipe for disaster. Different tasks have different complexities and require different skill sets. Consider these factors when estimating. Building an E-commerce shop does sound easy initially, and Iâm sure youâve built many before. But this task can take somewhere between a month and many years, depending on a huge list of factors of Inventory Management, Warehouse Integrations, ERP Integration, Ordering Flows, Marketing automation, etc.
đĄ Itâs essential to understand what the stakeholder means when they say, âI want a Shop,â as what you think and what they think can be very, very different things.
Ignoring external factors:Â Don't forget to account for meetings, code reviews, and other non-coding activities that will inevitably eat into your team's time. Failing to do so will leave you with overly optimistic estimations and a whole lot of explaining to do. Iâve known many developers who, when asked for an estimate, think of the raw coding time only. That. Is. Never. The. Case. Rule of thumb, 1.5x-2x your raw coding time estimation to get a good approximation of the real-time (incl. communication) you will spend on a task.
The fixed mindset:Â If you think your initial estimate is set in stone, you're setting yourself up for failure. Be open to revising your forecasts as new information becomes available and the project progresses. The cycle is as follows: 1. Work on the task happens 2. New information comes to light 3. Estimations are improved and communicated 4. Go back to 1.
Now that we've covered the usual pitfalls let's move on to the proper ways to estimate, so you can go forth and conquer the wild world of software development with confidence and panache.
Proper Ways to Estimateâ
With a clear understanding of what not to do, let's delve into some effective practices that help with software estimation.
However, my system does have one critical characteristic that I believe any effective estimation technique should have: it captures both time and uncertainty. An estimate that only includes time implies a high degree of certainty: if you tell me âthat will take 10 daysâ, and we have a deadline 14 days out, Iâll assume weâre in great shape. If, on the other hand, we have that same deadline but you tell me âthat will take 10-15 daysâ, now I know our deadline is at risk.
[..]
Thus, the next step is to capture that uncertainty. Initially, you might think of capturing a best-case and worst-case scenario, but I donât find that too useful. Instead, I prefer to capture expected-case and worst-case. Coming in early is never a problem, and in my experience, the best-case rarely happens. But it is important to capture how long something could take if things go poorly. Therefore, my uncertainty system starts with the expected time (captured above), and then applies an âif-things-go-wrongâ multiplier:A good estimation method that's based on uncertainty
These approaches are primarily subjective, but they helped me and hopefully will help you and your team navigate the uncertainties of development with confidence:
Break tasks down into manageable units:Â Large tasks can be overwhelming and difficult to estimate accurately. By breaking tasks into smaller, more manageable units, you can better assess the effort and time required for each component. You can imagine this as a recursive function: Input is the product you want to build. Can you build it in two weeks? No? Break it down into two components and call the same function on the two new components; if yes, return a Numeric value of two weeks, then sum up all the numeric values.
Use historical data and experience:Â Draw from your past experiences and projects to inform your estimations. Consider how long similar tasks have taken in the past, and use this information as a foundation for your estimates. Thereâs no way around it. As much as you learn from this article, you will have to overestimate/underestimate/fail a few dozen projects before you get enough experience.
Employ estimation techniques:Â Familiarize yourself with estimation techniques, such as Planning Poker or Wideband Delphi, which can help your team reach a consensus. If you have the time, especially in a big corporation, you can use those techniques to greatly increase the estimate accuracy. These techniques encourage collaboration and communication, ensuring everyone's input is considered. I mostly prefer planning poker for its efficiency and fast pace, but itâs a taste thing, and there are tens of different methods that you can look up on the internet. They rely on multiple people with different domain expertise to figure out how they would build it and then have a follow-up discussion with other experts.
Chapter 23 Estimation Software Engineering: A Practitionerâs Approach 6th Edition
Roger S. Pressman.
Factor in uncertainty with ranged estimates:Â Instead of providing a single estimate, consider offering a range that reflects the degree of uncertainty involved in the task. This will help set realistic expectations and allow flexibility as the project evolves. Though sometimes people only hear the lower band, so you must explicitly mention that itâs a range and that âAt least a yearâ does not mean âIt will take us exactly a year.â
Seek out mentorship and guidance:Â Learn from more experienced developers (youâre already here, so good on you!) who have honed their estimation skills over time. Watch YouTube videos of how Spotify, Google, Meta, and Shopify built their stacks. Read case studies on how others have implemented modules that interest you. Read how Redis was built, find out how Databases are made, etc. These insights and advice can help you avoid common pitfalls. There are so many valuable things you can find on the internet.
Donât get influenced by authority:Â If your estimate is five weeks but your manager/senior dev/someone else wants it to be 3 weeks â stick to your guns. Unless there are objective reasons to refine your estimate, thereâs absolutely no good outcome if you give in to some persuasion to lower it.
Remember, no one is born a master estimator. Becoming proficient at predicting project timelines takes time, experience, and continuous learning. So, be patient with yourself and keep striving for improvement.
Refining Estimatesâ
As we discussed above, estimates are constantly evolving numbers; if youâre lucky, they always go down. If youâre unlucky, they go up, but hopefully, they trend long-term downwards. Estimations are not made to be static; they need to be updated as the project progresses and new information becomes available.
Here's how to effectively take care of your estimates throughout the development process:
Track progress and compare to initial estimates: Regularly assess your team's progress and compare it to the initial estimations. This will help identify any discrepancies and allow you to address them promptly. Be honest and prompt if you think something is awry or not going as planned. Thereâs this excellent clip for Silicon Valley where Nobody Knows the Real Timeline , which you should try to avoid. See a problem â tell it now rather than later.
Revisit estimations during project milestones:Â As you reach significant milestones, take the opportunity to reevaluate your estimations. This can help you identify any changes in scope, new requirements, or other factors that may impact the project timeline. Milestones = Big things that usually get presented to the management, and the next portion of the budget gets approved. Itâs a crucial time to voice concerns for the next phase so everyone is aware if thereâs a delay. If you know that a third-party vendor will not finish the required integration until the deadline, itâs better to voice it and get an extra budget and/or timeline extension during the milestone meeting.
Communicate changes to stakeholders:Â If your estimations need to be adjusted, ensure all stakeholders are informed and on the same page. Transparent communication is vital in managing expectations and maintaining trust. I had a few cases where a developer did not communicate that he needed assistance and could not finish the task alone right until the deadline. They were pushing all their brain power to solve the job, and it was more important for them to solve the task ALONE rather than keep the overall project timeline intact. This is a no-go approach. If you feel overwhelmed and your estimates do not represent reality â communicate transparently and honestly.
The goal is to lead the overall project to a successful outcome and satisfied stakeholders.
Helping Your Manager Plan Betterâ
At the end of the day, we're all in this together, and helping your manager plan more effectively is a crucial part of making your team's projects successful. You can think of your manager as a spider trying to process information from the different webs they have put in and making sure that none of them snap.
Here are some ways to assist your manager in their planning efforts:
- Be proactive in communication:Â I know Iâve said this a couple of times already, but I will repeat it. Don't wait for your manager to come to you for updates. Instead, take the initiative and share your progress, any potential roadblocks, and changes in estimates. This will help them stay informed and make better decisions long term.
- Share your expertise and insights:Â Your knowledge as a developer is invaluable in project planning. Share it. Help your manager understand the technical aspects of the project, and donât get angry when they donât know why your C++ compile error is blocking you from delivering on the timeline.
- Collaborate on risk management:Â If you have doubts â speak up. Work with your manager to identify potential risks and develop mitigation strategies. Think of worst-case scenarios, and prepare for them. This can help prevent surprises down the line and keep the project on track.
- Be flexible and adaptable. There is nothing to say here, but when your manager is doing the âNeo dodging the bulletsâ thing to keep the project on track, it would be decent to at least not be rigid and open to alternative ways.
I'm sure your manager will be thankful for the support and the team-spirit.
Work Culture Affects Estimations Accuracyâ
The environment you work in plays a significant role in how accurate your estimations can be. A supportive work culture that values collaboration, communication, and learning from mistakes will enable you and your team to provide better estimates.
If youâre already in a position where you can impact the culture at your company, here are a few examples that can make a difference:
- Encouragement to ask questions and clarify requirements:Â A positive work environment encourages team members to ask questions and seek clarification. Stupid questions are allowed, and even if you have to repeat the requirements a few times, it shouldnât be seen as degrading to clarify ALL THE SMALL DETAILS.
- Emphasis on continuous improvement:Â A culture that values learning and improvement fosters an atmosphere where team members are motivated to refine their estimation skills over time. Failing is normal across many industries where experimentation is encouraged. You should normalize that and ensure people donât fail twice with the same mistake â documenting the experiences and post-mortems.
- Openness to experimentation and adaptation:Â A flexible work culture that embraces change and is open to trying new techniques or methodologies can help the team discover more effective ways to estimate tasks and manage projects. Our industry is moving forward at a fast pace, and new tools, methodologies, and paradigms are being made public every year that make our lives better. Donât be against change â embrace it. Evolve.
- Support for collaboration and teamwork:Â A work environment that promotes collaboration across all seniority levels can lead to more accurate estimates as team members share their expertise, helping each other improve. Even interns/junior developers should share their thoughts on the estimates â this allows them to learn from their seniors and try their hand at imagining how they would build this specific task. More experience = better estimates in the future.
Advice going forwardâ
I hope that at this point, we all agree that accurate task estimation in software development is a critical skill that can significantly impact project success. As a software engineer, it's essential to recognize the nuances of real-world development and understand that itâs not always about the accuracy but more about some relative value that helps secure the budget and deliver a project in a limited time frame.
Here's some TL;DR; to help you navigate:
- Always be estimating.
- Youâll be wrong most of the time until you get good. And then youâll still be wrong sometimes.
- Group estimates are more accurate than your single estimate.
- Embrace a growth mindset and be open to feedback and continuous improvement.
- Communicate effectively with your manager and collaborate with your team to ensure everyone is on the same page.
- Be adaptable and willing to experiment with new techniques and methodologies.
- Consider the impact of work culture on estimation accuracy and seek out environments that support your growth.
Remember, no one is perfect when it comes to task estimation. It's a skill that takes time and experience to develop. By remaining dedicated to your growth and learning, you'll become a valuable asset to your team and contribute to the success of your projects. So, keep at it, and never stop striving for excellence.