Project Cost Estimates
What patterns and, more importantly, specific tips have we got for ProjectCostEstimates?
Background
Year is 2004. c2 is known to be a good resource in practical patterns. Therefore I started searching this site for TaskEstimationPatterns, to get insights to do ProjectCostEstimates. I searched for Cost or Estimate in the PageName, and I long to hear more current information from experienced practioners here.
Having no access to the revered MythicalManMonth, I am feeling ItProjects are inherently DeathMarches, and ultimately relying on RelationshipManagement to rescue the day.
Implication of the validity of the last statement above is huge. It means the ability to be unconditionally submissive to the whims of IT management, especially those who excel in RelationshipManagement, is the highest requirement amongst CriticalItSurvivalSkills.
Over the longer term though, the developers suffer because invariably these managers give IT industry a bad name, by overcommitting and underestimating. At times they cost a drain on organizations' resources (e.g. the YearTwoThousand hype resulted in short-lived IT boom and a protracted IT depression afterwards.)
So how can an IT technical person take charge of ProjectCostEstimates, and therefore take charge of their own destiny?
Discussions
Programmers are bad at giving accurate estimates because of a number of reasons:
- They produce buggy code. But bugs are sometimes hard to detect, so some bugs go unnoticed for years. The ones that are detected earlier are not fixed but worked around. Is this a GoodThing or not? I've seen many shops who really don't care how many bugs they have, since it just means that the customer will have to call and sign a new check. This is a very shortsighted way to manage a company, the company can't grow very fast, but it seems to allow for survival, which would be good in economic downturns.
- Their estimates are always challenged by managers, it doesn't matter if they say 1 hour, 1 day, 1 week, 1 month, the schedule is always cut in half. By the way, if they say 1 year, they are fired. Some shops really believe they are doing the right thing even though they are almost bankrrupt. No experienced developer is willing to work for them. That's a clear sign that this is not a nice place to work at.
- Developers are never allowed to estimate. Estimates come from management and developers must simply comply. Managers reduce estimates or simply add more tasks without allowing extra time for the new tasks. Of course this means developers must work mandatory unpaid overtime and they become sick and they are supposed to go to work anyway, since there is no contract and no work means no pay.
- GopherHoles, alleviate with lots of short estimates, part of the reasoning behind why EstimatesLongerThanThreeDaysConsideredHarmful
See NegotiateEstimates.
The aforementioned four reasons that developers cannot give accurate estimates strikes me as a bit troublesome; the second and third (estimates are always challenged, management dictates estimates) are not reasons why accurate estimates are not given. They are not, strictly speaking, reasons that developers cannot give good estimates.
Perhaps this is an uncommon trait, but the senior developers at my previous company were actually quite good at giving estimates. Management would force estimates down ala conditions two and three, but the initial estimates were met with alarming consistency. If one estimated four weeks, the project took almost exactly four weeks, with interruptions and other warts. The fact that management asserted a two week estimate is not a valid criticism of the developers.
Conditions 2 & 3 prevent discourage accurate estimate tracking and feedback, preventing developers learning how to estimate better. In fact, Condition 1 is actually not a complete picture - the presence of bugs not found for years does not mean the estimate was inaccurate, since the program ran for a long time with apparent success. Bugs that prevent expected function matter - edge cases that don't hit are a different problem.
- Moving targets - the difference between the rough initial requirements when estimated, the finalized requirements at the time the project begins, the actual requirements as discovered during the course of the project, and the revised requirements not discovered until testing or deployment. Changing requirements should have corresponding changes to the estimates, but that doesn't jive with people's perception of the initial estimate as one of the fixed parameters and key metrics of the project.
So when estimating, you have to estimate not only the requirements that you're looking at, but enough to cover all possible extensions and variations thereof (including the unforeseen and unpredictable) that may arise, and redoing everything a time or two to make those changes. This results in unacceptably high estimates. (Also violates YAGNI...or does it?)
- Unspecified assumptions - everyone involved has their own vision of what a requirement means, and things that are clearly obvious go unspoken. Until later, when each discovers that what was obvious or implied to them was quite different from what another person was thinking. The person asking for an estimate just wants a number, but it needs pages of delineated definitions, assumptions, and clarifications to have any actual valid meaning.
This requires lengthy planning and brainstorming involving multiple people who think about the project from different angles. A single person can't just look at something and give an estimate. This makes estimating unacceptably expensive and slow. (Violates DTSTTCPW... if you think that a single POV with little communication can produce an estimate that works)