Posts Tagged Estimation
This review covers Software Estimation by Steve McConnell (Microsoft Press, 2006).
[T]he typical software organization is not struggling to improve its estimates from ±10% to ±5% accuracy. They typical software organization is struggling to avoid estimates that are incorrect by 100% or more (p. xv).
I wrote this book for developers, leads, testers, and managers who need to create estimates occasionally as one of their many job responsibilities. I believe that most practitioners want to improve the accuracy of their estimates but don’t have the time to obtain a Ph.D. in software estimation…If you are in this category, this book was written for you (p. xvi).
The first five chapters introduce some basic concepts:
- Chapter 1, “What is an Estimate?”: You can get into some confusing situations when some of you are talking about an effort estimate and others are talking about a plan to hit a target. This chapter explains the difference between estimates, targets, and commitments.
- Chapter 2, “How Good an Estimator Are You?” presents a quiz that helped me see that “90% confident” wasn’t what I thought it was!
- Chapter 3, “The Value of Accurate Estimates”, goes to some length to demonstrate the value of knowing the size of the project you’re embarking on, and the great cost to an organization that thinks a project will be significantly smaller than it turns out to be.
- Chapters 4 introduces the four general answers to the question “Where Does Estimation Error Come From?” In this chapter was a helpful and thought-provoking flow of questions beginning with these:
- When telephone numbers are entered, will the customer want a Telephone Number Checker to check whether the numbers are valid?
- If the customer wants the Telephone Number Checker, will the customer want the cheap or expensive version of the Telephone Number Checker? (There are typically 2-hour, 2-day, and 2-week versions of any particular feature — for example, U.S.-only versus international phone numbers.) (p.34)
This chapter also introduces the concept of the Cone of Uncertainty, where the maximum precision of an estimate is impaired by not knowing what the project consists of. To get to a more accurate picture of how big the project is, you have to narrow that uncertainty. McConnell explains, “The Cone of Uncertainty doesn’t narrow itself. You narrow the Cone by making decisions that remove sources of variability from the project” (p. 39).
- Chapter 5, “Estimate Influences”, introduces the concept of diseconomy of scale and discusses personnel and other factors that influence the amount of effort that will be required to complete a project.
I was going to stop the chapter summaries after the first five, but some of the next chapters have such good meat in them too…
- Chapter 7, “Count, Compute Judge”, introduces the concept that counting yields more accurate results than computing a number and computing yields more accurate results than expert judgment. This theme comes up again other places later in the book.
- Chapter 8, “Calibration and Historical Data”, introduces the importance of collecting historical data on your projects and gives details, guidance on what data to collect, and pitfalls to avoid. Historical data is another theme that comes up many times throughout the rest of the book.
- Chapter 9, “Individual Expert Judgment”, offers a standard formula to use when you perform expert judgment estimation, so that you factor in best case, worst case, and expected case estimates with proper weight
Chapter 10 covers another quick and helpful technique — Decomposition (and its brother, Recomposition); Chapter 15, “Use of Multiple Approaches”, gives an example of using a combination of decomposition and expert judgment (p. 166) — a combination I have found helpful in my own estimating.
Chapters 18 through 20 cover special issues in estimating size, effort, and schedule, respectively. Chapter 21 (p. 245) introduces a way of dealing with schedule risks (things that could come up and delay the project) by weighting them by the amount of time they would delay the schedule, times their probability of occurring, yielding a “Risk Exposure” (so for example a two-week potential delay that is 25% likely would have a 0.5-week Risk Exposure. Using this concept you can figure what kind of contingency buffer your project is probably going to need.
Chapter 21, “Estimate Presentation Styles”, has several helpful ideas in it, such as an estimate with risk quantification, for example “+1 month if the graphics-formatting subsystem is delivered later than planned” (p.252), and “Don’t try to express a commitment as a range” (p.257).
Chapter 23 is gold. McConnell talks here about people stuff — why negotiations between technical staff and executives tend to be difficult for technical staff to navigate. “Estimate negotiations tend to be between introverted technical staff and seasoned professional negotiators” (p. 259). McConnell then gives some help on how to engage in win-win negotiation with executives: keeping the best interests of the company in mind, positive assertiveness, taking schedule requirements seriously, and maintaining the courage to tell the truth.
A couple of quotes:
You hold the key to a vault of technical knowledge, and that puts the responsibility for generating creative solutions more on your shoulders than on anyone else’s. It’s your role to propose the full range of possibilities and tradeoffs (p. 266).
Avoid saying, “No, I can’t do that”; instead, redirect the discussion toward what you can do. The more options you generate that support doing what’s best for the organization, the easier it will be to show that you’re on the same side of the table as the person you’re problem solving with (p.268).
These ideas come up time and again throughout the later chapters
- The value of historical data from your own company’s projects for accurate estimates
- Prefer counting over computing and computing over expert judgment
- Using a range to express uncertainty in an estimate
- Taking care not to mislead by presenting a level of precision in your estimate that does not reflect the true precision. (For instance, if your estimation techniques produce an estimate of 573.275 hours, you would probably want to present that as 575 or 600 hours)
- The limits of estimation
Chapter 20 clarifies the purpose and limits of a schedule estimate:
The purpose of the schedule estimates in this chapter is not to predict your final schedule to the day but to provide a sanity check on your schedule-related plans. Once you’ve used schedule estimation to ensure that your plans are reasonable, detailed planning considerations (such as who is available when) will take precedence over the initial schedule estimation described (pp. 230-231).
And again in Chapter 21:
Recognize that these allocations of effort to phases are approximate. They represent useful starting points for planning. Once the estimates get you into the right planning ballpark, detailed planning considerations should take precedence over these initial estimates (p. 237).
The book covers both iterative and sequential project styles.
A Peril Averted
As I read in the first few chapters about the dangers of corporate wishful thinking regarding estimation and scheduling and began to dig into the smorgasbord of techniques that are available to get a picture of the size of a project, my pride began to swell and my attitude began to sour. Chapter 23, “Politics, Negotiation, and Problem Solving”, was like an unexpected bucket of cold water on the head that snapped me out of my arrogant stupor and really helped to put the lessons of the book into proper perspective. My recommendation: don’t skip this chapter!
It’s small (270 pages of main text) and a quick read that makes a wealth of study accessible — McConnell pulls from many sources to give organizations the tools they need to get a lot better at project planning, without having to take a lot of time to get up to speed on what he calls the “science of estimation”. Well worth the small investment of time and money.
Some co-workers and I recently finished Software Estimation by Steve McConnell (Microsoft Press, 2006).
In Chapter 3, “Value of Accurate Estimates”, the book contains a graph of the extra cost of underestimating vs. the cost of overestimating, something like this:
On the left side, the graph pictures a nonlinear increased costs of underestimating (you don’t schedule enough time for the work; more errors introduced, more status meetings, those types of things). On the right side, the graph pictures a linear increased cost of overestimating: you pay your employees longer before the project is done (Parkinson’s Law). McConnell says “In software, the penalty for overestimation is linear and bounded…But the penalty for underestimation is nonlinear and unbounded“(p. 24).
As I look at it now, I believe that when I looked at McConnell’s graph I failed to distinguish in my mind between the cost of an incorrect estimate of effort and the cost of a too-aggressive or too-leisurely schedule. McConnell is contending that there is great value in having accurate estimates, and great cost to an organization that does not realize the true size of a project it undertakes. What he did not say (but in my mind I extended his statements to say) was that a larger effort estimate translates linearly to a larger project timeline. In other words, in my mind instead of saying
Don’t underestimate your project’s size
I read it as
Don’t go for aggressive project schedules
Again, McConnell did not say the latter, but I inferred it. Let me explain why I think I erred in doing so.
Why An Aggressive Schedule?
There are big costs to an organization that doesn’t develop at a fast enough pace. Here are a couple I can think of:
- The cost of lost business due to your competitors getting ahead of you. This would take the form of lost sales now (the product isn’t ready so your prospect chooses a different solution) as well as lost future sales (if the prospect had chosen you now and were happy with you over time, they would tend to purchase more licenses, buy ancillary offerings, etc.)
- The cost of losing your market leadership position. How valuable is it to a company to be the clear leader among its competitors? Once you lose that status, it must take great effort, possibly over a period of years, to gain it back.
These costs are nonlinear and unbounded as well. Suppose we could estimate the strategic cost to the organization in addition to the technical cost. We might get a cost graph like this, with the schedule increasingly aggressive toward the left and increasingly leisurely toward the right:
(The red is the increasing strategic cost to the organization of pushing the date out.)
This graph assumes that the project schedule corresponds linearly to effort, which is pretty simplistic (there is a whole project management discipline that deals with how you coordinate effort to meet a target date, and project management can be done with varying degrees of excellence or not, which would affect the mapping of effort to schedule)…
There are many costs associated with trying to get the project done on an aggressive schedule — but here we see pictured the potential costs to the organization of not getting it done sooner than the estimate. This helps us see why a more aggressive commitment could be the right choice for an organization. (When I was thinking of McConnell’s estimation error graph as a scheduling graph, on the other hand, I was wondering why executives would ever push for a more aggressive commitment when the numbers were clearly against it!)
Of course, there is a minefield of issues to navigate when you try to complete more work in less calendar time; for instance:
- How do you keep your quality at acceptable levels in the face of staff fatigue?
- What is the cost of lowered morale if the deadlines seem impossible to meet?
- What costly technical and architectural blunders do you commit because you’re focused on meeting the deadline and so you’re not as thoughtful about what you’re producing?
and many others — I think that we should lift these up as appropriate so our executives do get a picture of what kinds of cost we’re taking on as part of the aggressive schedule — but then I guess it’s our executives’ job to weigh the relative costs and so navigate!
It was the excellent last chapter of Software Estimation (Chapter 23: “Politics, Negotiation, and Problem Solving”) that got me thinking about this issue — in that chapter McConnell helps you see through the eyes of an executive and understand the kinds of issues you grapple with at that level.