Book review: Software Metrics

This book review covers Software Metrics: Establishing a Company-Wide Program by Robert B. Grady and Deborah L. Caswell (Prentice-Hall, 1986).

Something Different

I thought I was getting a book on measuring developers’ individual personal productivity.  I lacked understanding of how it would be possible to do this without being overwhelmed by unintended side-effects, and I thought that reading a case study of a successful implementation would help me understand better and maybe be convinced. That’s not what this book turned out to be, though.

The book’s purpose is actually to help others set up a company-wide software metrics program.  The case study this book presents was an effort at HP to apply metrics to their software products and processes rather than their people.  Lots of good stuff in here, though. Let’s see…

Problem Solving Highlights

A primary benefit of the book for me was to see Grady and Caswell’s problem solving skills in action and learn from them.  Here are some things they do in the book that I found valuable:

• Keeping the Goal in View. In any effort this size, it would be easy to lose focus on one’s real purpose — what you’re trying to accomplish.  We catch glimpses of the authors keeping the goal in sight when they speak in terms such as these:
  

  …we now had our own database of metrics information to help guide us. But guide us where? …Data is meaningful when it is relevant and significant. Therefore, accuracy and applicability of both the data and the model must be better than one’s intuition (p. 151);

  and

  For complexity metrics to be of value, they must be used to create appropriate flags or trigger actions (p. 204).

• Rationale and Historical Context.  The authors don’t just say what they did, but the circumstances surrounding the decision and what it was that led them to decide the way they did.  For instance, on the topic of manual versus automatic data collection:
  

  A common complaint is that the whole collection process should be automated, leaving only the results to be analyzed. However, there is a positive aspect in doing manual data collecting. It gives the manager a detailed awareness of the data and its significance. Forms and questionnaires expedite the manual processes. Also, automation too early may “freeze” useless measures into the project process. Some experimentation is always needed at first (p.117).

  As another example, in addition to telling how and why they revised the class training objectives and content (Chapter 13), they show the first revision of their data collection forms, how they changed and why (see Appendix B).  So again, rather than just being given the answer, you see how they went about analyzing and solving the problem.

• Whose Problem Is It? What do you do when people don’t do as it has been agreed they should do?  As Gause and Weinberg said, Whose problem is it? When things didn’t go as expected with the new metrics program, the authors said, “This is our problem” and modified the program accordingly.  Once again, they were focused on the goal of having the program succeed.  Examples:
  • Early feedback from a new metrics training class led them to rework the course’s objectives as well as content (Chapter 13).
  • They responded to ”required” fields not being filled in in the data collection forms, not by increasing the pressure on managers to supply the information, but by digging deeper to find out why the information wasn’t being filled in (and ultimately splitting the form into two so that the smaller form for required data wasn’t so intimidating) (p. 258).
• Risk Management in action. Should the project name and/or project manager name be attached to the metrics data?  And what level of care is needed in keeping such information confidential? It’s instructive to see their thought process in deciding these issues (p. 136).
• Concepts of organizational transitions. The team anticipated and planned for the natural stages people were going to be going through when they encountered the changes proposed to support the metrics program:  “All changes require individuals to adapt to new circumstances.  It is a grave mistake to assume that any announcement, no matter how insignificant, is the end of the process.  All transitions begin with an ending” (p. 91).  The authors quote  a paper titled “Managing Organizational Transitions” by William Bridges, where Bridges outlines three phases people go through in such situations: letting go of the old situation; going through “a difficult ‘wilderness’ time in the gap between the old reality and the new one”; “Then (and only then) emerg[ing] with new energy, purpose, and sense of self to make a new beginning.”
• There seemed to be a natural process of discovery that took time. Primary metrics (the metrics directly based on the data you gather) were not sufficient to support process analysis and improvement; secondary metrics (calculated from the primary metrics) were generally needed for that purpose.  The authors go on, however, to add the helpful detail that in their experience you didn’t tend to be able to jump right to those metrics:
  

  Each project uses a slightly different development process. Understanding a process involves defining what the process is and deciding what metrics make sense to collect. We will call these more detailed metrics secondary metrics, not because they are less important, but because the need for them was recognized only after a basic understanding of the primary metrics was reached at a given division (p. 105).

Other Highlights

• Gestalt. One of the forms (p. 103) for gathering data about a defect contains the question
  

  How was problem found?

  One of the choices is

  gestalt (flash of inspiration to try something)

  Gestalt!  What a fun concept – I think I have quite a bit of that.  Perhaps I should add it to my resume… :)

• Ishikawa. This book contains my first introduction to an Ishikawa or “Fishbone” diagram, which is a simple, yet expressive way to depict contributing causes to an effect.  Seems like this could come in handy for whiteboard discussions.
• Estimation. This book reinforced some themes from the Software Estimation book: the importance of reestimation; the cone of uncertainty (pp. 143-144).  And there is a story about how good estimation early on helped the project by pointing out a scheduling issue early enough that it could be addressed by project management(pp. 156-157).
• Kinds of quality. This was a really eye-opening concept for me: we can’t just “improve the quality” in some generic sense.  As the authors say:
  

  For example, adding a new function might improve functionality but decrease performance, usability, and/or reliability.  Since each project has different priorities, it is necessary to make an early decision of what kind of quality is most important (p. 159).

• The need to look at productivity and quality together. When you look at productivity (lines of code or whatever your metric measures) alone, you don’t get the full picture.  As the authors say, “Looking at productivity alone could encourage efforts which result in productivity improvement at the expense of quality” (p. 170).

Overall

I found the book to be a very worthwhile read and would recommend it not only to those contemplating the implementation of a company-wide metrics program, but also for any problem solver – it’s a great opportunity to get inside the heads of a couple of master problem solvers and learn from how they think.

Book review: Software Estimation

This review covers Software Estimation by Steve McConnell (Microsoft Press, 2006).

Summary

McConnell says:

[T]he typical software organization is not struggling to improve its estimates from ±10% to ±5% accuracy.  They typical software organization is struggling to avoid estimates that are incorrect by 100% or more (p. xv).

And

I wrote this book for developers, leads, testers, and managers who need to create estimates occasionally as one of their many job responsibilities.  I believe that most practitioners want to improve the accuracy of their estimates but don’t have the time to obtain a Ph.D. in software estimation…If you are in this category, this book was written for you (p. xvi).

The first five chapters introduce some basic concepts:

1. Chapter 1, “What is an Estimate?”: You can get into some confusing situations when some of you are talking about an effort estimate and others are talking about a plan to hit a target.  This chapter explains the difference between estimates, targets, and commitments.
2. Chapter 2, “How Good an Estimator Are You?” presents a quiz that helped me see that “90% confident” wasn’t what I thought it was!
3. Chapter 3, “The Value of Accurate Estimates”, goes to some length to demonstrate the value of knowing the size of the project you’re embarking on, and the great cost to an organization that thinks a project will be significantly smaller than it turns out to be.
4. Chapters 4 introduces the four general answers to the question “Where Does Estimation Error Come From?”  In this chapter was a helpful and thought-provoking flow of questions beginning with these:
   

   • When telephone numbers are entered, will the customer want a Telephone Number Checker to check whether the numbers are valid?
   • If the customer wants the Telephone Number Checker, will the customer want the cheap or expensive version of the Telephone Number Checker? (There are typically 2-hour, 2-day, and 2-week versions of any particular feature — for example, U.S.-only versus international phone numbers.) (p.34)

   This chapter also introduces the concept of the Cone of Uncertainty, where the maximum precision of an estimate is impaired by not knowing what the project consists of. To get to a more accurate picture of how big the project is, you have to narrow that uncertainty. McConnell explains, “The Cone of Uncertainty doesn’t narrow itself. You narrow the Cone by making decisions that remove sources of variability from the project” (p. 39).

5. Chapter 5, “Estimate Influences”, introduces the concept of diseconomy of scale and discusses personnel and other factors that influence the amount of effort that will be required to complete a project.

I was going to stop the chapter summaries after the first five, but some of the next chapters have such good meat in them too…

• Chapter 7, “Count, Compute Judge”, introduces the concept that counting yields more accurate results than computing a number and computing yields more accurate results than expert judgment.  This theme comes up again other places later in the book.
• Chapter 8, “Calibration and Historical Data”, introduces the importance of collecting historical data on your projects and  gives details, guidance on what data to collect, and pitfalls to avoid.  Historical data is another theme that comes up many times throughout the rest of the book.
• Chapter 9, “Individual Expert Judgment”, offers a standard formula to use when you perform expert judgment estimation, so that you factor in best case, worst case, and expected case estimates with proper weight

Chapter 10 covers another quick and helpful technique — Decomposition (and its brother, Recomposition); Chapter 15, “Use of Multiple Approaches”, gives an example of using a combination of decomposition and expert judgment (p. 166) — a combination I have found helpful in my own estimating.

Chapters 18 through 20 cover special issues in estimating size, effort, and schedule, respectively.  Chapter 21 (p. 245) introduces a way of dealing with schedule risks (things that could come up and delay the project) by weighting them by the amount of time they would delay the schedule, times their probability of occurring, yielding a “Risk Exposure” (so for example a two-week potential delay that is 25% likely would have a 0.5-week Risk Exposure.  Using this concept you can figure what kind of contingency buffer your project is probably going to need.

Chapter 21, “Estimate Presentation Styles”, has several helpful ideas in it, such as an estimate with risk quantification, for example “+1 month if the graphics-formatting subsystem is delivered later than planned” (p.252), and “Don’t try to express a commitment as a range” (p.257).

Gold

Chapter 23 is gold.  McConnell talks here about people stuff — why negotiations between technical staff and executives tend to be difficult for technical staff to navigate. “Estimate negotiations tend to be between introverted technical staff and seasoned professional negotiators” (p. 259).  McConnell then gives some help on how to engage in win-win negotiation with executives: keeping the best interests of the company in mind, positive assertiveness, taking schedule requirements seriously, and maintaining  the courage to tell the truth.

A couple of quotes:

You hold the key to a vault of technical knowledge, and that puts the responsibility for generating creative solutions more on your shoulders than on anyone else’s.  It’s your role to propose the full range of possibilities and tradeoffs (p. 266).

Avoid saying, “No, I can’t do that”; instead, redirect the discussion toward what you can do.  The more options you generate that support doing what’s best for the organization, the easier it will be to show that you’re on the same side of the table as the person you’re problem solving with (p.268).

Themes

These ideas come up time and again throughout the later chapters

• The value of historical data from your own company’s projects for accurate estimates
• Prefer counting over computing and computing over expert judgment
• Using a range to express uncertainty in an estimate
• Taking care not to mislead by presenting a level of precision in your estimate that does not reflect the true precision.  (For instance, if your estimation techniques produce an estimate of 573.275 hours, you would probably want to present that as 575 or 600 hours)
• The limits of estimation
  Chapter 20 clarifies the purpose and limits of a schedule estimate:

  The purpose of the schedule estimates in this chapter is not to predict your final schedule to the day but to provide a sanity check on your schedule-related plans.  Once you’ve used schedule estimation to ensure that your plans are reasonable, detailed planning considerations (such as who is available when) will take precedence over the initial schedule estimation described  (pp. 230-231).

  And again in Chapter 21:

  Recognize that these allocations of effort to phases are approximate. They represent useful starting points for planning. Once the estimates get you into the right planning ballpark, detailed planning considerations should take precedence over these initial estimates (p. 237).

The book covers both iterative and sequential project styles.

A Peril Averted

As I read in the first few chapters about the dangers of corporate wishful thinking regarding estimation and scheduling and began to dig into the smorgasbord of techniques that are available to get a picture of the size of a project, my pride began to swell and my attitude began to sour.  Chapter 23, “Politics, Negotiation, and Problem Solving”, was like an unexpected bucket of cold water on the head that snapped me out of my arrogant stupor and really helped to put the lessons of the book into proper perspective.  My recommendation: don’t skip this chapter!

Overall

It’s small (270 pages of main text) and a quick read that makes a wealth of study accessible — McConnell pulls from many sources to give  organizations the tools they need to get a lot better at project planning, without having to take a lot of time to get up to speed on what he calls the “science of estimation”.  Well worth the small investment of time and money.

Book review: Exploring Requirements

This review covers Exploring Requirements: Quality Before Design by Donald Gause and Gerald Weinberg.  New York: 1989, Dorset House Publishing.

Summary

On page 1, the authors summarize the outcome of experiments they performed to find out “how computer programmers were influenced by what they were asked to do”.  Their conclusions based on these experiments were:

If you tell what you want, you’re quite likely to get it.

If you don’t tell what you want, you’re quite unlikely to get it.

The book is about getting to a shared understanding of what your product should be, for the purpose of making it much more likely you’ll get that.

Some interesting topics the book covers:

• Costs of ambiguity (p.17), and how to avoid it
• The side effects a hastily chosen project name can have on the design of the product (p. 57)
• How to design meetings so they minimize wasted time (p.89)
• Arguments about facts in the future (p.143)
• Functions, Attributes, Constraints, Preferences, and Expectations (chapters 14-18)
• The value of making requirements traceable, so designers can tell where they came from (choices versus assumptions versus impositions, chapter 24). For example (p.271):

  2. The elevators will travel vertically only.

  could mean

  2. The elevators will travel vertically only because none of the states in our marketing area legally allow nonvertical elevators.

  or

  2. The elevators will travel vertically only because we don’t see any appreciable market for nonvertical elevators at this time.

  The difference could be important for designers to know.

Things That I Found Helpful

(This book is enough outside of my area of expertise that I hesitate to use my normal “Good/Bad” categories.)

1. Chapters are short and to the point.
2. Not only points out dangers, but provides help in avoiding them.
3. Helps you remember to move on.  Example: Chapter 12 explores the influence a project’s name has on the project, but also notes that you need to know when to move on.  (“At some point…we’ll stop and use what we have because the important thing is not the name, but the naming” (p.132)).  Another example: Chapter 11 talks about the value of sketching rather than trying to be too precise early in the process (pp. 120-121).
4. Gives good reasoning for things.  For example:
   

   Some people resist black box testing of requirements with the argument that this opens the danger of “studying for the test.”  They fear that the designers will build a product or system that does no more than these tests specify, rather than one that will be wise enough to cover cases nobody thought of.  Well, if the designer knows other important questions and their proper answers, why not put them on the list?  There they will be subject to public scrutiny and also be available for constructing acceptance tests. (p.257)

5. The Studying Existing Products chapter (Chapter 23) provides guidance for when you’re developing a new product that is akin to an existing product:  ways to make sure that the new product is like the old one where it should be and not where it shouldn’t be.
6. Chapter 25, “Ending”, provides helpful thoughts on the courage, risk-taking, and professionalism  necessary to do good requirements work.
   

   The requirements phase ends with agreement, but the requirements work never ends until the product is finished.  There simply comes a moment when you decide you have enough agreement to risk moving on into the design phase (p. 277).

   and

   The purpose of requirements work is to avoid making mistakes, and to do a complete job.  In the end, however, you can’t avoid all mistakes, and you can’t be omniscient.  If you can’t risk being wrong, if you can’t risk being inadequate to the task you’ve taken on, you will never succeed in requirements work.  If you want the reward, you will have to take the risk  (pp.282-283).

Things I Wished For

1. Could use an update.  It’s twenty years old and cites experiments performed ten years before that.  (I doubt that the nature of large-scale human efforts has really changed since then though.)
2. Doesn’t address agile software development (being written in 1989!).  I’m thinking that changes  in light of agile principles would affect more the organization-level aspects (suggested meetings, client interactions) than the specific advice.  Would delivering working software obviate the need for some of these activities?  It would be neat to see the author’s opinion on this.

Overall

The book covers a topic that is outside of my day-to-day responsibilities: by the time work gets to me the requirements have solidified quite a bit.  But sometimes things slip through as undocumented assumptions and bite us later.  This book raised my awareness of some common places those assumptions sneak in and gave me some simple tools to help turn them into documented decisions.

To use the book as I did was 283 pages of quick reading.  There’s more depth available if more of your work involves exploring requirements (how to run effective meetings, questions to ask clients, etc.)

I recommend this book.

Book review: JavaScript: The Good Parts

This review covers JavaScript: The Good Parts, by Douglas Crockford.  Sebastopol, California: O’Reilly Media, Inc., 2008.

Summary

This book undertakes to introduce the most important  things you need to know to use the good parts of JavaScript well, in 100 pages.  The author’s premise is that within the JavaScript language is a much cleaner subset, and that if you restrict yourself to that subset you’ll be able to write better programs.

There is also an appendix of Bad Parts, and another appendix of Awful Parts (Bad Parts that cannot be avoided).

The Good

1. Terse!  Succinct! The book says so much in so few pages.  I’ve bought 800 or 1000 page books before that had almost nothing to say, but this book’s every word is there for a purpose.  Even the code snippets, while supporting some statement in the text, sometimes provide additional value by illustrating a language detail not mentioned in the text.  For example, a code example on page 22 shows var a = {}, b = {}, c = {} and a = b = c = {}, demonstrating a couple of ways JavaScript supports multiple assignments in one statement.
2. Dry, non-sugarcoated  style. This helps keep the tone light when dealing with a programming language that admittedly has a lot of warts.  Examples:
   

   undefined and NaN are not constants.  They are global variables, and you can change their values.  That should not be possible, and yet it is.  Don’t do it. (p. 107)

   And:

   Unfortunately, for in makes no guarantee about the order of the properties, and most array applications expect the elements to be produced in numerical order. Also, there is still the problem with unexpected properties being dredged up from the prototype chain. (p. 60)

3. Points out surprises in the language. This is one of the strongest points of the book.  Two quick examples:
   • The typeof operator returns object sometimes when you wouldn’t expect it (p.16);
   • Dealing with array objects:
     

     Because an array is really an object, we can add methods directly to an individual array…Since the string ‘total’ is not an integer, adding a total property to an array does not change its length.  Arrays are most useful when the property names are integers, but they are still objects, and objects can accept any string as a property name.  (p.62)

4. Introduces you to the benefits of a loosely-typed language. Crockford says:
   

   JavaScript is a loosely typed language, so JavaScript compilers are unable to detect type errors.  This can be alarming to people who are coming to JavaScript from strongly typed languages.  But it turns out that strong typing does not eliminate the need for careful testing.  And I have found in my work that the sorts of errors that strong type checking finds are not the errors I worry about.  On the other hand, I find loose typing to be liberating.  I don’t need to form complex class hierarchies.  And I never have to cast or wrestle with the type system to get the behavior that I want.  (p. 3)

   It was really interesting to read this book soon after finishing Effective Java, since that book  had such a focus on using strong type checking to prevent errors.

5. Helpful railroad diagrams (p. 6).  I found these really helpful in making a grammer intuitively understandable.  You can take your finger and trace around to see what’s allowed and what’s not (for examples, see the JSON site).
6. Explains closures. (pp. 37-39)

The Bad

1. Organization. The author says at the beginning of the book:
   

   There is a lot of material packed into it.  Don’t be discouraged if it takes multiple readings to get it.  Your efforts will be rewarded.  (p. xi.)

   I understand the desire for terseness and succinctness — these are strengths of the book.  But there are organizational elements that make the material more difficult to absorb than it needs to be.  A couple stand out:

   • Forward references. There are points in the book where a term or concept is stated in the text without explanation and you haven’t been given the tools to understand what you’re seeing yet — by later in the book you have what you would need to go back and understand, but at the moment, you about have to shrug and go on.  Examples:
     • The term “truthy” is introduced on page 10, but not explained until page 12
     • The term “refinement” is introduced on p. 14, but not explained until page 17
     • “The / operator can produce a noninteger result” (p.16)…  Huh?  (I now think this may be referring to NaN)
     • The Object.create function described at the bottom of p.22 was a mystery to me the first time through
   • Grammar front and center. Chapter 2 is full of railroad diagrams of the language grammar, with accompanying explanation.  The diagrams are helpful in understanding the syntax — but as I proceeded through the chapter I wished I had had more concepts first.  Then I would be able to dig into the grammar with some understanding.
2. Lack of cross-references within the text. Even though it’s only 100 pages, it would still be quite helpful to have a page reference instead of just “There will be more about X in the next chapter” (three examples of this on page 17) or “in Chapter 6″ or “at the end of this chapter” (p 31).
3. Index is mediocre. The index is a Bad Part of this book.  It is missing several entries that would have been helpful to me.  Examples:
   • Forward references such as truthy, falsy and refinement are missing from the index
   • The topic of “global abatement” is not specifically in the index so you have to sift through three entries about global X looking for the page that deals with global abatement.  Similar for “semicolon insertion” — the index entry is for “semicolons” and you have to check out the two pages listed to find the part about automatic semicolon insertion.

   The index falls short of what I expect for an O’Reilly book.

4. Doesn’t understand what positive and negative regex lookahead are for (p. 75).  A minor issue that I’m tuned into since reading Mastering Regular Expressions, 3rd ed. Crockford says lookahead is not a good part of the language… but would he maintain his opinion if he had seen my wonderful examples of how they can help?

Not Sure If Good or Bad

1. Defines a new language? The author suggests augmenting the language in several ways to neutralize deficiencies in the language as it exists:
   

   Suggestion Strength	Suggestion	Page
   we can…	Add method method to Function.prototype	32-33
   we can fix it	Add integer method to Number.prototype	33
   That is an easy oversight to fix	Add trim method to String.prototype	33
   we can fix that	Add curry method to Function.prototype	43-44
   we can work around that	Add is_array function	61
   can easily correct	Add dim method to Array.prototype	63
   we can correct that too	Add matrix method to Array.prototype	63
   You may want to	Add isNumber method	105

   Maybe this is a good idea.  And it’s so little code — about 45 lines total!

2. Explains how to create tamper-proof objects (pp. 52-55).  After getting a glimpse of some of the flexibility and benefits of a loosely-typed language, I’m not sure what to make of tamper-proof objects — they seem to work against all that flexibility, and Crockford does not offer any analysis of how this fits in with a philosophy of normally not locking down things.  Is this something that you’d normally reserve for special situations?  Use all the time?

Overall Impressions

Reading the book was like tagging along with an expert.  Sometimes he’s teaching you, but other times you don’t understand what he’s doing.  Sometimes you just have to be patient, and watch and learn.

I give the book an A in Content and a C in Organization/Indexing/Cross-Referencing.

The faults I have mentioned are not showstoppers.  The main part of the book is only 100 pages, and you can work around the forward reference problem by skimming through it a second time.  Crockford’s book helped me quickly get up to speed on the language and avoid many issues I would likely have stumbled against as a JavaScript beginner, and that was my hope for the book.  I would recommend the book to anyone hoping to do the same.

Book review: Effective Java

This review covers Effective Java 2nd ed., by Joshua Bloch.  Stoughton, Massachusetts: Addison-Wesley (an imprint of Pearson Education, Inc.), 2008.

Summary

From the Foreword:

If you have ever studied a second language yourself outside the classroom, you know that there are three things you must master: how the language is structured (grammar), how to name things you want to talk about (vocabulary), and the customary and effective ways to say everyday things (usage)… This book addresses your third need: customary and effective usage.

And as Bloch says in the Introduction:

This book consists of seventy-eight items, each of which conveys one rule.  The rules capture practices generally held to be beneficial by the best and most experienced programmers.

The items are organized into ten categories:

• Creating and Destroying Objects
• Methods Common to All Objects
• Classes and Interfaces
• Generics
• Enums and Annotations
• Methods
• General Programming
• Exceptions
• Concurrency
• Serialization

The format of the book is based on Scott Meyers’ Effective C++.

The book is designed as a next-level book for those who already have some experience with the Java programming language rather than as an introduction to the language.

The Good

The book:

1. Helps the Java non-expert toward usability, robustness, and flexibility.  There are a lot of things to know about Java to use it well.
2. Gives reasons for his statements. In cases where you may decide to deviate from Bloch’s advice, this gives you the opportunity to examine his reasoning and understand the implications of your decision, so you’re not flying blind.
3. Makes recommendations concrete by giving examples. For example, in Item 20, the exhortation to prefer class hierarchies to tagged classes is illustrated with a tagged class and its alternative representation as a class hierarchy.  There are helpful code examples like this sprinkled throughout the book.
4. Includes a succinct summary of the advice at the end of many Items.
5. Provides expert help in doing common but tricky tasks. For instance, the naive developer may not know the contract that they’re responsible to uphold when overriding the equals() method (Item 8), or may not realize the maintainability implications of marking an object Serializable (Item 74).  Much time and distress can be saved by reading about common pitfalls ahead of time.
6. Points out how to avoid many surprises. For example, several important details that should be dealt with when making a class Clonable (and the consequences of ignoring them – Item 11); and details about the relationship between arrays and generics (Item 25).
7. Provides help in doing some advanced tasks well. For example, the chapter on concurrency points out the benefits and details of using the Executor Framework in java.util.concurrent rather than the Thread class (Item 68); the Serialization chapter presents the Serialization Proxy pattern (Item 78);  and Item 17 discusses issues that should be addressed when designing a class for inheritance.
8. Includes some just plain old great ideas. The book includes several ideas that made me think, “Huh!  Why aren’t we doing this?”  An example is his advice to put methods on your checked exception that help your callers extract the information they want out of it without having to resort to parsing the description string (p. 245) and to give your custom exception types constructors that receive that information (p. 254).
9. Is honest about Java design mistakes. Bloch believes in the Java platform, but he is not shy about pointing out its warts.  For example, p.86: Stack should not extend Vector and Properties should not extend HashTable.  I appreciate that honesty.
10. Helps you design for robustness. Don’t leave room for error where you can prevent it — Bloch gives many techniques for making correct use happen naturally.
11. Is scientifically up-to-date: Doesn’t call Pluto a planet (p.149) : )

The Bad

1. Often doesn’t take testability into account. This is my number one gripe with the book.  Examples:
   • Item 13, “Minimize the accessibility of classes and members”, doesn’t address how its advice interacts with making a class unit testable.  In practice, there tends to be some tension between minimized accessibility and full testability.
   • Item 17, “Design and document for inheritance or else prohibit it”, similarly recommends marking classes final when they’re not designed for inheritance.  If you follow this advice naively, though, you could be setting up your code to be untestable (see the section in Michael Feathers’ Working Effectively with Legacy Code chapter 10, titled “The Case of the ‘Helpful’ Language Feature”).  Bloch does mention in passing an alternative that “provides the flexibility to use subclasses internally” (p.91), so if you have your testability goggles on that could point you in a helpful direction.
   • Item 60, “Favor the use of standard exceptions” – but it can be quite helpful to extend the standard exceptions so that test code can be certain that when an exception occurs, it was thrown by the place the test is testing.
2. Sometimes bogs down in the details. Examples are Item 11 on cloning and Item 30 on enums, both of these being several pages longer than their sibling Items.  There were maybe five such extra long Items.
3. Lacks numbered subheadings under Items. The book makes do with bold text, but it would be more navigable if there were numbered subsections.  This is especially the case for the longer Items.
4. Could benefit from flowchart diagrams. Several times near the end of a chapter there is a series of questions leading to a decision tree.  For example, on p. 180:
   

   If you answered no to the first question, ask yourself one more: Do I want to limit the use of this marker to elements of a particular interface, forever? If so, it makes sense to define the marker as a subinterface of that interface.  If you answered no to both questions, you should probably use a marker annotation.

   It takes effort to process this prose, and I would find it helpful if such conditional logic were presented in the form of some kind of flowchart-like diagram.

5. Directing attitude. This is the flip side of “helps you design for robustness” — not uncommonly, in the quest to close the gaps where a class could be used improperly with poor results, Bloch’s advice is to prevent, guarantee, or force.  (An example of this type of advice is Item 58, where the recommendation is to use checked exceptions to force clients to deal with an exception.   I’m not sure this is A Good Thing.)

Overall

I’ve used the word “details” a lot in my descriptions.  Bloch gives you the details you need to use Java effectively.

By reading this book I became aware that there’s more to understand about Java than I realized.  There were several things I didn’t know I didn’t know — the caveats of serialization and cloning, for instance — and some things I knew I didn’t know (concurrency comes to mind)

My biggest concern is the lack of attention given to testability.  The reader who cares about testability is on their own to work through which advice can be taken at face value and which must be modified by testability concerns.

Still, I highly recommend this book.  Its imperfections are surmountable, and every professional Java developer should understand the issues it addresses.  Even in the cases where you  may not follow Bloch’s rules,  you will gain valuable understanding of the trade-offs you’re making; and you’ll avoid many costly mistakes.

Book review: Mastering Regular Expressions

This review covers Mastering Regular Expressions, 3rd ed., by Jeffrey E.F. Friedl.  Sebastopol, CA: O’Reilly Media, Inc., 2006.

Background: The Pain

Regular expressions:  They tend to be difficult to write and difficult to read… but it’s hard to get away from them.  They can help you manipulate text in sophisticated ways.

Often it is possible to avoid regular expressions and get by with simple (non-regex-enabled) text search and replace — and for several years I have done so where possible; but every so often the task is complex enough that I fumblingly try my hand at regular expressions again.  When it works, I am glad; when it doesn’t work, I’ve often not been clear about what went wrong.  But the persistence with which problems continue to arise for which regular expressions would be an elegant help has convinced me of this:  Regular expressions are a tool that a professional programmer should have in his or her toolbox.

Tired of my stabs in the dark, I decided it was time I work to gain a deep understanding of these regex beasts.

Summary

This is a book about mastering regular expressions.  It’s not primarily a regex quick-reference guide, nor a “Get up to speed on regular expressions in 24 hours” book.  Rather, it’s a steady climb from the chapter 1 bunny hills to chapter 6′s double black diamonds.

The book is divided into three sections: the introduction (chapters 1-3); the details (chapters 4-6); and tool-specific information (chapters 7-10: one chapter each for Perl, Java, .NET, and PHP).

As Friedl says at the beginning of chapters 8, 9, and 10:

[T]his book’s foremost intention is not to be a reference, but a detailed instruction on how to master regular expressions.

The author recommends reading the first six chapters before jumping into one of the tool-specific chapters.

The Good

• Great topic coverage: Here are some of the topics covered by the book:
  • Greedy vs. lazy quantifiers and how they affect matching
  • Backtracking
  • How a regex engine’s “transmission bump-along” works
  • Comparison of the three types of regex engines: DFA, Traditional NFA, and POSIX NFA – and how the engine type affects matching and efficiency
  • In what ways the “language” used for a character class is different from the language used for the larger regex
  • How to be careful when using greedy quantifiers like .*
  • Atomic grouping and possessive matching
  • Non-capturing parentheses
  • Positive and negative lookahead and lookbehind (collectively, “lookaround”)
  • Differences among regex flavors
• Grizzled wisdom: In addition to all the topics covered, Friedl frequently notes caveats such as “this chart is only the tip of the iceberg — for every feature shown, there are a dozen important issues that are overlooked” (p. 91) and then explains what he means.  He’s not only explaining the table at hand — he’s also helping you learn how to think about tables of regex engine features — to learn what information you can safely glean from such a table and what important details tend to be left unstated.
• Attention to detail: (This is similar to the previous point, but seems distinct in my mind.)  Some technical books assert that if you do X, the system does Y — but leave out the rare (but important) cases where if you do X, the system doesn’t do Y, and the cases where the system does Y without you doing X, leaving you to stumble into those cases on your own.This book does vanishingly little of that.  As just one example, on page 442 while he is explaining PHP’s m and D pattern modifiers, Friedl discusses the effect if you don’t use either modifier; the effect of the m modifier; and the effect of the D modifier.  But then, in characteristic form, he adds that “If both the m and D pattern modifiers are used, D is ignored.”  This type of attention to precision is bound to save the reader research and debugging time discovering such cases on their own.
• Steady guidance along to the advanced topics: If Friedl started out with some of the advanced topics from chapters 5 and 6, I might have lost hope and given up.  Instead, he starts out simple and builds as he goes.  While I did not always take the time to fully understand each example in chapters 5 and 6, I found chapters 1-4 very approachable when taken in order.
• Diagrams and tables: I found the diagrams sprinkled throughout showing the text and what parts of the regex match where, very helpful.  There are others — the backtracking diagrams on 229, 230, and 231; the tables on pages 92 and many other places.
• Helpful cross-references everywhere: Whenever a concept is mentioned that is developed further somewhere else, the text points to the page number of that further development.  Also, at the beginning of some chapters there is a table-o-pointers (like a mini table of contents) to topics discussed in the chapter, for later quick reference.
• Brain-jogging quizzes: I found the quizzes sprinkled through the book to be helpful in getting my brain going.  If the quizzes had been lumped together at the end of each chapter, I would have skipped them — but since they were few and sprinkled in among the reading at odd times, they piqued my interest, and my comprehension was aided by doing them.
• Respectful tone: Though he starts from the beginning building a foundation to help the reader understand regular expressions, Friedl avoids a condescending tone.  He also avoids an apologetic “I want the reader to think I’m cool” tone when dealing with much deep technical content.
• Good craftsmanship: Each diagram, table, section, quiz — as well as the organization of the chapters and the progression of the examples — has a purpose and contributes toward the single purpose of helping the reader master regular expressions.  Even the unique typographic conventions contribute to understanding.  No diagram is there just for fluff.  This coherence is refreshing.
• Now it’s a quick reference: Now that I have read the book, I can use it as a quick reference.

The Bad

• Not a quick reference at first: I had barely started the book when a regex question came up in our development.  I thought that lookaround might be the solution to our problem, so I flipped to the point in the book where the concept is introduced (Adding Commas to a Number with Lookaround, pp. 59 and following)… and had little idea what I was reading.  I found I could not skip to later sections without going through the sections leading up to them.
• Takes some commitment: Reading this book was less like using a vending machine and more like a two-month apprenticeship.
• You have to do work: You have to think!

Overall

There’s territory to master,  and I can’t fault the author for that.  This basically neutralizes the “bad” comments about the book.  I had fiddled around with regular expressions for several years without growing much better at them, but this book has has launched me into being able to use regular expressions in much more advanced ways.  What a joy it was, for instance  — I think I was probably in chapter 4 at the time — to be able to help a co-worker using my newly gained knowledge of lookbehind!

After studying through Friedl’s book, I’m finally not a regex beginner any longer.  I understand the territory better, and if a regex didn’t match like I expected I believe I could look into it and have a shot at figuring out the cause (before, my practice was more hack-and-hope).

I heartily recommend Mastering Regular Expressions for anyone who feels the time has finally come for them to take the time to really understand regexes.

Book review: The Java Programming Language

This review covers The Java Programming Language, 4th ed. by Ken Arnold, James Gosling, and David Holmes.  Stoughton, MA: Addison Wesley, 2006.

Summary

From the preface: “This book teaches the Java programming language to people who are familiar with basic programming concepts.”

The Good

Coverage: The book makes a great attempt at covering the basic topics you need an understanding of to make good use of the language.

Authority: The book’s description of how the language works is solid.  (If, like me, you’ve ever experienced one of those programming books where the author’s statements about how the language works tend to  be on the level of “I ran a simple test on my PC and hastily draw the general conclusion that it works like X, but I don’t know any of the edge cases or caveats”, you’ll appreciate this!)

Tone: The book’s tone is not condescending.  Instruction and tips are presented in a respectful, matter-of-fact tone that doesn’t

Guidance: The book not only tells what’s possible with the language, but offers advice at times on what things might or might not be a good idea (for example, the subsections of 23.3 on Shutdown; and the footnotes on page 42, 206, 428).

Honesty: The authors aren’t afraid to occasionally say “Java does X because of a design error that at this point cannot be fixed”, or to note something that isn’t necessarily the cleanest but is there for historical reasons (examples: the spelling of Cloneable (section 3.9.1) and the footnotes on pages 85, 184, 326, 391, 408).  I find that refreshing.

Shows where you need further study: For instance, in the second paragraph of section 14.10 talking about the Java memory model, the book’s explanation of when one thread’s change to a variable is or is not guaranteed to be visible to another thread with or without synchronization made me realize that I need to find a good book just on Java concurrency to master this area.

Helped me see the big picture: A lot of work has gone into the design of the Java language, and while there are a lot of details in the book, it didn’t neglect the big picture.  The book helped me see some of the things the designers were trying to accomplish with the language.

Makes you think about design: Beginning to see the big picture (see previous point),  I also saw areas where the design of the Java language is not mistake-free, or where its designers made decisions I think I disagree with.  For instance, was it a good idea for the containers to implement a fat interface and throw UnsupportedOperationException on certain operations?  It seems to me it would have been cleaner to honor the Liskov Substitution Principle (PDF) and slim down those fat interfaces to what each collection type really must support.

Describes the old kitchen sink: The book spends a fair amount of time covering such topics as the legacy collections.  These legacy-but-not-deprecated aspects of the language are important to understand so that you know to prefer their newer replacements and can effectively read and work with code using these older constructs.

Index: A big ol’ honkin’ index.  Nice!

The Bad

Thick: Not a quick read.

Not a One-Stop Shop: Even at 760 pages not counting the index, the book doesn’t go into enough detail to confer expert knowledge of all the topics it covers.  For example, after reading the concurrency chapter, I mainly realized that this is a topic that I’ll need to study more.

Describes the old kitchen sink: The book spends a fair amount of time covering such topics as the legacy collections.  I listed this as a positive, but it’s also a negative — it’s distracting if you want to concentrate on learning how to do things the new way.

Doesn’t take testability into account: This is my biggest beef with the book.  Language constructs such as final and protected are discussed with a view toward what you’re enabling extending classes to do or preventing them from doing — and that’s good!  But the book says nothing about how  such design may affect the testability of the code, and it’s easy to accidentally write untestable code using some of those language features (see Michael Feathers’ Working Effectively with Legacy Code, Chapter 10).  So, you need to bring your existing knowledge of writing for testability with you and use discernment when the book recommends a design style that might result in difficult-to-test code.

Overall

This is a very strong book, which I heartily recommend.  Some might chafe at the somewhat slow pace, but I appreciate the book’s thorough coverage of language features and inclusion throughout of consequences, trade-offs, and caveats to know about.  For topics such as concurrency that the book did not cover in sufficient detail for me to use well, it gave enough detail to help me realize that I need further study on that topic.  It was written by folks who understand what they’re talking about, and it’s now my primary Java reference.

Book review: Are your lights on?

This is the first in a series of book reviews I hope to do.  The books are mostly high-quality and deserve better cover shots than the ones I have for them (maybe someday I’ll get a scanner!)

First up is Are Your Lights On? How to Figure Out What the Problem Really Is by Donald C. Gause and Gerald M. Weinberg, reprinted in 1990 by Dorset House Publishing.

Summary

Organizations (and the problem solvers within them) often make expensive mistakes by solving the wrong problem, solving the wrong person’s problem, and several other ways.  The book explores important questions that should be answered before jumping to a solution.

The book’s title should not be understood to be asking the question rudely; rather, the question all at once:

• Is the solution to a puzzle in the book;
• Is a good reminder to would-be problem solvers;
• Illustrates quite well the point of chapter 10, “Mind your meaning”!

The Good

Eye-opening: First and foremost, the book got me to think about seeing problems from a variety of angles and perspectives, and it helped me see that how you perceive and state a problem has a big impact on what solutions you drive toward.

Captured and maintained my interest: The book starts right out with an engaging problem that drew me in and helped me experience the issues the authors want to address.  By about the third page of text, I was already being helped to see flaws in my thinking process.

Accessible: The book maintains a light, entertaining style throughout.

Quick read: This book can be read in a couple of evenings.

Covers a wide breadth of issues: In just over 150 pages, the authors deal with flawed assumptions, various forms of miscommunication, moral dilemmas, when you’re the problem, whether we really want to solve the problem, and many other facets.

Connects with real life: The authors demonstrate a grizzled understanding of human nature, and their advice reflects that understanding.  At no point do you worry, “That sounds nice, but do people really operate that way?”

The Bad

The bad in the book is in the form of some undertones.  It’s hard to tell if these undertones are really there or if it’s a misperception on my part, but I thought I’d report them as  I saw them:

Low view of personal integrity? This is not seen constantly throughout, but it pops up in chapter 16′s “whatever works” philosophy suggesting that certain classes of problems be solved by trickery — playing deceptive games with one’s superiors — and maybe also in chapter 12, “The campus that was all spaced out”.

Tendency toward a contemptuous tone toward non-technical management: Though addressing many mistakes technical problem solvers commonly make, the authors tend to have a compassionate tone toward these mistakes and a tendency toward a contemptuous tone toward those in management positions… with the notable exception of chapters 14-15, which almost led me to delete this section (but not quite!)

Overall

This was a really eye-opening book for me.  I recommend it as worthwhile  reading for anyone in a position with problem-solving aspects to it.  For software development organizations, I wouldn’t limit its applicability to the official requirements-gatherers for product features; anyone who needs to understand requirements should understand the concepts this book addresses.

Have you read the book?  What do you think of it?