Measuring productivity?

Our new CEO is not afraid to take us where we’ve never been yet.  (In fact, I think that’s a major reason he came!)

One of the things he wants to do is to reward individual excellence*.

Will this require being able to measure (individual) productivity?  (Understand, I’m all for producing excellent work and being rewarded for it!)

Whenever the issue of measuring productivity is raised, I always think of Martin Fowler’s 2003 post in which he says that you Cannot Measure Productivity. However, while Fowler gives examples in that post of productivity-measuring attempts backfiring, he doesn’t demonstrate that it’s intrinsically impossible.

Measuring productivity is hard, and it’s a people-stuff thing that as software developers we’re not trained to do.  But is it really impossible?  Let’s re-evaulate here.  What do others have to say?

I appreciate Johanna Rothman’s writings.  She has some things to say about measuring productivity:

  • In Measuring Productivity #1: Defining Productivity: “…a single-dimension measurement is inadequate for measuring a person’s productivity, and that on a project, it’s the productivity of the project, not the productivity of a given person that matters.” (She re-emphasizes the  the team productivity rather than individual productivity thought in Why Hire Junior Contractors?)
  • In Measuring Productivity #2: Measurement Considerations:

    If we want to start measuring developer or tester productivity, we have to define output per hour (or whatever time period you desire).For developers, we can measure designs considered, lines of code generated, the number of defects generate along with the code, unit tests generated, unit tests run, how good that output is,and the time it took the developers to generate all that output.


    Not so simple, eh? If we just knew how to measure how good our work was, we’d have a chance to measure individual producitivity. The more I think about productivity, the more I know it’s a project-by-project thing, not a person-by-person thing.

  • In Measuring Productivity #3 she gives some potential ways to measure team productivity.

So again, measuring individual productivity is difficult… easy to do wrong… but can we get a picture of how we might work through those difficulties?

Steve McConnell adds to the discussion in his Measuring Productivity of Individual Programmers post and gives some caveats; for example:

The problem with measuring productivity in terms of lines of code per staff month is the old Dilbert joke about Wally coding himself a minivan…This really just speaks to the management chestnut that “what gets measured gets done,” so you need to be careful what you measure.

He gives four caution-laden guidelines to use when attempting to measure individual productivity, and concludes,

On real projects it’s hard to find a use for individual productivity measures that is both useful and statistically valid. In my experience, aside from research settings the attempt to measure individual performance arises most often from a desire to do something with the measurements that isn’t statistically valid. So while I see the value of measuring individual performance in research settings, I think it’s difficult to find cases in which the effort is justified on real projects.

(Measuring team productivity and organizational productivity is a different matter — I’ll blog about that soon).

So if we’re not careful, it seems we could easily cause unintended side effects through a reward system (like in the Dilbert example)

I’m hopeful, though.  There are many difficult things that we need to learn to do well, not just this — wouldn’t it be cool if this were one where we persevered and overcame?  I’m curious to see how this will play out!

*Was that the term?  I think it was something like that — it made me think of individuals rather than teams, but that may be a misperception on my part.

Advertisements

  1. #1 by Ben Burkle on November 12, 2008 - 10:21 am

    Thanks for putting your thoughts out there. It would be easy to just complain (like me) rather than get to the bottom of things.

    Seems like I naturally go straight to individual measurements as well, but have thought we need both productivity and quality measurements to balance things out. Like lines of code, plus number of defects. I think over the course of a year, the numbers would become valuable. Probably not on a project by project basis. Another reason I like these measures is that we should be tracking them anyway per method/label to find code that should be refactored.

    Another item for consideration is whether our measures are subjective or objective. I prefer objective measures. Is there an objective team-based measure?

  2. #2 by danielmeyer on November 12, 2008 - 11:06 am

    Thanks for the comment!

    Johanna Rothman talks about this in her Possible Measurements post. She mentions there how not measuring defects produced vs. measuring them affected the project for one team she worked with. Interesting stuff!

  3. #3 by Matt on November 12, 2008 - 1:37 pm

    At my company, every one in the company track personal, team and company wide goals and performance. It gets really hard to determine personal performance with pair programming. We defiantly don’t have a good answer on how to do it but the issue comes up on my team from time to time. With Agile, it seems that the emphasis should be on the group productivity.

  4. #4 by jon fuller on November 19, 2008 - 8:41 am

    I’ll say I like the idea of rewarding based on personal excellence (if that was indeed the word used, and even if it wasn’t!). I think this breaks down, however, when you try to correlate personal excellence to productivity/quality/etc. (which, I guess, is what your post is all about nailing down).

    I personally don’t think it’s appropriate to do any kind of reward system based on a given individual’s statistics (LOC, defect density, etc.) because as objective as those numbers may seem, they will certainly never be normalized across members of a team, nor across teams, so it will be nigh impossible to set quantifiable objectives that will cater to all persons/teams.

    I like a reward system that is based on evaluating project success (did we meet the stated project goals? is the customer happy? did we make money?, etc.). This requires clear project start/end dates, and clear project acceptance criteria. This is, of course, a team based reward system.

    Just my $0.02 ;)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s