Sizing Up Software, Part I (0)
Note: This article originally appeared in the August, 2009 edition of TEQ Magazine.
Older brothers can be very cruel to their siblings. Just ask my little sister. As kids, we often split the chore of washing the dishes after dinner. I dreaded this task, but one evening I had some fun with it at my sister’s expense.
After enjoying our mother’s chicken tetrazzini, I raced to the sink to wash my half of the dishes first. But instead of washing the dishes to perfection until half remained, I carefully washed 50% of every single dish in the sink. Each plate resembled a half-moon made of Parmesan cheese and chicken skin. Hilarious.
Sure, if I had done the job properly it would have taken less time, but you should never underestimate the wicked dedication of an older brother.
The moral of my story is that work assignments and job performances can be measured and interpreted in different ways. This is especially true with software projects.
As a software developer, I am commonly asked how “big” my applications are. (The cheeky side of me wants to answer with a physical size. “Well, I once built an iPhone app that was 2×3 inches!”) In all seriousness, when I’m asked this question I assume the person wants to hear about the scale of my software, so I tell them how long it took to build, how many concurrent users it can handle, and how large the underlying database is. Interesting stuff if you’re trying to gauge a developer’s experience, but what if you want to measure the progress of a project under development?
Some programmers measure their progress by counting how many lines of code they’ve written, but this practice makes me taste my lunch again. Why? First, the numbers aren’t isomorphic across different languages. (Twenty lines of C++ might accomplish the same as five lines of Java, or one line of Ruby.) Second, measuring progress by counting the lines of code encourages the development of bloated software. To quote Kanye West quoting Daft Punk, I’ve seen many applications that would be “harder, better, faster, stronger” if they were rewritten with half the code.
Last year this issue became an important one for my company. During negotiations with a client, I proposed that our fee be split into thirds, with the second payment due after three months. However, my customer wanted to cut the second check when “half the software was complete.”
But what does “half the software” mean? Half the lines of code? If so, then we’d have to somehow predict the final line count in advance. Since our software was web-based, it was suggested that we measure the number of pages built, but even that’s not perfect. If 75% of the pages are 65% complete, is that considered half? Besides, when web applications are measured by page count, developers are rewarded for building clunky user interfaces that use more pages than necessary.
Fortunately, I was able to convince my customer that “half the software” was too vague, but others aren’t always so lucky. I once saw a project whose members were unwittingly encouraged to ignore problems, because their success was measured by the number of bugs filed. If there’s no Dilbert cartoon for that, there really should be.
Good software developers view their work as a craft, and their software as a work of art. To them, judging software by silly metrics makes no more sense than judging the beauty of a painting by the canvas size, or the strength of a bridge by the number of rivets.
Instead of using lousy software metrics, companies should focus on what really matters: software quality. Yes, this is harder to measure because it’s more subjective, but it can be done.
What makes software good? For starters, good software isn’t bad, and bad software is easy to identify. Some of the many things that can make software bad include a lack of comments, poor formatting, mismatched naming conventions, copy-and-pasted code, deprecated method calls, and broken syntax that doesn’t even compile.
Just as it can be difficult to smell your own B.O., sometimes it’s hard to recognize when your own code stinks. That’s why my team uses an open-source Java application named Hudson. Hudson is a continuous integration tool, which is a type of program that automatically compiles, tests, and analyzes software on a recurring basis. Think of Hudson as a friend who sniffs you once an hour and tells you when it’s time to reapply the Old Spice. (To my fellow geeks: Old Spice is a form of deodorant.)
Once an hour, Hudson downloads and attempts to compile the latest source code for my team’s software projects. It then runs our code through a gauntlet of unit tests to ensure that our latest changes haven’t broken anything. Next, Hudson sends our source code through another tool named Checkstyle, which analyzes the code against a set of known bad practices.
In the end, Hudson generates a lovely web-based report that spotlights our projects’ health. Each project is summarized with an icon, showing either a shining sun for the really healthy projects, or a stormy thundercloud for the projects that should be taken out behind the shed. Clicking a report reveals more detail, including which unit tests failed and why, along with beautiful graphs that highlight the results of Checkstyle. For each issue, you can drill deeper into the reports to discover the source of the problem, along with suggestions for how to fix it.
Hudson is extremely customizable, and it does a million things I haven’t mentioned here. There are also many other tools available for measuring the quality of software, but Hudson and Checkstyle are the ones my team prefers.
I’ll admit that Hudson does a better job of telling you when software is bad, rather than telling you when software is good. For this I think you’ll always need code reviews and the opinion of an expert human being. Also, measuring the health of a project doesn’t really help you track its progress, so please don’t use tools like Hudson for this. Measuring the progress of a software project is such a difficult thing to do, that I’m going to revisit that one next time.
Project managers and developers, please stop measuring code with silly metrics.
At best they provide useless data, and at worst they encourage good people to do bad things. Instead, focus on measuring the health of your software and improving its quality. And if you’re still not convinced, just ask my dad about the time he asked me to mow half the lawn.
You can learn more about Hudson at http://hudson.dev.java.net , and Checkstyle at http://checkstyle.sourceforge.net.
Special thanks to Joe Gallo for editing and contributing to portions of this document.


