Few things are less sexy than command-line parsing. It is one of the most mundane tasks a programmer has to execute in their career. But, it surprises just how much code is required to do basic command-line parsing in a lot of languages, including Ruby. So I got to thinking, why does this bug me so much? I think the answer is that requiring so much code for such a relatively trivial task violates my sense of proportionality in the code. I hate having to say so much more about this teeny little task than I do about the “theme” of my code. I think it distorts the narrative of the code.

Let’s say that the processing of writing your program is like launching spacecraft. Ideally you would like to get from launch to cruising around in space as quickly as possible. The Star Trek universe solves this quite elegantly with the transporter. We don’t put you in a box and launch it, we break your atoms apart and transmit them to another location! That’s pretty cool, but maybe we’re just not quite that cool yet. Another solution is one proffered by the Star Wars universe. A ship like the Millenium Falcon can leave just about any planetary atmosphere any time it damn well pleases without the use of special equipment. It just flies away. Not bad.

However, here on earth, our primitive space craft need a tremendous amount of disposable apparatus to reach escape-velocity. The proportion of useful vehicle (the shuttle) to the orbit-busting mechanism (the rocket boosters and fuel tanks) is a staggering 5.4:1 (based on liftoff weight).

Command-line parsing in code exhibits a similar disproportion. The interesting part of your app isn’t the command-line parsing. Why should it take up such a disproportionate amount of space in your code? Those boosters quickly become “space-junk”, once the launch vehicle has left Earth; expensive trash that is never to be used again.

Space-junk is dangerous for the next guy that wants to launch into space. It’s also dangerous for the folks on the ground as it may decide to come crashing back down to earth. And those boosters have also been responsible for one of the worst disasters in NASA’s history. If we could only get rid of those damn things the whole space program just might be a little better off. Unfortunately physics, and our current space-flight capabilities currently require them.

But our code is a different story. We don’t have such physical barriers that handicap us. Any barriers we run into are usually of our own making. So why not try to reduce those as much as possible? Why say in more code, what you could say in less? Wouldn’t that cut down on bugs? Wouldn’t that ease the burden of maintenance? Wouldn’t that reduce the amount of information overhead you have to maintain each time you revisit that code?

Command-line parsing is a stupid, menial task that should require very little attention. By extension, it should only be given a stupid, menial amount of code to make it work. We have big ideas! They shouldn’t get bogged down by handling command-line options!

This is why I wrote Clip. Clip is an expression of the need to make the simple things simple, but no simpler. If you have modest command-line parsing needs, Clip rewards you with minimal investment. If you need something trickier, Clip allows you to say a little more to it and gain more benefits from it. You get to decide how much you want to engage—not the library.

This is one of the things I like about Ruby. The language is extremely flexible which gives me a lot of ways to “pack” ideas into code in a variety of ways. Having more than one way to do things isn’t all that useful by itself. But it’s essential when you want to write expressive code. Things like object literals, or one-line control-structure alternatives help me keep the lines of code proportional to the ideas they express.

This is also something I find challenging to do in Java. In languages like Java, even just creating a collection of objects requires quite a bit of code:

    1 import java.util.*;
    2
    3 public class Designer {
    4   public void makeItWork(List<Trash> trash) {
    5     // today's challenge: convert trash to wearable garments
    6     List<Garment> garments = new ArrayList<Garment>(trash.size());
    7     for (Trash t : trash) {
    8       Garment g = new Garment(t.getName());
    9       garments.add(g);
   10     }
   11
   12     submitToJudges(garments);
   13   }
   14 }

In Java we often solve this by pushing all of that code into a private method that is named something meaningful. This works pretty well, but does tend to result in an explosion of “helper methods”. Sometimes folks take the “cheap” route and simply prefix these riffs with explanatory comments like “convert each Trash into a Garment”. I’m not a real big fan of this. Generally I don’t care about the object conversion in the first place because the rest of the code is presumably doing something interesting with the Garments and I don’t give a damn about the Trash.

So let’s look at it in Ruby:

    1 class Designer
    2   def make_it_work(trash)
    3     submit_to_judges(trash.map { |t| Garment.new(t.name) })
    4   end
    5 end

By my count there are five lines in the Java example (including the comment) just to convert trash to garments. In contrast I boiled that down to one line in Ruby. OK I could have done this in two lines if you think that’s too much of a long-train. But I think there are couple of important points here:

1. The importance of the concept being expressed diminishes from left to right
2. The attention-span of the reader diminishes from top to bottom

The Ruby example beats Java on both counts. I don’t waste a lot of the reader’s attention span up-front on book-keeping details (in the vertical space) and I state the important thing I’m trying do (submit my top Foos) quickly (on the left). The details of which Garments I’m dealing with are merely a qualification of what I’m trying to do.

How you handle these two dimensions is greatly affected by both the language you use and the APIs you deal with. This is one of the reasons that I do not find the use of scripting languages for Java’s Swing API all that compelling. Scripting languages like JRuby or Jython help me with the horizontal space, but don’t do a damn thing for the vertical requirements. With an awful API like Swing I have to say a lot of words to make it go, regardless of the language I do it with.

Getting back to my dumb example, being on such a small scale this may not seem like a large impact. But multiplied several times over to match the size of most projects, this kind savings can really pile up. The difference in the amount of code required by these two approaches is manifested in a savings in cognitive investment required to grok these projects. This is the very essence of maintainability and sustainability. Anytime you can do more with less, you come out ahead.

At great peril to my own geek cred, I will say that this is why I find The Lord of the Rings to be such an awful piece of writing. It is so full of peripheral and non-essential information that finding the real story or characters requires extraordinary patience and concentration on the part of the reader. If Tolkien had been more concerned with the story and less with “world-building” I’ll bet he could have gotten that story boiled down to a single book.

Now I realize that a lot of folks love the Tolkien books for the very reasons I criticize it. That’s fine, that is an argument about aesthetics, not facts. However I would strongly argue that “world-building” in your code is a bad idea. I think you’re much more likely to build a decent piece of software if you pack your ideas tightly like a William Gibson novel than as a sprawling “trilogy” of epic code. Go ahead, prove me wrong. I double dog-dare ya.

OK, so by this point any credibility I may have had is gone. Look at the size of a post about saying more with less. In the hope that you might be lazy and like to skip to the end:

Do as much as you can…with as little as possible.

Emacs. Love it or hate it, it is undeniably a monument of software engineering. At best it’s an amazingly customizable work environment that can be shaped to your every whim. At worst it’s a giant time-sink where productivity is skewered by endless “fiddling”.

Since switching to the Mac over a year and half ago, I’ve used TextMate as my non-Java editor. TextMate is great text editor. It’s relatively simple to extend, has a very active community and only takes a little investment before a user gets productive. But I’ll admit that in the last month or so my eyes started wandering from TextMate—I felt like maybe I was ready to go back to Emacs.

Perhaps it’s worth a little history first. Back in the Dark Ages of Java Development (anyone remember Gnu Server Pages?), I was a pretty hard-core XEmacs guy for my Java dev. At the time Java IDEs were painful exercises in waiting and crash-recovery. JBuilder, early NetBeans, Visual Cafe were all valiant attempts but terrible failures. So I rode with XEmacs until the IDEs caught up.

Then one day I met Eclipse. Unlike any other Java editing tool, Eclipse was much more than a text-editor. Rather than treating your code like text to be manipulated, Eclipse actually understood your code. This allows you to think in terms of manipulating Java syntax and symbols instead of simple text tokens and lines. For a static language like Java this is incredibly powerful since you can determine up-front all of the possible things you might want to write. This is why a full-on “intelli-sense” feature for dynamic languages is hard to do as well in static, compiled languages.

I spent the next five years learning just about every nook and cranny of Eclipse and would put my raw Java-writing skills up against anyone. Even now, I still use Eclipse for Java and I can’t imagine going back to a plain text-editor—even a souped-up hot-rod like Emacs. This says less about the deficiencies of any text editors as it does about the requirements for developing in Java. Writing Java without all of the code-completion and refactoring tools an IDE like Eclipse has would simply exceed my pain threshold. So while Eclipse has become quite bloated lately and, when coupled with tools like Maven, is an exercise machine-servitude, it will be a cold day in hell before I give it up for Java. (NB: I plan on giving up Java-dev first.)

Now I’m writing a lot more non-Java than I did a year ago. That’s what brought me to TextMate. The integration for Ruby, HTML, CSS, JavaScript and shell was outstanding. I bought the TextMate book and was on my way to editor ass-kicking.

Unlike my Java days, now I identify myself less with a single language. Back in the day I was a Java Programmer™ dad-gummit. It was The One Language To Rule Them All and it served me well. Do you remember those days? Yeah, that was also the time when certified Oracle dudes were writing their own checks. But as time went on, I noticed that those Oracle guys pretty much knew how to do one thing and if and when the Day of Database Reckoning ever came, these guys were gonna be left standing in the cold wondering what the hell happened. I think that day has come and those dudes are becoming the equivalent of COBOL programmers—relegated to propping up legacy systems and talking wistfully about “the good ol’ days”.

So it got me to thinking about what I needed to do to survive in this business. Clearly I was going to have to evolve. So I started looking around for some inspiration and came across Ruby. Right now it’s my favorite language, but it’s not the only game in town. Ruby borrows a lot of features from other languages and a lot of other Ruby-ists are also spending time with other languages like Haskell, Erlang, io, and Lisp just to get exposure to other ideas. Moving away from a single-language focus to broader interests has been an important part of moving from rookie to journeyman. Evolve or die, right?

So how does this relate to Emacs? Well, like a lot of folks, I’ve been interested in learning more about functional programming. The problem is, I simply haven’t had the time to crack open the Erlang book and bend my brain in new ways. Don’t get me wrong, it’s on the list, but it’s going to be a while before I get to it. So I was eager to dip my toe in FP without a major commitment. Enter ELisp.

Before I dropped out of college, my brief career in academic computer science was spent with Scheme. My time in school didn’t end well and so I’ve associated Scheme (and by extension, Lisp) with that particularly unpleasant time in my life. I’ve only recently overcome the shakes when I see car or cdr. But a little time and some professional success has healed old wounds. In my first go-round with Emacs, I was really just using it as a power-editor. The internals of Emacs pretty much escaped me and in my mind there was a pretty major barrier between an ELisp user and an ELisp writer.

This time I’ve made a concentrated effort to learn more about how Emacs works. I’ve been racing through the O’Reilly Emacs book and this time it makes sense. I don’t know what the hang-up was before, but this time it all seems so logical! The entire editor is simply a collection of ELisp functions and variables built on top of other functions and variables. It’s all self-documented and incredibly dynamic. Wanna try something out? Just put a little Lisp in your *scratch* buffer and evaluate that sucker. Hell, in Eclipse I’d have to download the freakin’ source and compile it or swim in XML writing an extension. That’s incredibly powerful—the barrier to “trying stuff out” is mind-bogglingly low.

So while I’ve definitely take a short-term hit by trading in TextMate for Emacs, but I’m reaching productivity-parity pretty quickly. A couple of modes/libraries I’ve found that I’ve really liked are:

• Steve Yegge’s JS2 mode
• Rinari, for Rails
• Org-mode
• Magit for Git integration

I plan on spending some quality time with some of the various snippets libraries too to port over some of the templates that TextMate uses. For something like Rails development, which is about the most idiomatic programming I can imagine, having short-hand snippets is a major productivity boost.

So why go through this? After all, switching editors can be like converting from Catholicism to Judaism for some. Well, I think I’m doing this because:

• I’ve already overcome the initial steep-learning curve of basic editing in Emacs
• I get a nice, no-commitment introduction to functional programming
• I get a great environment to learn new stuff in
• With a little effort and a willingness to learn, I can make this sucker to anything
• Emacs has quite a track-record. It’s close association with Unix is no accident.
• Emacs customization is easy to write and a snap to track in SCM

I can’t say that I’ll use Emacs for everything. I’m getting into Cocoa development too and it looks like doing that outside of XCode is a fool’s errand. But that’s okay. I don’t need a single tool to do everything for me, just a handful of tools that help me get a variety of things done. That said, I’m pretty sure I’ll be solving a lot more problems in Emacs than I ever did in any other single tool.

Did you hear that? Did you feel it? I did. Agile just took another punch to the gut. The Agile Backlash is picking up a full head of steam. Rather than becoming a term to embrace, it’s become a term of ridicule. I think it’s probably deserved. As a guy who picked up the first Extreme Programming book as soon as it was released and is a long-time practitioner, I feel qualified to comment on this.

The second-generation acolytes turned a very pragmatic set of principles and practices into an esoteric scripture that could never meet the needs of reality. The “agile” moniker makes me cringe almost as much as “best practices”. Both are cavalier substitutes for critical thinking, too often thrown about in the hopes that their mere mention will work magic. Ugh.

A real unfortunate side-effect of the backlash is the disregard some have for automated testing. Before I go any further, let me be the first to say that, like any other recommendation or practice, automated testing is not a panacea. I shouldn’t have to say this. It’s painfully obvious, like explicitly stating up front that I am against beating up defenseless children. Who is?

However, just because automated testing isn’t a cure-all doesn’t mean that:

• testing isn’t worth doing
• testing isn’t hard
• sometimes testing is so hard that it isn’t worth doing

Note that last point. That there is what we in the biz call pragmatism. Yes kids, there are times when the cost of getting sufficient test-coverage is enormously expensive. Assuming that economics are part of your project somehow, you need to pay attention to this (which is why code coverage metrics can be dangerous in the hands of the nuance-free thinker).

Critics of automated testing like to point out that if developers write buggy code, how can they not write buggy tests? Assuming that it’s as impossible to write perfect tests as it is to write perfect code, is there a point to writing the test? There is if the benefits you get from testing exceed the cost of developing and maintaing them.

Test-Driven Design/Behavior-Driven Design

The TDD/BDD philosophies are nearly as abstract and misunderstood as the entire concept of testing itself. So rather than parrot the original manifestos, let me break down the tangible benefits of driving your code with tests. The first misconception is the notion that you write all the tests first then you write all the code. This works almost as well as gathering all the requirements up front first and then writing all the code. I believe the technical term is “Waterfall”. This is just dumb. Honestly folks, use your brains. Only a complete rookie would think that’s a sensible practice to adopt.

Putting testing before writing code provides no magical effect that radically increases the quality of your code. Where testing does improve your code is when you use them as part of the code-building process. I like to use unit-tests as a form of mental scaffolding while I develop.

Breaking It Down

Any non-trivial problem needs to be broken down in some way. Few of us have the mental capacity to keep an entire solution in our heads. Even if we could, it generally doesn’t last beyond the period in which the solution is developed. And even if you could do this, chances are few of your teammates would be able to. So at worst, breaking a problem down is a pro-social practice; at best it’s a tool to ensure that you really understand what’s going both now and in the future.

Generally when you break a problem down, you get into the business of assigning responsibilities to various components. This is a fractal pattern that starts at the highest-level architecture and repeats itself all the way down to the lowest-level classes, modules and functions of your system. It must be an inherent part of human-nature—we simply love putting things in boxes. Perhaps this is because that in doing so, we can hang a simple tag on a collection of related concepts as a form of shorthand. That shorthand allows us to talk about things in an efficient way. Imagine a design conversation where you couldn’t name anything, but had to describe a piece over and over by its details. Yuck. So yeah, boxes are important.

So where does testing fit in? Testing allows you to write automated assertions about each of those boxes. It’s like a logic game where you can start draw conclusions based on what you already know. It’s like being able to say, “If this test is passing and that test is passing, the only thing that could torpedo this is if that doesn’t work”. I’ll come back to this later when I talk about troubleshooting.

What unit-testing does not cover here, are the connections between the boxes. It should be obvious that it’s entirely possible to have a totally sweet set of unit-tests that never fail that cover each piece of your solution in full and still have bugs or represent and incorrect solution.

This is because unit-tests only cover the boxes, not the connections.

Tests that cover the entire system are the best way to exercise the connections. They cover the boxes too, but they cover the integration (hence the name) of the boxes and connections. Since unit-tests can’t cover the system as a whole, are they still worth writing? You bet. Keep reading…

Efficient Troubleshooting

When something goes wrong, you enter a different frame of mind from programming. Now you’re in the detective business. You start with some scraps of information (stack traces, error messages, late-night pages) and with a combination of logic, guile and intuition (hopefully) find the root cause.

I could do an entire post on root-cause analysis, but let it suffice to say that most folks don’t ask enough questions when troubleshooting. Often the conclusion people come to is an observation of a symptom rather than a thoughtful, system-aware diagnosis. In these situations it’s a good idea to consider the 5 Whys Approach.

How do unit-tests help with troubleshooting? Like I said earlier, unit-tests are really good at testing the boxes, but not the connections. Let’s assume that your unit-test accurately covers the box that you believe is the root of the issue you’re seeing. Given that, there is a handful of conclusions you can draw:

• The problem isn’t really in this box
• The problem is the connection coming into or going out of this box
• Your understanding of the box is incorrect

That’s it. It has to be one of those. Unit-tests allow you to confirm that your box is working correctly. If it’s being invoked or interacted with in a way you didn’t expect, it’s generally a trivial exercise to write a test that does use your box in that way. If it passes then you probably need to keep looking.

If you’re pretty confident that your box is good, then checking the connections is the next thing to look at. It’s just as likely that the output of your box is being misused as the input to it. Integration tests are usually the best mechanism to catch this, but are generally much more expensive to write and maintain. For this reason they generally don’t exhaust all of the possibilities. But with a real-world bug you now have a candidate for a new integration test.

I can’t stress enough the benefits of using bugs as opportunities to strengthen your test-suites. This is as true for integration tests as it is for unit-tests. It’s generally not practical to write tests for every situation. So we use our intellect to make educated guesses that cover the most likely situations. When you observe funny behavior in your system, consider it a gift to your understanding. It’s a golden opportunity to add a test for a real behavior, not a speculative one that may or may not occur.

So let’s consider a world in which we have no tests. When something goes haywire you have to start from first principles every time. You have no mechanisms to get any leverage on the issue. Instead we have to rely on understanding the entire solution and holding it in our head at once. What do you think the chances are that we’ll effectively troubleshoot an issue under these conditions?

Finding Smells

It is simply no fun to work on low-quality code day after day. As anyone who has every read Zen & The Art of Motorcycle Maintenance can attest, “quality” is a pretty squishy word. But in concrete terms, quality code means:

• You enjoy working on it (or at least don’t hate it)
• You can expand its capabilities with reasonable effort
• Somebody else can work on it too

I find the process of test-driving the development of code is a great opportunity to find “smells” in your code. These “smells” generally detract from code quality. The best catalog of these smells that I’ve encountered is the Refactoring book by Fowler and Beck. You don’t need to read the book for a detailed description of the mechanics of refactoring, but it is worth reading to learn the terminology of “code smells”.

At a minimum tests are the first “customers” of your code. If it’s painful to test your code, you probably haven’t factored it very well. Paying attention to this while your test-driving the development is a great way to keep your code high-quality. To really benefit from this, you need experience, a highly-developed sense of aesthetics and intuition. If you keep at it, you will gain these.

Note that I don’t equate code quality with “success”. There are plenty of systems out there that are successful that are not built on what I would call quality code. That’s fine. I’ve made a conscious choice not to work on those systems. I spend an awful lot of my waking hours working on code—personally I don’t want to waste that time being frustrated. I want a sense of accomplishment and satisfaction when I’m done for the day. Honestly, I’d trade success for quality code just about anytime. I expect to be doing this for a long time, so why not enjoy it?

Safety Net for Refactoring

When you need to extend or enhance a system, having automated tests serve as a great safety-net for refactoring. If you’re confident in the efficacy of your unit-tests, you can be confident when you make changes and your tests still pass. You can refactor with a higher level of confidence when your tests keep passing. Of course there are no guarantees that you haven’t introduced some kind of regression bug. But let’s face it folks, the testing game is one of probabilities. It’s too expensive to make sure you’re right all of the time—instead shoot for being right most of the time.

There will always be times when something goes wrong and the only way to diagnose and fix it is to walk through the code painstakingly, line by line. No amount of testing will ever make that go away. My goal is to reduce that as much as possible. It’s inevitable, but it doesn’t have to be the norm.

Consider the case where you don’t have any tests while you refactor. Your code/build/test/debug cycle gets a whole lot longer and more expensive. Because of that, you do it less often. Which means that when you do go through the cycle you have more things that can go wrong and more stuff you have to hold in your head. That increases both the likelihood that you will mis-diagnose a problem and that you will write more bad code to fix the latest bug.

I think this is how really twisted code is born. It almost reads like a blow-by-blow account of the developer’s mounting frustration. The pace and phrasing of the code gets more and more frenetic: more odd comments, more TODOs and FIXMEs littered throughout, more long procedural stanzas. It becomes obvious that they were trying to get this done as quickly as possible, and that’s generally not the path to quality.

Confidence

When it comes human-nature and software I’m a pessimist. I think we’re really good at the intuitive parts and achieving the “ah-ha” moment. We are not good at holding giant complicated systems in our heads. There are people who can do that, but it’s not the norm. For this reason, I think it’s extremely foolish to build a software organization that relies on raw mental capacity. This definition of “smart” is one that can’t scale and is unsustainable.

Automated testing is one teeny-tiny way we can offload the need for that capacity to a better long-term storage mechanism. While I write tests my mind is very focused on a particular part of the system. As I flesh out the tests I fill out my understanding of the problem-space at that time. Now that it’s written down in code, I can safely leave it there and let my mind free up that space to do other things.

If I’ve written my tests well, I can get back into that focused frame of mind when I need to (e.g. for new features or bug-fixes). Without these, I have to spend a lot more time understanding a larger system just to get “up to speed”. The tests can serve as an index into the design of the system. This is particularly effective when you have been paying attention to code smells and have been refactoring aggressively—your tests will map effectively to the underlying concepts of the system design.

This kind of confidence has a real tangible benefit. Without it I’m easily distracted by small details and dead-ends. As soon as I have more I have to worry about at once, I have less of a chance of succeeding at the task at hand in that moment. When I have such a thing in place, I have an instant force-multiplier in my hand. I can get something done. I don’t have to repeat the entire experience of developing the code the first time. Instead I can leverage that experience to be more productive the next time I come back to that part of the system.

Conclusion

So there you have it. That’s where I stand.

Is testing hard? Yes.

Does it catch everything? No.

Is it still worth doing? I think so.

It takes a brave author to give a book this title and keep it at 150 pages. The number of jokes about the proportionality of the “good parts” to the size of the book are endless. Be that as it may, when I saw this book at the Powell’s stand at RailsConf ‘08, I figured anything written by Crockford on the subject of JavaScript was probably worth taking a look at.

Given the diminutive nature of the the book, I suspect the author was attempting to do for JavaScript what Kernighan and Richie did for C with their book. I found an abused first edition copy of that C book and it taught me more about C than any other book. It is a triumph of restrained, focused technical writing (for other examples check out any of Kent Beck’s books). At times I felt that some chapters in “The Good Parts” were superfluous, but I imagine that they were included for completeness if nothing else. For example, the second-longest chapter in the book is about the grammar of the language. While important, it seems disproportionately large to rest of the contents.

If you have any exposure to JavaScript a good part of this book will be a review for you. However, there are three chapters worth reading for any JavaScript programmer. Chapter 3, Objects, discusses the properties of objects in JavaScript and covers everything from enumeration, to prototypes to attributes. Do you know why property enumeration is usually worthless as a debug tool? That’s because of the prototype mechanism. If you use typeof filters and the hasOwnProperty method, you can significantly cut down the properties you dump on an object. I never knew that before.

Chapter 4, Functions, is the real money-maker of the book. This is the chapter that I put the most post-its in and will certainly be one I’ll have to revisit. Here Crockford goes into the different ways functions can be invoked and what the consequences are of each invocation style. There is a lot of good functional programming material here. He also discusses how Functions are intimately tied to Objects. Take heed, thar be dragons!

Chapter 5, Inheritance, closes the loop on Functions and Objects. It explains the important differences between JavaScripts prototype-based inheritance model and the class-based model used in many other popular object-oriented languages. Like the previous chapter, this will be one that I’m sure I’ll have to re-read in the future.

It’s taken a long time, but JavaScript seems to finally be growing up a bit. Given how truly awful it was to use it in the Early Days on more than one browser, it’s a bit surprising that it has come to dominate the web landscape. Now it seems to be finally getting (begrudging) respect from the rest of the software community. Unfortunately to be effective, you really need to learn some of the deep voodoo that, to me, is a bit counterintuitive. Is the prototype approach really all that innovative? I’m not sure. Arguing that it’s better than inheritance is like saying that something is better than poking yourself in the eyes with a sharp stick—faint praise indeed. But I do love passing around functions as objects with complete abandon. When I finally grokked this part of JavaScript, things really clicked for me.

Is this book a must-read? No, probably not. If you’re doing any serious JavaScript in the browser you’re probably using one of the popular JavaScript frameworks out there that hide some of the yucky details of Function/Object interaction. But, lacking a decent language spec (and the ECMA spec sucks rocks), it’s not a bad resource. It certainly won’t take up much space in your bookshelf.

2.5 stars out of 5.

UIs are, essentially, collections of widgets. These widgets act as visual and manipulative proxies between the user and the underlying conceptual model. If we think of the ultimate UI as being one which minimizes the mental distance between what the user wants to do and what they have to do to accomplish it, perhaps the ideal interface would be like the fictional jet-fighter, Firefox. What could possibly be more direct than thinking, fire that missile, and seeing that missile burst forth from under the wing? Unfortunately, since most of us work on slightly less fantastic technologies than those portrayed in the movie, we have to figure how to work best within our milieu. This is the real challenge of what we do. How can we take the crude medium of computer software and diminish the distance between thinking and doing?

Yesterday I checked out a site that plays in the same space as a project I’m working on. These sorts of things are bit like going to the dentist, you don’t necessarily look forward to it, but you do it because it’s good for you. It’s not that I’m trying to ignore what is out there, but for a side-project there is a certain bliss in being ignorant of what else is out there–especially when there really isn’t any money at stake. But I was a good boy and checked it out. I went in fearing the worst–that these guys would have nailed the concept and the interface, and there was simply no reason to put any more time on our version. After about fifteen minutes I was pretty convinced that we could come up with a more compelling user experience.

Like mom always said, if you don’t have anything nice to say, say it anyway–just don’t name names. So let me try to provide some detail on what I found without tipping my hand. This app had all the right rounded-corner-drop-shadow-muted-blue patina that any self-respecting Web 2.0 app should. But digging a bit under the surface revealed a pretty face on what was essentially a Windows for Workgroups-era application mentality. The user flow felt like it was driven from the underlying relational data model, not necessarily from sensible use cases. As I filled out wizard form after wizard form, I couldn’t help imagining each one mapping to a particular table in a database. In addition, each form had several tabs with a mix of required and optional features to be identified by the user. The tabs were clearly meant to further sub-categorize the input, however because of the limited real-estate in a tab, only single-word descriptions were available. I had to click on each tab to see if I needed fill anything out in there or not. Too much clicking to satisfy the data model, not enough payoff for the user to keep it up.

Then I ran into something called “templates”. I can’t help but wonder if templating concepts aren’t a smell rather than a feature. In the case of this application, it wasn’t really clear what the benefits of templates would be. Normally templates are there to avoid repeating work and providing a model of re-use. Looking at this “feature”, I couldn’t help but wonder why I should care. If templates are so damn important, do I really want to use an app in which templates are a necessity? Again, too much work, no clear payoff. Humans serve the machine. Bah!

So why would somebody put something like templates in? The answer is re-use. Software developers simply love re-use wherever they can get it. I can imagine the developers and product team sitting around reviewing this complicated system they’ve built and finally having to address the fact that the complexity makes it hard to use. Templates to the rescue! We can reduce the user’s work-load with reuse! Ugh. It’s only an economic win if I really care to continue the app. Otherwise the real problem lies in the original interaction model.

The Golden Age of Perfect Software is nowhere on the horizon so building software will continue to be a type of gruesome sausage-making for the foreseeable future. One of the great lies about software development is the false economics of reuse. The theory goes like this: if applications can be built by assembling more-powerful, higher-level components together, the overall cost of software development should decrease. At face value this theory makes a lot of sense. I’m sure many of us can think of time we have wasted (or seen wasted) on building rounder wheels. Why should anyone spend time working on a library of sorting algorithms when for 99.999% of all cases the sorting routines available with your platform are adequate?

The problem is that the component-based thinking doesn’t always match the needs of the application. Sure we may have built it more cheaply than if we had written it from scratch, but we have to ask ourselves how much did we compromise in that effort? Software is a notoriously tricky business and the number of failures greatly outweigh the number of successes. Google wasn’t built in one shot. Heck, the first version of Google Reader was so badly panned, they threw it away and wrote another one. Apple is on the tenth major version of the OS software and fifth major revision within the latest family. Let’s not kid ourselves, we’re much more likely to get it wrong than get it right. So let’s just assume that it’s possible that none of the available, reusable components is going to suffice for your needs. That doesn’t have to be your starting assumption, but we have to keep the possibility in the back of our minds.

This leads to a second point I’d like to make about re-use. Many UI toolkits come with a pretty rich array of widgets that are used to build nearly every application out there. This is true of native clients as it is of web applications. If our goal is to minimize the conceptual distance between the user’s mental model of the application (remember, the developer’s mental model is almost irrelevant) and what they need to do to meet that model, then restricting ourselves only to what the widget palette offers will be fraught with compromise. Developers may see applications built as collections of widgets, but users often don’t. It’s quite possible that using the “standard” widgets for your application greatly increases the mental distance between thinking and doing for the user. When this happens the user is serving the application–not the other way around. Uh oh. Wrong turn. Back up. Try again.

Don’t misunderstand me. I’m not saying that every application has to be written from the ground up because there is no hope in getting the interaction right by reusing off-the-shelf components. That’s ridiculous. We’d never get anything done if we approached our apps like that. Rather, my point is that we can’t necessarily limit ourselves to the widget sets and conventions of our platforms. In doing so we may leave the user out in the cold. However, visual or interactive exceptionalism for the sake of differentiation invites similar risk. Dressing up your app to stand out is akin to putting lipstick on a pig. If you app is compelling, dressing it up will be unnecessary.

Let’s return to our anonymous whipping boy…er, sample web site. Not only did the wizards and multiple-layers of object hierarchy force me to learn the tool’s model (one I don’t care about), but the expression of those wizards (the widgets) similarly required me (the user) to play along, rather than getting me to the payoff. Some applications are very sophisticated and will require more of the user, but that’s not the case for this one. Indeed the selling point is how easy it is to use and how user-focused it really was! Boo, hiss. F-.

So while the wizards had a very consistent and recognizable look and feel to them, they did a horrible job of getting me to where I wanted to be. I can’t help but feel that this application was built with a primary focus on a relational data model and component reuse. This ended up with fat proxies between me and what I wanted to get done–an increase in the mental distance between thinking and doing. This is the real shame. Increasing that distance is the primal sin of application design.

One final note: a lot of the inspiration for this post came from a fantastic post by Cathy Shive on, what she terms, “Administrative Debris”. I interpreted “debris” as that which increases this conceptual distance I keep harping on. Her post, unlike mine, provides a lot more visual examples to get the point across. I would highly recommend reading her article to get another angle on these ideas.

Like many folks in this age of immediacy and information overload, I’ve been a big fan of multi-tasking. We multi-taskers believe that by multi-tasking we are cutting out any “down-time” and making nearly every previously-idle second a productive one. However, lately I’ve come to realize that the amount time and effort spent on switching my brain from context to context is incurring an overhead that exceeds any gains I might get in multi-tasking.

So lately I’ve decided to be a little more single-threaded in my approach to “getting things done” these days. For example, when I first got my shiny new MacBook Pro I installed every kind of whizzy Growl notification I could find. However I realized that the constant interruptions were really taking a toll on my ability to focus on any particular task. So these days I have very few notifications turned on. I check email when I damn well feel like it. I make myself available via IM when I feel I’m in a position to be interrupted. That’s about it. Anymore and I know I’ll never get anything useful accomplished.

So despite all of what I’ve just said, my personal geek-time has been quite split-brained between two separate efforts. The first is a methodical (and sometimes maddeningly slow) port of the Python code in Toby Segaran’s excellent “Programming Collective Intelligence” to Ruby. The second project is working my way through another fine technical book, Aaaron Hillegas’ “Cocoa Programming for Mac OS X”. Oh yeah, I’ve also been noodling around with the latest darling of the SCM world—git. This is to say nothing of the three or four other personal projects I’ve let dormant for the moment.

I’m a hands-on kind of guy and I get so much more out of nerd books when I can actually write, test, debug and run some code. So while porting code from from one language to another very similar language may feel ridiculous donkey-work, I feel like I internalize the underlying concepts better. As a sort of interesting side-effect I’ve worked up a number of nice collection-processing snippets around Ruby’s Enumerable that I hope to bundle up someday soon as a separate post. Also, if I can figure out a simple, secure way to host a read-only repository (Subversion or git) I’ll put the code out in case anyone gives a hoot.

Meanwhile, I’ve fallen in love with the Cocoa development environment. Objective-C is an interesting language, but perhaps not without its negative points (a compiled language with no type checking?). But the clean separation of MVC components coupled with the easiest WYSIWYG GUI designer is such a huge improvement over pure code-only environments like Java Swing. Anyway, I’ve had the idea for a simple little Growl app for some time. I’m pretty sure that once I get through the Hillegas book I can knock that out in pretty short order.

In the spirit of Peter King’s Monday Morning Quarterback, here are ten things that I think I think:

10. Maven 2 Moves The Ball Down The Field (well, sortof): All Java build systems suck. They all suck in different ways and the selection of one over the other is largely dictated by what types of pain you are willing to tolerate. I’ve used Maven since it’s inception. It solved some things Ant didn’t but also came with its own problems. Maven 2 attempts to solve some of the problems with Maven Mark I (namely scripting in XML, yecch), but lacks coherent documentation and a clear roadmap for learning the tool.

That being said, it makes tasks like creating a WAR file and deploying to a web server relatively trivial when compared with the amount of Ant XML you would have to write to do the same thing. There’s still too much XML for my liking and god help you if you want to do something outside of the box.

9: Maybe Static Imports Ain’t So Bad: When Java 5 first rolled around, I identified static imports as one of those features I would never use. Well, heh heh, never say never I guess. I have started to use static imports in a few places where I need something like the equivalent of a Ruby mixin. Interestingly enough the the only places I’ve felt comfortable with their use have been in unit-testing when I want to use EasyMock’s methods or JUnit 4 methods.

8: Java Annotations Are Curious Creatures: I can’t tell yet whether or not Java annotations are really all that great or not. If you’re a hard-core-no-good-thing-has-come-in-the-damn-JDK-since-the-collections-API kinda guy, you can make the argument that annotations are simply syntactic sugar for problems that could just as easily be solved with a proper object model or external configuration. In the old days (like the first version of EJBs) we wrote our configuration as actual code or, perhaps, as properties files. Then XML and Java went out on a hot date during the first dot-com bubble and spawned all sorts of illegitimate XML configuration children. I can’t tell you how many XML configuration systems I’ve seen that essentially reinvented properties files in XML (same configuration, twice the typing!)

Configuration via a pure object model can be, at times, extremely clunky (anyone remember writing EJB 1.0 deployment descriptors?) I think the evils of external file configuration have been experienced by just about any moderately experienced Java developer (Struts config anyone?) So it seems to me that annotations are simply a way to get some configuration closer to your code in something approaching a DSL (which most configuration files are attempting to be) without external files. I haven’t played enough with them to be totally sold, but the idea is intriguing…

7: I Couldn’t Code Java Without Eclipse: I say this not because Eclipse is just the greatest Java IDE of all time, but because I’ve invested so much time learning so many of its tricks. Really any modern Java IDE probably has equivalent functionality, so the issue is not so much that I can’t imagine coding Java without Eclipse, but rather that I can’t imagine coding Java with a plain ol’ text-editor. I don’t think this says as much about me as it does about the language itself. Does anyone doing Java professionally on a daily basis still use Vim or Emacs?

6: I Code Java, I Code Ruby. When Will I Use JRuby?: It’s funny that for as much Java and Ruby as I’ve done in the last eighteen months that I still have done no more than dabble in JRuby. The day can’t be far off, but I’m a little surprised that it hasn’t yet arrived.

5: I’m Tired of Algorithms: Well, that’s not exactly right. I’m tired of algorithms as the measure of one’s knowledge of software development. It’s important, but it’s not the most important. Apparently most of the software world doesn’t agree with me so it’s up to me to decide whether or not I want to continue swimming upstream or simply relax and go with the flow. Some little voice in my head tells that the latter is not a good idea…

4: I Think About Scaling All The Time: Aside from sustainable development, scalability is probably the most important thing I think about at my current position. I’ve worried about it before, but the scope of previous projects that I’ve worked on don’t approach the ambitions of this one. I’ve spent a lot of time in the last year reading up on all sorts of technologies around scalability. I’ve been pretty interested lately in things like the Spread Toolkit and Wackamole. I haven’t used them yet, but I’m keen to try them out. I’ve been using Jini lately…the jury’s still out on that one.

3: RDBMS Is Dead! Long Live RDBMS! Part of the result of all this scalability reading and thinking is coming to the realization that a relational database doesn’t work everywhere as the be-all, end-all of persistence. To that end I’ve been immersing myself in a number of different technologies. Inverted indices provided by the likes of Lucene or Ferret are fascinating alternatives to the traditional RDBMS approach. If you twist your brain the right way you can satisfy most requirements you would have of an RDBMS with inverted indices.

Similarly various key/value databases such as CouchDB and even Sleepycat make very sensible replacements for a general-purpose RDBMS. I find these interesting not so much because of their performance characteristics viz-a-viz RDBMS solutions—though that is certainly compelling—but rather because of their simplicity. For a lot of problems I really only care about storing and retrieving heterogeneous data via simple keys. Why make it any harder than that with a traditional relational column/row structure?

2: I’m Going To Ask Erlang Out On A Date: Erlang has gotten a lot of hype (and criticism) lately. Normally the simple churn of the blogosphere isn’t enough to get me interested in some johnny-come-lately, but Erlang strikes me as something with a little more staying power than the normal flavor-of-the-month technology. Perhaps it’s the track record as being “carrier-grade”. Perhaps it’s the move by some towards functional programming for solving particular problems. This, in particular, fascinates me. I’ve been an object-oriented guy for a long time, but there have been occasions when the OO approach seemed a little obtuse, but I didn’t really have a good alternative. I’m hoping that the process of learning Erlang will be a great foil to “get” functional programming.

1: I Think The OLPC Project Is Brilliant: I can’t think of a project that embodies more of the things I value than the One Laptop Per Child Project. It’s about putting the right amount of technology in the hands of kids. It’s about giving them a sturdy playground where they are encouraged to learn through experimentation. It’s about building sensible collaboration so that students can learn how to achieve more together as a group than they could as individuals. It’s about choosing and using a tool that gets things done. It’s not about learning a particular platform and it’s not about market-share.

I signed up for the Give 1 Get 1 program, but I can’t really justify having my own OLPC laptop. However I may get one just to play with it for a few days and then find a worthy home for it. I can’t imagine a better way to empower a kid than to put a tool like this in their hands.