Rediscovering the obvious

(Rant alert: you have been warned)

Been plowing through The Art of Unix Programming, which, in general, is excellent – should be required reading for anyone developing software.

Joel Spolsky wrote a good critique under the title Biculturalism, at the end of last year, and agree that Eric S. Raymond does seem to be occasionally (but not always) unreasonably blinded / biased against Windows and the notion of GUIs.

One particular example crops up in his discussion of documentation, where I feel he underrates the value of WYSIWYG editors, even when used to edit structural markup (vs. presentational), where it serves as a form of code generator. Having used Epic Editor (Reviewed by Kevin here), which is a commercial docbook editor, found it worked well – many times better than hand coding, even if you are using emacs. I’d argue that the lack of decent WYSIWYG editor for docbook is why PEAR is largely undocumented, by way of example.

Despite that (and it’s interesting insight in itself to see where he get’s it “wrong”), it’s an excellent book.

What stuck me as perhaps the remark of the whole book was this;

In retrospect, it is rather startling how oblivious we all were to the implications of our own behaviour.

He’s talking here in the context of Open Source development and The Cathedral and the Bazaar but I think that remark can be applied more broadly to software development as a whole – we’re too blind to the human factor. That’s where development methodologies like extreme programming come in. If you read between the lines of hype and jargon surrounding XP (which is par for the course, if you want your ideas taken seriously), it’s really about adopting an approach well suited to normal human beings, bearing in mind our limitations.

Returning to Raymond, he goes on to say;

This experience should make any thoughtful and humble person wonder what other important unifying concepts are still implicit in our behavior and lurking right under our collective noses, hidden not by their complexity but by their very simplicity.

Having originally studied Physics (perhaps the original form of systems analysis, excepting Philosophy) my fundamental view is any system that has observable rules can (and should) be understood by anyone. Once understood that system becomes predictable. Of course the more complex the system, the harder it can be to determine the rules which govern it and experience plays a part in that process, while an individual’s conditioning can lead to bias that obscures the rules. That said, the first impressions of a “naive” observer often carry real insight, and should be valued.

The reverse situation is when intelligent individuals sacrifice what their “gut feeling” and buy into the latest vision of the “right way” to do things, as proscribed by the expert of the day.

An example outside of software development (but relevant to all the PHP fathers out there – congrats to George who’s the latest) is the debate which revolved around violence in entertainment, and it’s effect on children. Back in the 80’s, as Hollywood upped the level of violence in films, there was huge discussion as to whether watching Arnie blow away another legion of minor actors was good for children. The entertainment industry managed to silence the critics by asking “Where’s your proof?” (or: where’s your expert?). Years later the Joint Statement on the Impact of Entertainment Violence on Children, having conducted countless studies, confirmed that screen violence can be damaging to children. Wow – there’s a surprise! Who would have thought it?

Returning to software development, what’s gets my goat is seeing experts (of the flavor that never got past “Hello World!”, if they even got that far) holding forth on the future of software development (as well as software houses riding the wave).

All time classic probably goes to Professor M. Eric Johnson (Professor of Operations Management and possessor of smile that needs wiping) who said, back in December 2003, in an interview on outsourcing with the NYT (registration required);

Out in the Bay Area there are plenty of folks who would love to create a little bit of protectionism around their I.T. jobs, but we are far better off letting a lot of those jobs go. Low-skill jobs like coding are moving offshore and what’s left in their place are more advanced project management jobs.

Nuff said. (there’s further discussion here and here).

Meanwhile, enter Software Factories, the next revolution. I sincerely do not mean this as MS bashing – the heart of the idea seems to the same as the advice Eric S. Raymond has to offer – take advantage of code generation and domain specific languages (which you might argue PHP is an example of). The problem I have with “Software Factories” is the tone used – more “follow us, we know better, sacrifice your reason to the revolution”. Software developers may not buy it but history tells us those in those “advanced project management jobs” most likely will.

Again it seems the human element is being lost. By contrast, ActiveState published a whitepaper in July 2004: Dynamic Languages-ready for the next challenges, by design. If anything can stake a claim to “revolution” (in an ironic sort of way), I think it’s languages like Perl, Python, Ruby and PHP, having succeeded in reducing the gap between human intent and machine instruction, resulting in a real impact on software development effort and costs.

Sadly no one with marketing dollar is really pushing them, although someone at Microsoft is, thankfully, on the case, having hired Jim Hugunin, author of Jython (Python for Java) and Iron Python (Python for .NET).

Anyway – not entirely sure what I’m driving at here but the general feeling I get is those paying for software and it’s development are getting increasingly frustrated by big promises followed by poor delivery. Yet more hype and “abstract practice” does more to damage what’s already a poorly understood “industry”, providing fuel to those who believe software development is a low skilled job. KISS.