Smaller steps, greater strides ... revisited

I often see colleagues of mine struggling to add new features to our software; I often see workmates struggling to rectify the existing features of our solutions. Frustrated managers wonder why code that only took 2 or 3 months to write has stagnated in an endless UAT cycle with little hope of breaking free. Small change requests end up taking 3 times longer than the estimates that helped justify the change in the first place. Customers, initially used to timely releases grow increasingly impatient as release periods are extended or bug littered deployments are made on schedule.

Why does this happen? Optimism play a small role in this predicament; as software developers we almost always underestimate the tasks that we are given. Pick up any Steve McConnel book and look at the graphs - it's an industry wide phenomenon. Although this plays a contributory role, it is by no means the root cause of the problem. I rarely see developers having difficulty putting new code together; I guess there's no getting away from the fact that software is easy to write but difficult to maintain. There is also the question of motivation, nobody likes to work with scrappy code; as Pragmatic Programmers we shouldn't have to live with broken windows.

So why do we write poor, complex, undocumented code when we know it will eventually come back to haunt us (or more likely someone else on the team). Sometimes it's laziness but I think there is an inherent trait that developers possess - we are always keen to see the results of our ideas. We start with the best of intentions but as we gain momentum we see the outcome beginning to materialise. We write code more quickly, absolutely determined to come back and add the comments and write the unit tests. It's not true TDD anymore but the result is almost tangible now, just a few more lines and we'll be home and dry. Sure we have some tidy up code to complete, exception handling needs to be incorporated and then there's the diagnostics namespace to worry about...but just a few more lines and the refactoring can begin...just a few more lines...

And so the problem begins. before we get chance to paint over the cracks a system bug is reported, the requirements change and a different solution is screaming for attention. We no longer have time to commit to our solution, there is no longer an end point to reach for - just boring changes and bug fixes. The momentum disappears and we move on to a newer, brighter challenge vowing never to cut corners again. Somebody else is assigned to the project, sees all the broken windows in the neighbourhood and continues to break a few of their own. Which leads us back to our frustrated managers and impatient customers.

There are many methodologies that address these issues and offer a strategy for preventing them. Personally I think the methodology we need is a simple injection of discipline. After all, if you can't be bothered to add a single function header now, will you really come back and add 100 once your class is complete? If the code seems to work ok will you really find the time to go back and write the unit tests. Even if you do, the opportunity to confirm your understanding of the problem before the coding started disappeared when the TDD process stopped.

As with any habit, good or bad, it can take a lot of discipline and effort to stop. The same is true, in my opinion, with software development practises. Jumping ahead may seem to shorten the journey but smaller steps really do lead to greater strides especially in a well disciplined environment.

It takes time to make a decision

Some people describe software development as a manufacturing process. Basically you take a recipe, try it out a few times and once you are happy with the ingredients and the cooking process you write it down and pass it on to your eager coders. Over time you collect a whole book of recipes that can be easily followed, producing code that is both repeatable and reusable. Software restaurants announce ticker tape openings around the globe to serve up cheap and reliable cookie cutter software solutions. Recipe for success? Actually, past experience tells us that more often than not it is a recipe for disaster.

The manufacturing process needs to guarantee repeatable results and this is very difficult given the amount of human intervention that programming typically involves. Of course, there are now many utilities that generate, parse, check, analyse and generally prettify source code but it's still essentially a solo practise between the developer and the IDE (apologies to all you XP pair programmers out there). There is also the issue of code maintenance to consider, generated code is not the prettiest of creations and, whilst it's fine when it works, any subtle imperfections are often very difficult to isolate and rectify.

Some people describe software development as an artform. Each contributor adds their own creative interpretation of requirements, applies keyboard brushstrokes to their electronic canvas and steps back to admire yet another unique creation. No two pixel paintings are the same; programmers the world over write different code to achieve the same objective. Even individual coders write the same routines differently, with or without varying degrees of standards and procedures, many times during their illustrious careers. An artistic masterclass? Very often this approach generates code that may be inventive but is seldom practical and rarely maintainable.

The problem here is consistency; code is produced that can be difficult to maintain where the layout of each class looks very different to its neighbours. There are a number of architectural decisions and considerations that a developer has to make that exist outside of what is generally perceived as the 'system design'. Does a particular piece of logic sit in the database layer or the business layer? How are exceptions handled? Should a method be instanced or static? Do you choose an interface or an abstract base class? In reality your code is simply a string of statements, loops and conditions but the potential number of combinations are astronomical even within a relatively small code base. Coding standards and recommended practises can help to reduce the confusion, code generation can produce generic classes and components and design patterns provide some level of guidance but dividing a limitless number of possibilities by any given factor still leaves a limitless number of possibilities.

So should we strive for a repeatable and consistent, painting by numbers development process where creativity and initiative are completely removed from the programming equation? Or do we promote flexibility and encourage our development teams to express their creativity and inventiveness in new and inspiring ways? Is there some magical, middle ground that satisfies the developer's creative aspirations whilst still adhering to repeatable templates, design patterns and agreed best practises?

This blog is littered with questions but unfortunately devoid of answers; each question corresponds to at least one, and often very many decisions. It is ironic that the decision making process is a key factor in making programming an interesting practise. It is also a subtantial contributor to the problems and complexities that lurk beneath the surface of even the most trivial of solutions.

Decisions take time ... so less choice, less freedom and fewer choices must surely facilitate a better programming environment...or do they?

Preparation is everything

During a particularly engaging World of Warcraft quest last month I decided that it was time to find a 'proper' hobby. This dawned on me as my fellow guild members started calling me old timer and logging off at 7pm on school nights. I'd always had a distant fascination with astronomy and chose to prefer star gazing into the heavens as a more intellectual alternative to star gazing in Hello magazine.

Armed with my battered credit card I went online shopping and 2 weeks later my new telescope arrived with built in gps device, object database and goto facility. My enthusiasm was not diluted even by the prospect of a multi-lingual, 200 page instruction manual.

After a hasty assembly I cursed and scowled my way through the remaining 5 hrs of daylight hoping sunset would deliver me a cloud free northern hemisphere. My prayers were answered and I found myself wrestling with a 100 pound glass / metallic hybrid which possibly outweighed the instruction booklet.

The next problem was to point the telescope at true (not magnetic) North, ensure it was aligned horizontally and then validate its position with one or two predetermined reference stars. This seemed like a daunting prospect but the next 30 minutes reaffirmed my belief in technology. The attached handset was intuitive to use, it found my local time and location (using the inbuilt gps device), and happily pointed the scope at true North. Whirring back into motion, the motorised mount pointed me at a reference star from its database of truly stellar proportions, a slight manual adjustment and I was ready to go.

This typifies, for me, one of the major benefit of software systems. They help you prepare quickly and efficiently for the real task at hand. Don't get me wrong, watching a telescope engaged in an automated, robotic dance while it aligns itself with an object 200 light years away is certainly an impressive experience...the first time around. On a cold, damp night it soon becomes a process that you would happily perform instantly given the opportunity. The real experience is looking at the comet battered surface of the moon or the perfectly formed rings around Saturn.

So do software systems prepare us for the tasks we need to complete and the activities that we enjoy rather than performing them for us? I would like to think so. After all, once your forecast reports have been scheduled for printing and your manangement reports have been automatically generated the real work can begin. Efficiency improvement discussions, marketing campaign brainstorming sessions, sponsor driven board meetings, the day to day human collaboration that constitutes the organic nature of an organisation...these are the processes that really make a difference. It's comforting to think the polished oak table that decorates the board room will not be replaced by server racks and an air conditioning system. So perhaps we should think about how our systems can assist rather than replace next time we are struggling to put realistic requirements together. It could save a lot of effort thinking about something that really isn't necessary.

And if the monthly reports find their way to the MD's desk a little earlier because of the new accounting system then so much the better. You can contemplate how software helps rather than replaces your daily functions as you walk down the first fairway on your sunny afternoon off. In fact, thinking about it; preparation isn't everything - it's just something that software systems are particularly good at. :-)

Vital statistics

As with many software houses, we have tried various ways of measuring and monitoring the 'quality' of the code we produce. We are not blacksmiths forging ornate, wrought iron gates that welcome visitors to the stately homes that they protect. We are not carpenters creating polished dining tables that provide the social focus for a very special dinner party.

We write code, code that is often compiled into some indecipherable intermediate language and then laid at the mercy of a virtual machine. It cannot be directly admired by its users; even when displaying information, it renders results that are presented on a screen that has been designed by a graphics designer or business analyst. Often it displays nothing at all, quite happy to sit far from the limelight in a cold, dark server room that provides more comfort to mushrooms than human beings.

Despite its intangible nature, it is what we produce and it defines and differentiates us as solution providers. It is every bit as important to us as the gates and tables are to the blacksmith and the carpenter. Few people would argue that quality of code is important but in order to measure it properly we first need a suitable definition.

Of course there is no absolute, irrefutable description for 'software quality' and there are as many opinions (most probably more) than there are IT professionals. Many runners jostle at the starting line in the race for software excellence: coding standards, performance, bugs per developer...even cyclic complexity! My suggestion is to judge code on its simplicity.

Simple code is easier to maintain. The problem with code maintenance is that it is usually done a long time after the original code was written and by somebody that didn't write the original code. Code maintenance often occurs after software has been released where the cost of change is highest.



Simple code is easier to understand. Being frustrated for hours by a design pattern implementation or sample code from an anonymous expert on the internet that doesn't quite work as expected is usually an indication that you should try something else. If you don't understand it now then how will it appear in 6 months time to someone else that has to fix a critical bug, immediately, with no previous knowledge of your code or the system it belongs to?

Simple code is self describing. It does not require copious comments or armies of related documentation fighting to remain up to date with each change that is made to the associated software. Of course, comments are a good thing but only if they accurately describe the code they represent; this is often not the case. It could be argued that no documentation is better than inaccurate or outdated documents where false assumptions are easily made. Either way, the simpler the code, the less need there is to refer to supporting documentation.

Simple code requires less effort to produce. The Cocomo II project provides comprehensive analysis of factors that contribute to increased effort in software implementation. Platform experience, appropriate software tools, multisite development and time constraints, along with many others, all play second fiddle to product complexity in terms of increasing effort.

A good story can be enjoyed even if scribbled in crayon on crumpled paper. However, print in a legible font, on good quality paper, add page numbers and chapters, a preface and an index, sprinkle with artistic illustrations and suddenly it springs to life. In a similar way, simple code is enhanced by good coding standards, proper indentation, sufficient comments and a pragmatic design. In fact, simple code is defined by these attributes rather than merely enhanced by them.

As developers, we often strive for the perfect design and implementation. Sometimes it is the ability to compromise effectively that produces the best results; it is left to the reader to decide whether worse can be better.

In any event, keep it simple and the rest will follow - and you won't find yourself cursing in 6 months time when you have to refactor your optimised code :-)