Smaller steps, greater strides ... revisited

I often see colleagues of mine struggling to add new features to our software; I often see workmates struggling to rectify the existing features of our solutions. Frustrated managers wonder why code that only took 2 or 3 months to write has stagnated in an endless UAT cycle with little hope of breaking free. Small change requests end up taking 3 times longer than the estimates that helped justify the change in the first place. Customers, initially used to timely releases grow increasingly impatient as release periods are extended or bug littered deployments are made on schedule.

Why does this happen? Optimism play a small role in this predicament; as software developers we almost always underestimate the tasks that we are given. Pick up any Steve McConnel book and look at the graphs - it's an industry wide phenomenon. Although this plays a contributory role, it is by no means the root cause of the problem. I rarely see developers having difficulty putting new code together; I guess there's no getting away from the fact that software is easy to write but difficult to maintain. There is also the question of motivation, nobody likes to work with scrappy code; as Pragmatic Programmers we shouldn't have to live with broken windows.

So why do we write poor, complex, undocumented code when we know it will eventually come back to haunt us (or more likely someone else on the team). Sometimes it's laziness but I think there is an inherent trait that developers possess - we are always keen to see the results of our ideas. We start with the best of intentions but as we gain momentum we see the outcome beginning to materialise. We write code more quickly, absolutely determined to come back and add the comments and write the unit tests. It's not true TDD anymore but the result is almost tangible now, just a few more lines and we'll be home and dry. Sure we have some tidy up code to complete, exception handling needs to be incorporated and then there's the diagnostics namespace to worry about...but just a few more lines and the refactoring can begin...just a few more lines...

And so the problem begins. before we get chance to paint over the cracks a system bug is reported, the requirements change and a different solution is screaming for attention. We no longer have time to commit to our solution, there is no longer an end point to reach for - just boring changes and bug fixes. The momentum disappears and we move on to a newer, brighter challenge vowing never to cut corners again. Somebody else is assigned to the project, sees all the broken windows in the neighbourhood and continues to break a few of their own. Which leads us back to our frustrated managers and impatient customers.

There are many methodologies that address these issues and offer a strategy for preventing them. Personally I think the methodology we need is a simple injection of discipline. After all, if you can't be bothered to add a single function header now, will you really come back and add 100 once your class is complete? If the code seems to work ok will you really find the time to go back and write the unit tests. Even if you do, the opportunity to confirm your understanding of the problem before the coding started disappeared when the TDD process stopped.

As with any habit, good or bad, it can take a lot of discipline and effort to stop. The same is true, in my opinion, with software development practises. Jumping ahead may seem to shorten the journey but smaller steps really do lead to greater strides especially in a well disciplined environment.

It takes time to make a decision

Some people describe software development as a manufacturing process. Basically you take a recipe, try it out a few times and once you are happy with the ingredients and the cooking process you write it down and pass it on to your eager coders. Over time you collect a whole book of recipes that can be easily followed, producing code that is both repeatable and reusable. Software restaurants announce ticker tape openings around the globe to serve up cheap and reliable cookie cutter software solutions. Recipe for success? Actually, past experience tells us that more often than not it is a recipe for disaster.

The manufacturing process needs to guarantee repeatable results and this is very difficult given the amount of human intervention that programming typically involves. Of course, there are now many utilities that generate, parse, check, analyse and generally prettify source code but it's still essentially a solo practise between the developer and the IDE (apologies to all you XP pair programmers out there). There is also the issue of code maintenance to consider, generated code is not the prettiest of creations and, whilst it's fine when it works, any subtle imperfections are often very difficult to isolate and rectify.

Some people describe software development as an artform. Each contributor adds their own creative interpretation of requirements, applies keyboard brushstrokes to their electronic canvas and steps back to admire yet another unique creation. No two pixel paintings are the same; programmers the world over write different code to achieve the same objective. Even individual coders write the same routines differently, with or without varying degrees of standards and procedures, many times during their illustrious careers. An artistic masterclass? Very often this approach generates code that may be inventive but is seldom practical and rarely maintainable.

The problem here is consistency; code is produced that can be difficult to maintain where the layout of each class looks very different to its neighbours. There are a number of architectural decisions and considerations that a developer has to make that exist outside of what is generally perceived as the 'system design'. Does a particular piece of logic sit in the database layer or the business layer? How are exceptions handled? Should a method be instanced or static? Do you choose an interface or an abstract base class? In reality your code is simply a string of statements, loops and conditions but the potential number of combinations are astronomical even within a relatively small code base. Coding standards and recommended practises can help to reduce the confusion, code generation can produce generic classes and components and design patterns provide some level of guidance but dividing a limitless number of possibilities by any given factor still leaves a limitless number of possibilities.

So should we strive for a repeatable and consistent, painting by numbers development process where creativity and initiative are completely removed from the programming equation? Or do we promote flexibility and encourage our development teams to express their creativity and inventiveness in new and inspiring ways? Is there some magical, middle ground that satisfies the developer's creative aspirations whilst still adhering to repeatable templates, design patterns and agreed best practises?

This blog is littered with questions but unfortunately devoid of answers; each question corresponds to at least one, and often very many decisions. It is ironic that the decision making process is a key factor in making programming an interesting practise. It is also a subtantial contributor to the problems and complexities that lurk beneath the surface of even the most trivial of solutions.

Decisions take time ... so less choice, less freedom and fewer choices must surely facilitate a better programming environment...or do they?