A Simple Matter of Software

Saint-Exupery on Engineering Elegance

"A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away."
                                - Antoine Saint-Exupery

Design principle from creator of first Wiki

“What is the simplest thing that could possibly work?”
                                                              - Ward Cunningham

Agile Manifesto - Annotated

I have been thinking recently about the Agile Manifesto. 

Here it is with some annotations. 

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Right on!  This gets to the heart of the "adaptive" vs. "predictive" debate.  The post here makes a similar point phrased in terms of "evolutionists" vs. "creationists" using analogies from Richard Dawkins.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Changing requirements is a reality and you need to embrace that and learn to succeed within that reality. 

However, I have two concerns with the way this particular item is presented. 

First "welcoming change late in an iteration" can directly conflict with your ability to achieve "early and continuous delivery of valuable software".  The manifesto leaves it as an exercise for the reader to figure out how to balance those conflicting desires.  It can be done, but there is no guidance here how to do it.

Second recognizing the reality of changing requirements doesn't give you a free pass to put off detailed requirements definition.  You still want to push hard to define the requirements as early as possible and then only change them based on "new information". Thrashing a decision because of internal indecisiveness is a bad idea, adapting as you get new insights into what works best for the user is a good idea.

The good thing about short iterations is that you can "freeze" and "reopen" requirements quickly (with each iteration).  Still at some point you have to at least temporarily freeze the requirements to get the iteration out the door.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Right on! This is a direct corollary of point 1 above.  Of course some initiatives will take longer than a "couple of months", but that does not mean they can't deliver working software along the way.

Business people and developers must work together daily throughout the project.

Right on! 

Engineers working in a vacuum will produce some beautiful code, which may be irrelevant to the problem that needs to be solved.  In the case where the engineers themselves fit the target audience  (programming tools, search engines) then engineers can produce great things without the business folks.  But those cases are the exception not the rule.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

100% agree.

However, when you have multiple motivated individuals they often don't agree on how to proceed forward.  The manifesto leaves as an exercise to the reader: how do you get alignment and get past such disagreements.   

The most efficient and effective method of  conveying information to and within a development team is face-to-face conversation.

It is true that face-to-face is the most powerful way to communicate an idea and it is true that face-to-face is often underused.

However, there are plenty of times when face-to-face is not the best option for "conveying information" especially information that needs to survive for a period of time. 

Tribal knowledge is a powerful thing, but if all information only exists as tribal knowledge in peoples' heads than you will have some real issues in achieving "sustainable development" (see two items below).

Working software is the primary measure of progress.

Right on!  Such a simple but powerful concept.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

100% agree.

Continuous attention to technical excellence and good design enhances agility.

100% agree.  An interesting question here is how you go beyond "attention" to "measurement".  It is much easier to keep your attention focused when you have some measure of how well you are doing.

Simplicity--the art of maximizing the amount of work not done--is essential.

Right on! 

It is easy to state the value of simplicity, it is harder to apply it in practice.  Though Dilbert had a good example of simplicity in action that has some lessons for the software world.

The best architectures, requirements, and designs emerge from self-organizing teams.

This is true. 

However, it begs the question: what do you do when the team fails to "self-organize"?  Give up and go home?  Hopefully not.  How to catalyze "self-organization" is left as an exercise for the reader.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

100% agree.  Any complex task that is worth doing demands a certain amount of self-reflection.

By the way, one implication of this last point is that you should make your behavior suit your particular needs and experiences even if that means not exactly following agile development method as described in the various books, training classes, etc.

                                                                               copyright 2007 Kerry Champion

Are you agile?

I ran across an amusing checklist to measure "Agility" in the agile programming sense.

Don't agree with every single point implied by this list, though most rang true.  Here are some favorites.

======================================================================

You are not yet Agile if:

“Test-driven” still refers to your car.

Developers only develop, testers only test, and managers just manage.

Simplicity is presumed to be simple.

======================================================================

A more serious checklist to measure agility is available from the same source here.

Don't just test, change how you code

When deciding on your testing plans and your process for handling bugs, keep in mind this question:

Which practices will go beyond reporting a snapshot of current quality?  Which practices will drive positive changes in how we develop going forward?

One unfortunate pattern you see in many software teams is:

1. Have a set of programmers who are responsible for making new functionality appear as quickly as possible.
2. Have a set of testers who are responsible for quality.
3. Have the programmers "hand off" new features to testers as soon as possible, so they can move on to the next new feature they need to implement.
4. Don't have testers interact with programmers (review design, do preliminary testing, create/review test plans) until the feature is 'done', and even then have minimal interaction.
5. Have the primary communication between testers and programmers be the bug database.  The only real 'output' of the testers is to create bugs in the bug database.
6. A manager will prioritize the entries in the bug database and an engineer will be assigned to resolve each high priority bug.
7. Metrics are kept for inflow, outflow, total bug counts for each priority level.  But other than that no analysis is done to look for patterns in the bugs.  Does a bug represent one of a common type? Are parts of the codebase much more prone to having bugs reported against them?  Typically those questions are not answered.
8. If there is a concern that the quality is not high enough, the reaction is to increase the number of testers and the volume of testing.

This approach is far from optimal, of the various issues here are three that stand out:

• 'responsibility' for quality should be on everyone not just on the testers who measure it.
     
• this process reinforces a "reactive" approach where the team is driven by what is "urgent" (handing off the current feature, responding to the bug report) rather than what is "important" (improving your skills and practices to increase the baseline productivity and quality of the team going forward).
     
• this process has no feedback loop that drives the team to improved results

I do believe in having dedicated testers and I do believe in the importance of a bug database and I do believe in tracking bug counts.  However, those things by themselves aren't enough.  You need to go beyond these things to implement QA practices that drive continuous improvement in how you develop, and not just improvement in how you test.

What specific practices make sense depend on your circumstances: size of team, maturity of developers, nature of software being developed, etc.  Here are some possibilities to consider:

• Require automated unit test to be delivered by the developer before new code is considered 'done' and ready for integration. This does not just result in a useful test, it makes developers think harder about their implementation decision and about how their code may be used.
   
  
• To identify potential side-effects require a critical sub-set of all automated tests (not just the ones for the new functionality) to be run as part of the check-in process.  Require that check-ins be held until these tests pass.  This does not just result in fewer broken builds, it encourages developers to think about cross module dependencies.
   
• Require that the entire universe of automated tests be run as part of daily (or more frequent) build.  If any test fails have the engineers who checked-in during that window stop their work and fix the build before continuing.  This practice reinforces the message that it is more productive to fix bugs in existing code first, before adding new code.
   
• Measure code coverage generated by automated tests on a daily basis.  Have specific numerical goals for target code coverage.  Have developers (not testers) be the ones responsible for achieving those goals.  This does not just ensure that you have a robust test suite, it also pushes developers to go back and remove redundant code and to simplify the code that they keep.  Both those steps in turn make the code easier to understand and to maintain.
   
• For bugs found after the developer believes the code is done, in the bug database track the nature and resolution of those bugs.  What part of the codebase was it in?  Was it a logic error, a syntax error, a misunderstanding of the language runtime behavior, a misunderstanding of some public interface, a misunderstanding of requirements, a performance deoptimization, etc?  Look for patterns in this info, share it with the team as a whole and see what conclusions can be drawn.
   
• For each bug in the bug database that was found through testing, ask the developer who resolves it "is it likely that there are similar undiscovered bugs in the codebase that could be found via inspection/search/analysis?"  If yes, make a specific trackable work item to do that inspection/search/analysis.
   
• For a sub-set of the bugs in the bug database have the relevant engineer go in front of the team and present the "lesson learned" from that bug.  Such presentations may be short (5-10 minutes, 1-3 slides) and tagged on to some other meeting that is occurring anyway. A steady stream of such presentations (3-6 a week) is best.
   
• For each non-trivial check-in have the developer write 1-5 bullets that answer "what is the most likely problem to be triggered by this change?" and 1-5 bullets on "suggested system testing to exercise this code."   These bullets can be very terse, it is ok if someone who wants to take action based on one of these bullets has to go back and ask the developer some questions.  However, even the tersest comments, made at the moment that that code is freshly in mind, can be incredibly helpful.
     
• Have developers take part in system testing.  It would be a mistake to have every developer take part in every system test.  However, it is also a mistake for developers to never help exercise the system as a whole.  They need to understand in a tangible way the system as a whole and not just their piece.  Also having a sense of the challenges facing the testers will drive them to pay more attention to the quality of their code and to do a better job of communicating with the testers.

There are some common threads in the above practices.  One thread is to get developers to foresee where bugs are going to be and resolve them before that bug is found by testing.  Another thread is to move away from "whip it out and hand it off" approach and have practices that encourage developers to review their code multiple times from multiples perspectives.

                                                                                 copyright 2007 Kerry Champion

Development via trial and error

My daughter has a t-shirt that says on the front:

There are 10 kinds of people in the world.

On the back it says:

Those that understand binary and those that don't.

She is in Junior High and thinks this is hilarious.  It has been a long time since I was in Junior High but I also think it is hilarious.  I love it when you can make good use of base-2 nomenclature. 

In any case, on to the real topic. 

There are 10 kinds of programmers in the world: those that code via trial and error; and those that don't.

When I first started programming as a teenager I was a trial and error coder.   Luckily I later worked with some consummate professionals who showed me a better way.

Unfortunately lots of folks who are nominally professional software engineers still use the trial and error approach.  The following is a typical scenario:

1. Grab some existing piece of code that has some overlap with what you are trying to accomplish
2. Modify that code so that it is at least somewhat in line with your current goal
3. Try to compile it, not really expecting it to work
4. Sure enough it has syntax errors.  Don't bother to re-read the all the code, just go in and fix the individual errors as reported by the compiler
5. Assume the compiler/linker errors/warnings are very accurate.  Only as a last resort go and pull the language or linker book off the shelf to read anything about required syntax and recommended usage.
6. Repeat steps 3-5 until you have a clean build
7. Now run your code, not really expecting it to work
8. Have some basic test that illustrates the most common use case for what you are trying to implement.
9. Manually execute that test.
10. If that test fails (which it almost certainly will the first several times), don't bother to go back and re-read all the code with this failure in mind.  Instead single-step through the debugger to see where things go off the tracks. 
11. As you step through using the debugger you will eventually realize that something is wrong.  Whatever code you are looking at when you make this realization, assume that is the code that needs to change.  Go in and modify that code to handle the current failure case. 
12. Don't consider the possibility of refactoring to solve the problem.
13. Always assume you must add more code to fix the problem, don't consider the option of simplifying the logic to address the issue.
14. Don't spend much time thinking about the side effects of your change, assume it is easier to catch those via testing than by thinking hard about logic and data structures.
15. Don't bother analyzing the error to see if it is just one example of a more widespread issue.  Assume any analogous errors that exist elsewhere in the code will be found via testing, don't bother trying to find them via inspection.
16. Now repeat steps 3-11 until your test case passes.
17. As soon as your test case passes, declare victory and stop work on this code.  Tell QA to start testing the functionality.
18. Don't bother to write automated unit tests or to do code coverage analysis
19. There is an urgent need for you to go back and debug problems in your previous code (that you wrote using this same approach).  Assume this need to fix what supposedly was working fully justifies your decision to not spend time on: reviewing your new code with others, refactoring based on new insights, executing multiple test scenarios, developing automated tests, searching for analogous bugs via inspection, etc. etc.

The problems with the above approach should be obvious.  However, it is shockingly frequent that this is exactly how commercial programming gets done.

Sometimes this is because the engineer is simply incompetent, however much more often it is other factors that drive this behavior.  For example:

• Management consistently sending the message "what really matters is getting new functionality 'working' as soon as possible" and never sending the message "sound programming practices will pay off in the long run and deserve our attention and time"
     
• The normal human tendency to prefer being driven by what is urgent, rather than basing our efforts on proactively defining what is important.  To break out of this habit requires conscious support from the leaders within the team.

I think it is critical to explicitly talk to the team about why "trial and error" programming is a bad idea, to set up practices that encourage a more mature approach, and to make it part of your recruiting process to screen out this type of programmer.  I will talk more about these last two points in subsequent posts.

                                                                                 copyright 2007 Kerry Champion