A Simple Matter of Software

The View From Mount Improbable

Penguin has published a nifty little book called "The View From Mount Improbable" in it Richard Dawkins explains how the eye evolved through a series of incremental steps.  This is an important argument to make, as creationists often use the eye as an example of an organ that is so complicated and so perfect that it could not have evolved but must have been created in one master stroke.  (Central concepts from Dawkins argument have analogs that are critical in planning software architectures, we will get to that  in a little bit, first the biology.)

Darwin himself admits the difficulty:

"To suppose that the eye, with all  its inimitable contrivances for adjusting the focus to different distances, for admitting different amounts of light,  and for the correction of spherical and chromatic aberration, could have been formed by natural selection, seems, I freely confess, absurd to the highest degree."

What Darwin, Dawkins, and the other evolutionists have done is show how the human eye was formed through a series of incremental steps, where each step required only a limited change, and each step resulted in a functioning organ.  In fact many of these intermediate forms still exist today in other creatures, the "non-directional light sensitive skins of starfish" are a good example of an early form.  While it is impossible to imagine that you could go from no eye to a fully formed human eye in a single mutation, it is possible to imagine a single mutation triggering each of these individual intermediate steps.  And since each intermediate step is a functioning organ that has at least slightly improved upon its predecessor it is easy to see how natural selection would spread that new version through the population.

In software development we also have arguments about what is the most plausible way to create something that is complex and perfect.  Two typical options can be thought of as the "creationist" approach and the "evolutionist" approach:

1. Creationist: Assign a very smart design team.  Have them create an elegant design that fulfills your full concept for the product.  Immediately start to implement this final form of the product.
      
2. Evolutionist:  Lay out a plan for series of intermediate versions of the product.  Carefully design the first step  (keeping in mind that the codebase will be evolved and reused, so put emphasis on compartmentalization, factoring, and good interfaces), don't take the time to fully design the later steps.  Make each intermediate step something that is deployed with users, or at least with beta users.  Be prepared to adjust the plan based on feedback you get at each intermediate step.

The "creationists"  argue:

• It takes too long to make all these intermediate versions.
• What is the point of writing a bunch of code that will only get reworked or replaced as you evolve the product.
• Intermediate versions lead to legacy problems (interfaces you have to support, migration plans you have to implement, etc.) that you will regret.
• The competition is not going to waste time with these intermediate versions.  They will leapfrog us.
• Why limit yourself to designs that can be reached through incremental steps, what if the "best possible design" can not be reached through incremental steps.

The "evolutionists" argue:

• Not understanding the real requirements is the biggest issue.  Users themselves typically don't know what they ultimately want.  Only getting something in use can resolve this issue.
• It is only after you have written it once that you really understand how to write it right.
• Naval gazing in your conference room does not lead to true understanding.  Putting software into use, careful analysis of the reaction, and rapidly evolving the offering is what leads to true enlightenment
• Having working intermediate versions gives you more flexibility when it comes to the point of making the features vs. schedule trade-offs.
• Working intermediate versions improves your ability to get continued funding.  This is true whether your funding comes from a VC firm or a senior management budget committee.
• The goal is not finding the "best possible design", the goal is finding a great design that meets your audacious goals with minimum risk.

Which approach is best depends on your specific circumstances: scope of the project, relevance of team's previous experience, clarity of requirements, etc.  However, most of the time I am an "evolutionist". 

Smart people tend to over-estimate their ability to go directly (via their own mental horsepower) to the correct final answer.  Usually getting the optimum offering requires interaction and evolution.

Let's return to biology for a moment.  The eye is not perfect.  Remember those exercises you did in Junior High science class, they proved that we all have a blind spot, none of us can see in the infrared and some of us are color blind.  A perfect design would not have these traits.  However, as Darwin pointed out the eye is a great design.  So great that many believed omnipotence was required to create it.  I believe it ended up great, because each modification was tested in real use before it was incorporated in the final result.  This lesson is worth keeping in mind when you go to create your next great design.

Previous posts touched on some related topics: why great is often better than perfect; why we so often miss the real requirements; why interfaces are more important than code; and why a "truly complete" design is not what you want.
                                                                                 copyright 2007 Kerry Champion

Fractal Geometry and Market Analysis

This post on "What is Marketing" got me started thinking about markets and common mistakes by entrepreneur's.

In a software venture it is very common to talk about "the market".  As in:

• No point entering such a well established market
• This is an early stage market
• We can have first-mover advantage in that market
• That market is not big enough for us
• This market is growing at 20% a year
• 40% of the customers in this market are actively looking for alternative solutions
• We are the clear leader in our market
• etc. etc. etc.

These statements are treated as absolute truths, when in reality their truth or falsehood is completely dependent on how you define the boundaries of "the market".

Often in these discussions my mind drifts to comparing fractal geometry and market analysis.  They are pretty different disciplines, but they do have one important trait in common.

A definition of fractals including some gorgeous pictures can be found here.  In brief, a fractal is a geometric shape.  Such as this one:

One of defining characteristics of a fractal is self-similarity.  Meaning that as you zoom-in and zoom-out you can see the same pattern repeated at different levels of granularity.

Fractals occur in nature as well.  For example the coastline of a continent has various bays, points, deltas, inlets, etc.  Then if you look at the coastline of one of those bays, you will see more bays, points, deltas, inlets, etc. just at a smaller scale.  And so on.

Markets do not normally form fractal patterns.  As you look at a product's market at different levels of granularity you may see very different patterns with each analysis. 

However, markets and fractals are similar in one important way.  In both cases doing your analysis once from one point of view is not enough!  In thinking about the market for your software venture you must look at the market patterns from many different points of view.  Until you do this you won't know what you have got.

For example, lets say you have a great idea for a massively multi-player (MMP) real-time strategy (RTS) game that is set in Europe in WWII.  Well what is the market that you should analyze when making your business plan and when deciding who is your target user?

1. market for European Theater WWII RTS MMPs
2. market for WWII RTS MMPS
3. market for RTS MMPs
4. market for combat MMPs (including both RTS and first person shooter)
5. market for all MMPs
6. market for all computer games

It is easy to see that your business and usability analysis will be drastically different depending which of these target markets you chose as the basis for your analysis. 

The above is an example of different ways to slice the market based on changing scale (as you would in looking at a fractal).  However, there are plenty of other ways to slice and dice markets beyond fractal-style scale changes.  The key lesson from fractals is not that scale rules all. Rather the lesson is that doing your analysis just once with one set of assumptions is not enough to answer your questions and tell you what you are looking at.

However, it is important that you don't get bogged down in analysis paralysis by trying to analyze every possible market definition. You should do more than one, but you should not do them all.

A good quick way to find what market definitions are most interesting is to do the following:

• Pick 3 to 5 comparable products
  Make these products as diverse from one another as possible, while still having at least one key trait in common with your product.  So in our example above you might pick: a non-MMP WWII RTS game; an MMP first-person combat game; and an MMP non-combat game as you comps.
     
• Find customers
  Find a half-dozen to a dozen people who have recently bought examples of each product
        
• Identify alternatives
  Ask each of those purchasers "when you were buying this product, what alternatives did you consider?  In other words if you had not spent the money to buy XXX what would you have spent that money on instead?"  In some cases you will get an answer like "I would have spent more money on my Mom's birthday present." which is not too useful.  But in other cases you will get very useful answers such as "I gave up my Everquest subscription so I could spend the money on World of Warcraft"  or "Well if I wasn't paying for World of Warcraft every month I would probably get the latest expansion pack for Age of Empires."

The point here is that what really makes the "interesting" market definitions is the customers' opinion about spending trade-offs.   If the customer has a certain amount of money set aside for monthly MMP fees, then the market is for MMPs.  If the customer has a certain amount to spend on all computer games, than that is the interesting market definition as they are making trade-offs among those choices.  If the customer is really making an impulse purchase without considering trade-offs than that is also interesting data, at minimum it will impact your pricing , advertising and distribution schemes.

The above mechanism for picking a market definition is not the right one for all circumstances.  So if you have a better approach use it.   The key thing is to use some kind of analytical approach that will encourage you to be open-minded to new data and new conclusions.  Too often market analysis is an exercise in reaffirming your existing opinions through careful selection of target market and relevant data.  Try your best to avoid this sort of wish-based planning.

Previous posts talked about the dangers of picking the wrong market, what gets in the way of listening to the customer, and how to deal with an unpleasant truth when you discover it.

                                                                                 copyright 2007 Kerry Champion

Not Seeing The Gorilla - User Requirements (part 1)

You might think that entrepreneurial software ventures are pretty much guaranteed to do a good job of listening to the customer.

You would be wrong.

Whether they are in a start-up or doing an innovative new project within a big company, folks working on software ventures are just as prone as anyone else to:

• Fall in love with their own ideas
  Something that welled up from the mists of their frontal lobe, is always more appealing than an idea that was tediously sifted from lots of conservations with strangers
   
  
• Fall in love with the early ideas
  When searching for an answer humans tend to latch on to the first reasonable answer (using a low barrier of acceptance) and then use a much higher standard when considering other possible answers in comparison to this initial answer.  Fondly known as "early latching syndrome", this may be a good thing in breast feeding, but not such a good thing in problem solving.
     
• See themselves as experts and users as uninformed.
  Folks in a software venture are way smarter than average and spend 100% of their time thinking about the issues around their target offering.  Users are average smarts and typically spend a tiny fraction of their attention around the issues associated with this offering. It is easy to assume the developer is right and the user is wrong.
     
• Assume that bad solutions mean an irrelevant problem
  What really matters is the users' (not the developers) definition of what is the real problem and what trade-offs are worth making.  However, users often (not always but often) are bad at proposing specific solutions (functionality, UI, architecture, etc.) to address those problems and implement those trade-offs.   Unfortunately once a developer hears a bad solution suggestion they often stop listening and don't bother to dig for the underlying problem the user is trying to address.  Finding the problem the developer might suggest an alternative solution that the user would love.
     
• Act as rationalizing machines
  What distinguishes humans from animals is not speech or tools, but our extraordinary ability to rationalize.  We are amazingly good at providing a rationale sounding analysis of a given set of observations, an analysis that just happens to support our pre-existing biases and assumptions.
     
• Suffer from situational blindness
  When stressed and really focused on a particular goal we can be blind to obvious facts present in front of us.  There is an fascinating experiment with two basketball teams and a gorilla, that illustrates this.  More relevantly I am sure we have all seen a more subtle form of this in software projects as well.
  

All of this gets in the way of really hearing the key messages from the prospective users.

I suspect we have all seen software ventures that have:

• Made it more complex than the customer wants and undervalued the power of simplicity
• Failed to tie technical innovations to customer valued differentiations
• Targeted the wrong customer for their offering
• Made bad assumptions on the rate of adoption
• Spent time and money on development efforts that were irrelevant to the user
• Suffered from NIH syndrome, when user preferences would have driven them to leverage existing technologies

All of these issues (and more) can be alleviated through a real commitment to understanding the user and finding the key messages.

Most of the time I believe the needed data is readily available through some basic legwork.  The real issue is overcoming our biases and wishful thinking. 

As in the situational blindness experiment, that gorilla is right there in front of us.  We just have to break out of our stress induced tunnel vision and apply the needed judgment and perspective.  Then we will see it.

Later posts will further explore how to capture user requirements.

Previous posts talked about when not to listen to the customer (here and here), how to deal with an unpleasant truth when you discover it, and the dangers of doing the right things (such as careful user analysis) to solve the wrong problem.

 

                                                                                 copyright 2007 Kerry Champion

Lilliput vs. Innovation

There are lessons for the software adventurer everywhere, even in Gulliver's visit to Lilliput.  As you might recall the story begins with a violent storm that only a select few might survive.  After great exertions Gulliver saved his own life while others drowned.  Eventually he reached a remote beachhead and crawled up on shore.  Very tired from his efforts he laid down to sleep.  When he awoke he

"attempted to rise, but was not able to stir for, as I happened to he on my back, I found my arms and legs were strongly fastened on each side to the ground; and my hair, which was long and thick, tied down in the same manner. I likewise felt several slender [strings] across my body, from my armpits to my thighs. I could only look upwards; the sun began to grow hot, and the light offended my eyes. I heard a confused noise about me, but in the posture I lay, could see nothing except the sky."

Gulliver had been tied to the ground by the six inch high Lilliputians.  Dozens and dozens of thin strings held him securely so he could not rise or even turn his head.  Hundred of Lilliputians with their needle sized arrows held him under guard.  These strings and archers were deceptive, each one was of no consequence but together they robbed him of the ability to act.

In the software world a similar scenario plays out except:

• Instead of "violent storm" you have an "exciting new market opportunity"
• Instead of a literal "beachhead" you have a figurative beachhead of your "initial customers and revenue"
• instead of "strings and archers" you have "specialized product requirements to close opportunistic sale"

You want to make it to your beachhead without entangling yourself in too many constraining commitments too early in your product's lifecycle. 

Some of these commitments may very well make sense when the product is more mature and you are extending a proven success, but don't make sense when you are still innovating and trying to break out from your beachhead.

All of us in the business have seen this happen:

• You have a crisply defined initial target market and product specification
• Your initial market is  big enough to have a range of potential launch customers, but small enough to let you focus
• You have good ideas about how to broaden from your initial market adjacent markets and usages as you mature over time
• Your initial product specification is constrained enough to be a tractable implementation challenge
• You have carefully designed your features and architecture so you can expand your capabilities over time
• Your first set of launch customers is happening a little slower than you would like, you happen to run across a specific customer or related market that you could pursue if only you supported: a non-standard browser, or ported to the partners server and added "powered by XXX" on your homepage, or added another localization, or added a specialized feature, or provided a temporary back-door for a customer specific integration or did some other opportunistic change that is not really aimed at your target user.

One of the advantages of a dynamic new venture (startup or in-house) is the ability to be flexible and adjust rapidly.  However it is important to distinguish between consciously changing the fundamental plan based on new information vs. chasing each new opportunity as it appears without really fulfilling any of them.  The former can be a very sensible thing to do.  The later leaves you with dozens of specialized product requirements and without a critical mass of similar users.

No one of these "specialized product requirements" is that much of a burden, but all of them together are like an army of small archers and entangling stings.  They limit your ability to act and stifle innovation.

Related Posts: discussed speaking out when you see trouble ahead, highlighted the dangers of accelerating towards the wrong target, and stressed the need to really understand what your must critical questions are rather than just reacting to the short term issues in front of you.

                                                                                       copyright 2007 Kerry Champion

One thing worse than no specification

With apologies to Oscar Wilde let me suggest

There is only one thing in life worse than having no specification at all, and that is striving for a truly complete specification.

The flaws of having no specification at all are fairly well documented.  Joel has his take on this point in item 7 here.  I have a somewhat different take on this topic that I will touch on in a later post.

Here I would like to discuss the opposite problem: not of too little, but of too much.

For some the problem is not immediately obvious.  If "no spec" is bad, than a "30 page spec" must be better, and a "300 page spec" must be better yet, and a "truly complete" spec that leaves no unanswered questions must be the best.  While that is an intuitive notion, it does not work that way in reality.

Let's start by discussing what it would mean to have a truly complete specification.  That would certainly cover:

• Design rationale
  Why this is worth building.  Why are we taking the current approach.  What alternatives were considered for the top-level design.
• Interfaces
  Every external interface.  What conventions and rules were applied in designing these interfaces.  Where do we expect future expansion of the interface.
• Implementation
  All internal interfaces.  All data structures.  Design and coding guidelines.
• User Interface
  All user interfaces including all conditional and implicit behaviors
• Test plan
  Complete plan for both unit and system tests.

That all sounds sensible doesn't it?  Why shouldn't we always write a complete spec that covers all the above to a level of detail that answers all questions?   Well here are some reasons:

• A blanket requirement for such a spec in all cases would stifle innovation, which in its early stages sometimes plays out best in code rather than in documents.
• A truly complete spec would certainly be longer (in terms of pages, characters, time required to read and understand) than the code it is intended to specify.
• It would take longer to write a "truly complete" spec than the combined time it would take to write a "good overview spec", a paper prototype of the UI, and the code itself.
• Keeping a "truly complete" spec and the resulting code in sync (through the normal multi-release life cycle) would be a nightmare
• It is harder to get intermediate validation on a monster spec that is evolving over time, then it is to get intermediate feedback on a combination of overview spec and code.  This is true for all aspects of the design but is espeically true for performance/scalability issues.
• Iterative development is tough if you have a goal of a "truly complete spec" before any code.
• It is just not as much fun :-)  Evey one gets fidgety if they are asked to spend months on design and spec without writing any code to test out those ideas.

So what are some rules of thumb that will help you strike the right balance.

• Anything that can be documented in the code itself should be documented in the code itself.
  Design rationale, test plans, UI are hard to put in the code itself.  However, interface definitions, data structure definitions, and algorithms certainly can be.   
• Never put the same information into two different documents.
  Cross reference where that makes sense.  If needed buy or write a utility that will extract the annotated sections from the source document (very often the code itself) and put it formatted into the target document.
• Define a goal for the spec
  Make that goal be a reasonable scope.  For example explicitly say the goal of the spec be to resolve the big issues that have broad ramifications, and to build a common understanding among the team; but it is not a goal to answer all questions.
• Test the specs completeness in live reviews (not just document mark-up)
  In doc review we are all nitpickers and we often lose site of the bigger issues.  Live reviews (in-person, phone, or IM) are a good way to see how far you have come in defining a common understanding
• Use change-bars
  So you can see how the documents are evolving with each version.
• Take note of diminishing returns
  When the new value (key questions answered, new insights captured) starts to fall per unit of work (another edit of the doc, another review meeting), than consider that you should declare victory and move on.

When Engineers get focused on solving a problem and internalize it as a task worth doing, they have a tendency to strive for "perfection" or at least "exhaustive completeness".   This makes it surprisingly  easy to fall into the too much spec trap. 

As Professor Moody likes to say "constant vigilance" is the only answer ;-)

                                                                                       copyright 2007 Kerry Champion