For the subject of numbers around software projects Level 4 of the Capability Maturity Model (CMM) is da bomb: software as a Quantitatively Managed process.

A quantitatively managed process is a defined (capability level 3) process that is controlled using statistical and other quantitative techniques. Quantitative objectives for quality and process performance are established and used as criteria in managing the process.  (CMMI)

Wow. Project Number Fetishist Heaven! I thought I check it out for some nice entries to put on this site. Well, I got more than I bargained for¦ thats just too much¦ but it is great fun¦

Fun? you ask.

Well in some weird way¦ Think of all those students that after years of university can do all those quantitative managed things, just to find out that for them as a novice it is useless. They need experience to do all those numbers.

He, it says so in the CMMI description self:

One essential element of quantitative management is having confidence in estimates (i.e., being able to predict the extent to which the project can fulfill its quality and process-performance objectives). The subprocesses that will be statistically managed are chosen based on identified needs for predictable performance.  (CMMI)

Great. You have learned all this stuff, and you will be ^%^%^ by an old frustrated programmer that provides you with bogus over the top estimates. Should be frustrating

And¦

Another essential element of quantitative management is understanding the nature and extent of the variation experienced in process performance, and recognizing when the projects actual performance may not be adequate to achieve the projects quality and process-performance objectives. (CMMI)

You see that your budget is going down the drain, but you need real experience to know the difference between real panic or just normal project behavior (BTW thats normal behavior).

Yeah, numbers are a funny thing¦ they provide you with an aura of precision, but you need sooooo much interpretation to give them the proper meaning. Like they say: you have lies, more lies and statistics¦ or something like that :-).

One of the subscribers of my newsletter brought one old rule of thumb to my attention (thanks Terry!). I almost forgot this one.

It is known as the 1:10:100 Rule: the cost to fix a defect increases exponentially the later in the development lifecycle that it is identified.

Photography by Absolutwade.

• A defect caught in requirements phase costs a factor of 1 (1x) to fix.
• A defect caught in construction costs 10 times as much as in requirements.
• A defect caught in production costs up to 100 times as much as in requirements.

In this context it makes sense to define the test scripts as early as possible, during the requirements definition.

Or, as Terry put it into a mail to me:

Most companies currently decompose requirements into test cases for system testing during the construction phase. If they would simply move this same effort forward to the requirements development and physical design phases, they could save a lot of money!

However, I am wondering how current techniques, and tools are affecting this. The costs of creating testable applications (proof of concepts, prototypes etc) to use as feedback mechanisms are dropping, so I wonder if the 1:10:100 factors are still holding true.

Of course for large complex systems it will still hold, but for smaller business apps, I am not sure. But incorporating the test cases already with the requirements definition is always a good idea. End of course, making a change in a design documens is most of the time cheaper than changing a running system. But if those numbers are still exponential?

I wonder.

Most development projects spend effort in creating specifications: functional, technical, detailed, global. Putting the designs in writing takes a lot of work, and it will not be used in the end result; specifications are supporting artifacts.

So, the question if specifications are worth the effort is legit. Jeff Sutherland quotes some research in this area in his article Future of Scrum: Support for Parallel Pipelining of Sprints in Complex Projects

Two metrics are mentioned: the productivity of the development group and the defect rate. In a tongue-and-cheek definition, the productivity is the amount of features / lines-of-code build in a given time frame, and the defect rate is the number of bugs found per amount of features / lines-of-code.

The article of Sutherland mentions that there is a strong relationship between the completeness of the formal specification and the productivity. The better the sense of the developers is of the desired end product from a functional perspective, the faster the program.

The other way round with completeness of design specifications and the defect rate. The relationship between them is found very weak. So, writing more detailed designs before programming doesnt reduce the amount of bugs.

Leaving us with the question what can we do about defect rate and productivity. In the mentioned article the following options are provided for lowing the defect rate: early prototypes, design reviews and testing at code check in. Increased productivity is reached by early prototyping and daily builds of the software.

In my previous entry I discussed the underlying theories of plan-driven PM methods (based upon an article from Lauri Koskela and Greg Howell). I ended with the cliffhanger that I would inform you about the problems these theories generate… Tada… on with the show!

We have the theory of project, management-as-planning, the dispatch model and the thermostat model. So why does this combination suck?

What is wrong here? Well, actually all assumptions underlying this way of planning are causing problems.

Problem 1

Management is thinking all by themselves, all alone about how the job has to be done (central planning).

In a complex environment as software projects it is impossible for non-experts to know what the exact planning should be. Of course the project manager will be the negotiator that integrates all different aspects, but it is a team effort.

Problem 2

Management is thinking that the paper plan will automatically be executed in the way it is described.

People are not machines. You cannot throw an instruction to the bottom of the corporate food chain, expecting that the employees will pick it up and without questions will execute the instructions. No way. People need motivation, people need to be involved in the creation of the tasks they will have to perform.

I once wrote: "how to ruin a perfectly good process? Give an order." With the highly educated people we have in the software business they have a strong wish of participation when it comes to determine what and how they should do things. Also for the time frame in which they should do things they have a strong opinion. Not letting them participate in the "planning" stage reduces the commitment they have for the work to be done. Things like "not-invented-here syndrome" and "if you think you know better…" will ruin the project (In this way it is linked to problem number one).

Problem 3

Management is thinking the plan is correct (there is no room for deviations).

With all the uncertainties that are in a real project there is no way you can predict the future in advance. The plan will not reflect the reality just because the simple fact that the plan is wrong. This is no biggie, this is just as it is. If you try to keep on changing the process to steer reality towards the preset expectations, you are doomed.

A side effect of this is that people are most of the time judged by how well they comply to the preset goals in the plan. If you do not comply to the goals this is seen as a failure. The creative solution to counter this is just to rapport that you met the goals, even if you didnt. A few years ago I worked under chief of a project management office that always changed the progress reports in such a way that the overall performance of the PM office looked good. The report used smilies (yuk) to indicate any risks. Red smilies signaled something potential not under control, and you can have none of that!

Even if you dont manipulate the indicators, you would still use a large amount of your time explaining why you didnt comply to the preset plan. Where as the real reason (the plan sucked) is not considered an sufficient answer.

It is amazing how few Project Managers that are trained in a certain method actually know the underlying theories that make up Project Management.

Who cares? Whats the use?

Well actually, every PM should care. If you know the ideas that are behind a specific method, you will easier learn it, and, more important in my view, you see its limitations before you run into a brick wall.

Fair enough, this theoretic mumbo jumbo is not everyones cup of tea; so let me help you out by giving you my version of it all; and believe me, that version will have every scholar in Project Management do a back flip, but heck, it is just how I get it, and how I store that stuff in my brain; so, though luck for all of them¦ (grin).

Lets turn to plan-driven methods for a start. In essence, with such a method you should define every step that has to be performed in detail up front: the actual task, the time lines, the organization, and the procedures that should be followed. This will increase the predictability, stability, and high assurance of the process and the products it produces. You design a plan, you issue the orders, and you make sure everybody sticks to this original strategy. Examples are PMP and Prince2.

Lauri Koskela and Greg Howell describe in this paper what the underlying theories are for the plan-driven PM methods. This entry is based on how I parse their text in my mind.

First of all, there is not one theory that explains project management; it is a collection of several fundamental ideas, the theory of project, and theories of management.

Theory Of Project

The theory of project views tasks and operations as a transformation process. So, you have some inputs, a change happens, and presto, you get some outputs. You throw some garbage in, the team has a go at it, and you get some garbage out. You provide requirements specifications as input, the operation programming starts, and the end result is some running program.

Like some Russian Babooska (the little dolls that have little dolls in them) each transformation can consist of multiple smaller transformation. Requirements specification A, B and C are input, Programming A, B and C happens, and you get as output program A, B and C. The management principles behind this all use the fact that you can play with the inputs, outputs and decomposition of the tasks.

Theory Of Management

To describe the management part, three theories are needed: management-as-planning, the dispatching model and the thermostat model.

The idea behind management-as-planning is, that management sokes up all the information about the process, creates a detailed sequence of actions, with time and resources assigned, throws the plan to the operational level and yells just do what the plan says. This last part is the dispatching model: you issue an order down the chain of command that someone has to start on a task, and that will be it; the worker will automatically without any hesitation or problem start working on it.

If you have the management-as-planning view of the world you think that there is a direct relationship between what is on paper (the planning) and what happens in reality.

If you are creating a plan that will be executed blindly, you must be very sure that you know exactly what must be done; you must almost be able to predict the future. And that is exactly what is the appeal of this approach to management: it provides a sense of predictability (no surprises will occur) and you have the ultimate control of the situation; change the planning, and all the working people will change what they are doing. Paper is reality.

If the paper plan is right, than any deviation from the plan in reality is evil. Enter The thermostat model. Control is in this model nothing more than looking for reality to be not in line with the plan, and kick the real world back into shape, so it fits the plan again. You define upfront the desired situation, you put in the thermometer ones in a while into the project, and when you dont have the desired temperature, you correct the process until you have your temperature.

From a management point of view, this is a good thing; the process is nice and predictable and you have ultimate control. But next time, I will turn to the part were reality kicks in the butt.