The Value of Success Stories

So far, I have focused on cost estimation failures. In my last post, I talked about the difficulty people have going from general results to specific cases.

I mentioned that the reverse situation is quite different. People have no difficulty going from a particular case to a general result. I found this to be quite important with regards to the performance measurement in defence and will talk, in this post, about two times when this occurred during my career.

The first involves the performance measurement conducted by the Department of National Defence that I studied in Cost Estimation and Performance Measurement in Canadian Defence: A Principal-Agent-Monitor Perspective . The government requires departments to produce a Departmental Performance Report in their annual budgetary “Estimates”.

I examined the Departmental Performance Reports for National Defence. My opinion of these Department Performance Reports was similar to the opinion of the Auditor General. These reports do not provide performance measures. They are instead a compilation of operational success stories from the past year.

The measurement of performance of a government department, and defence in particular, is very difficult. The question of whether the resources to conduct defence operations are used efficiently is nearly impossible to estimate since no counter-examples are available. However, the fact that operations were successfully conducted is undeniable and this positive impression of performance resonates with the public.

The second example that I worked on was when the Assistant Deputy Minister of Science and Technology wanted his staff to estimate the impact of the investment in defence research and development on the Canadian economy.  A number of approaches were attempted with inconclusive and sometimes contradictory results.   I was asked for an unbiased opinion on these approaches and their results.

I suggested that the impact of defence research and development on the Canadian economy could not be estimated macroeconomic terms as described by Vernon Ruttan in Is War Necessary for Economic Growth?: Military Procurement and Technology Development .

I suggested that the ADM focus on irrefutable success stories from investment of defence research and development. I noted that there are many irrefutable success stories from US investment in defence research and development, such as jet engines and aircraft, nuclear power and the internet to name just a few.

I realize that the investment in Canadian defence research and development has been more modest. However, there was a time from the 1950's to the 1970's in which Canada was a world leader in jet engines when building the Avro Arrow, hydrofoil ship technology with the Bras D'Or, shipborne helicopters for small destroyers, and shipborne helicopter landing gear with the “bear-trap”.

I am unaware of recent success stories in the Defence Research and Development Agency but expect there must be some. These success stories need to be well-publicized. If there are no success stories to be found in the Agency, I think there may be a question about the performance of defence research and development.