Subjectivity masked as unbiased, scientific accuracy
From my speech at Public Consultation #4 in September. I thought it was worthwhile resurrecting and publishing it to reinforce some of the key messages:
———-
The recommendations of the consultants are based on a mathematical formula, giving the illusion of scientific accuracy. 91 subfactors have been diligently analyzed and quantified, each one feeding data into what I will refer to as the ‘bridge formula’.
A few examples:
- Mathematical models have been created to forecast daily traffic volumes in the year 2031 right down to the individual car and truck.
- Construction costs, which range from $360 million to almost $2 billion for one of the west end tunnel options, are precisely presented to the nearest $1 million.
- Ongoing annual operating costs are refined even further, and have been calculated to the nearest $100,000.
- Shorelines have been measured to the nearest centimeter.
The consultants have gone to great lengths to present a precise, defensible recommendation. One that they claim to be analytically sound and unbiased, and one that identifies Kettle Island as the best technical alternative based on a bridge formula that scores each alternative on a scale of 1-100…to two decimal places.
Very impressive.
As with any formula, the results are entirely dependent upon the values plugged into the formula. In the bridge formula, results are calculated based on utility factors and the weighting applied to each factor. Obviously, if the utility factors and/or weights change, the result of the formula changes.
I believe there to be flaws and omissions in the factors and subfactors that make up the bridge formula, and I am note alone.
A few examples:
- There are no factors to address the NEGATVE traffic impacts on people and communities associated with heavy traffic increases on residential roads in and around the corridor – every traffic factor is focused on maximizing traffic flow through the new corridor.
- Socio-economic benefits to both downtown and to the new crossing area are based primarily on maximizing traffic volume through the new corridor. This amplifies the already heavily weighted traffic flow factors. Furthermore, there is no consideration for land use within the corridors. According to the consultants, routing trucks through the communities along the Kettle Island corridor is good for economic development in the corridor…even though there are no industrial areas anywhere near the corridor!
- Cultural impacts from disruption and possible relocation of the RCMP stables and Musical Ride, a national institution and treasured facility in the heart of the city, are completely neglected.
- There is no measure of the negative economic impacts of closing the west-bound 417 exit at St. Laurent, effectively isolating east-end residents from the businesses along St Laurent, including the St Laurent mall.
- Ancillary costs such as potential relocation of the RCMP stables and musical ride, potential relocation of the runway at the Rockcliffe airport, and necessary mitigating measures for Montfort Hospital have been excluded.
- And one of the biggest oversights, I believe, is that there is no simple measure of the total population of people negatively impacted by the corridor.
I could go on, but what I really want to focus on are the weightings.
As we know, the weightings were determined by a closed-door technically oriented committee with no effective community representation. Based on the composition of this committee and the interests represented, it is not surprising that traffic considerations specific to trucks were given high consideration.
To the detriment of the people living in the impacted area and to the broader National Capital region, factors contributing to the negative impact on people and communities in the impacted areas were given low consideration.
Traffic and Cost factors represent 55% of the total score, while the impact on people and communities, only partially represented within the Cultural Environment (where, for the record, Kettle Island ranks 12th of the 12 crossing options considered), accounts for only 9% of the total score.
And make no mistake: If the weightings change, the results change. Without getting too far into the details, Kettle Island rises to the top of the list only as the weightings applied to people factors diminish, and as the weightings applied to Cost and Traffic factors increase.
Despite the precision of the consultant’s analysis and recommendations, it is critical to remember that there is a difference between precision and accuracy. For example, given the early stages of this analysis, I believe there to be a wide margin of uncertainty in the cost and traffic numbers, so I would argue that these number can’t possibly be accurate. Changes to these numbers in particular have a very real impact on the final scoring and ranking, given the disproportionally high weightings currently applied to these factors.
So as much as the consultants would like us to believe that this is an objective, unbiased study:
- Subjective decisions were made about which factors to include and which factors to exclude;
- Subjective decisions were made about how to quantify and measure each factor;
- Subjective decisions were made about who would sit on the weightings committee, and about how the weightings would be applied to each factor;
- Subjective decisions were made about if or how feedback from public consultations would be incorporated into the analysis;
- And ultimately, subjective decisions were made to put trucks before people.
At the end of the day, the final scores and rankings outputted by the bridge equation as it exists today are just numbers. These numbers should be inputs into the final decision, but they should not be the sole basis of the final decision.
The numbers need to be balanced with common sense.
And common sense says that trucks and people don’t mix.
———-
February 5th, 2009 at 3:02 pm
Great article…thanks. Who posted it?
February 5th, 2009 at 4:03 pm
Thanks Pat – It’s mine.
Christophe