Monday, March 3, 2008

Escalation of Commitment and Jonathan Swift

Escalation of Commitment (or irrational escalation) occurs when people persist in investing in an idea or course of action, despite an inflow of new information indicating that the idea or action is irrational. Psychologists and Organizational Behavioralists have been studying the phenomenon for quite some time; the table (click on it to enlarge) above summarizes several published studies (Mahlendorf, 2007).

Examples abound of ideas that were pursued to great economic loss, despite readily available rational information that could foretell the probable outcome. From a market research perspective, understanding why this information was not used -- or counter-information was fabricated to negate the rational information -- is central to the understanding how new insights from research can be exploited by organizations.

It is useless to attempt to reason a man out of a thing he was never reasoned into... J.S.

In response to these papers, some have argued that the largely emotional causes (optimism, selective perception, over-confidence, self rationalization, bias towards weighing project completion higher than the project value, etc.) do not adequately explain the behavior, and that the behavior at some level is rational.

What if both sides in this argument are correct: Escalation of Commitment can be irrational at the local level, but rational at the species, or super-organism level?

In a social species, genes guarantee a bias for following amongst the masses -- via tendencies that express themselves in feelings like the need for conformance, and to the extereme, in "separation anxiety". At the same time, there must be sufficient genetic variation to provide a varying amount of more aggressive individuals to lead. These innate desires may be in constant need of satiation; forming & maintaining social structures automatically.

It is difficult to argue, looking at the evolutionary success of species that seem to function as "super-organisms" -- ants & humans -- that these genetic biases have operated in a way that appears rational, contributed to this success.

At the local level however, where evolutionary competition is largely intra-species and typically between groups, the constant need for commitment on the part of individuals requires socially uniting hopes and goals to be generated at regular intervals.

This would imply that the most important attribute of a hope or goal is its availability when needed -- the extent to which a goal is perceived rational is secondary, and will be accentuated by the emotions involved in anxiety reduction and belonging to a group. A key to leadership is often keeping a simple goal or hope foremost in the minds of followers.

Faith! he must make his stories shorter or change his comrades once a quarter… J.S.

While there is no doubt that more rational goals (from an economic perspective) will be more salient -- stronger and longer lasting -- than irrational goals, the key for organizational and social stability may just be the presence of goals and hopes of any kind. For leaders, the key may be to keep goals in front of people, with the hope that some will be deemed successful at a later date.

I have always believed no matter how many shots I miss, I'm going to make the next one…. J.S.

From market research perspective, the questions then become:

When and where should research be done and communicated, to maximize the transference of rational information about customers and the market? Can it be funded in this manner?

If much of the funding for research derives from projects already undergoing Escalation of Commitment, can rational insights be incorporated into the findings in a way that they stand out from any biased information produced by directed research activities?

As market researchers strive to survive and thrive as a coherent function at the service of other organizations, what prevents them from falling victim to enabling irrational commitment, rather than ameliorating it?

Finally, with market researchers organized into groups, in competition with other such groups, how immune are they to also experiencing Escalation of Commitment, and in their roles as advisors to others, how much damage can result to not only their reputations, but their clients’ businesses as well?

Ambition often puts Men upon doing the meanest offices; so climbing is performed in the same position with creeping… J.S.

1 comment:

Anonymous said...

This is my favorite posting to the patterns and profits blog thus far from Spandrel. Perhaps it's because I share the initials of Jonathon Swift and the quotations chosen are ones that I'd wished I not only said, but believed and lived.
As with most of your posts, the questions posed at the end are, within the context of the pre-amble (no matter how important and weighty that pre-amble is), the most interesting, leaving us asking, "RIGHT! What's the ANSWER!"
The first question - can market research that is rational be funded - get to the core moral dilemma of the researcher: Honesty and Rationality, vs. Insuring there is a Paycheck next week/month/year.
The second question suggests a delicate balance, a dance, to weave the rational with the Escalation of Commitment, in such an artful way that future paychecks are enjoyed.
The final question - does the reputation of a market researcher suffer if they are not rational any more than it might suffer if they in fact weren't employed and engaged as a market researcher? Does the clients business always need to suffer from the support of irrational hope? Perhaps gain isn't maximized, but engaging in simple hope may, in fact, better insure the survival of the "species".

John S.