Unintended Consequences – Part 1

400 400 Artificial Intelligence in Medicine Inc.

Part 1 of 2

Unintended Consequences

I graduated from the University of Toronto, faculty of Applied Science and Engineering in 1986. As a Canadian graduate, I was able to take part in a uniquely Canadian ceremony—the Ritual of the Calling of an Engineer—also known as the “Iron Ring” ceremony. Wearing our black robes, we were escorted to a large hall at the center of which lay a mangled piece of metal. A heavy chain was bolted to the metal scrap and arranged around it in a large circle. Each of us took positions around the circle and picked up the chain. We then recited the Obligation of an Engineer (akin to the Hippocratic oath) and swore our oath on “cold iron”. The piece of metal to which we were chained was a remnant from the “Ocean Ranger”, an oil drilling platform that had sunk during a storm off the coast of Newfoundland killing over twenty people. The iron ring ceremony forces graduates to face the consequences of human fallibility in a starkly dramatic way. At the conclusion of the ritual, each participant is given an iron ring to be worn on the smallest finger of his/her working hand for as long as one practices engineering. That ring is not only a symbol of fraternity, but a constant reminder that no matter how well trained and how diligently we plie our trade, things can and will go wrong, sometimes at a dire price. I have been interested in why things go wrong ever since.

Have you ever heard the aphorism “The road to hell is paved with good intentions”? It means that wrongdoings can be disguised by good intentions, but more often, it means that acts carried out with the best of intentions can have unintended consequences—unexpected outcomes that can sometimes be disastrous. The saying traces its roots to ancient times. In the epic Aeneid, the poet Virgil wrote “The gates of hell are open night and day; smooth the descent, and easy is the way.”[1]

[1] Aeneid, Book 6, Lines 124-141, Translation by John Dryden 1697

Unintended consequences fall broadly into three categories:

  1. A Paradox: A result that is contrary or opposite to what was desired.
  2. A Drawback: An unexpected detriment occurring in parallel with the desired result.
  3. A Benefit: An unanticipated positive result which we might call luck, serendipity or a windfall.

Let’s look at some real-world examples.

Paradoxical Outcomes

The most unexpected consequences of purposeful action are those that seem counter-intuitive, even perverse, in that the results are often completely contrary to what was intended.

The Streisand Effect

One example is a phenomenon referred to as the “Streisand Effect”, an example of a psychological reaction to the attempt to hide or censor information. It is named after an incident involving Barbara Streisand concerning her residence in Malibu, California. In 2002, a photographer, working for the California Coastal Records Project took 12,000 pictures of the California coastline to document coastal erosion. The photos were posted on the project’s website. One of those photos showed Streisand’s Malibu mansion. In February 2003, the project received a cease-and-desist notice from Streisand’s attorneys claiming infringement of privacy. This was followed by a $50 million lawsuit. Up until this point, sixteen people had viewed the photograph. However, following the initiation of legal action there were 420,000 views.

The Streisand effect is an example of immediate motivation of interest—in this case, keeping secret the location and appearance of Streisand’s residence—without considering the human emotional response to the attempted act of censorship. Instead of suppressing the information, the attempt to hide it resulted in mass disclosure—exactly the opposite of what was intended.

The Photo that Launched the Streisand Effect (Image Credit: By Copyright (C) 2002 Kenneth & Gabrielle Adelman, California Coastal Records Project, www.californiacoastline.org, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=22417977)

Risk Compensation

“Risk Compensation” is another example of paradoxical consequences. It is a type of behavioral adaptation that works oppositely to the intent of safety measures taken to reduce risk in the first place. It is sometimes called the “Peltzman Effect” in recognition of Sam Peltzman, an economics professor at the University of Chicago, who wrote an article in 1975 titled “The Effects of Automobile Safety Regulation”, in which he claimed that offsets due to risk compensation negated the effects of safety regulations and resulted in no net decrease in highway deaths.

Here’s how the Peltzman Effect works. Consider anti-lock brakes in cars. These are designed to increase vehicle safety by reducing skidding and allowing drivers to maintain control on slippery surfaces. However, various studies have shown that drivers of vehicles equipped with ABS drive faster, follow other cars more closely, and apply brakes later[1]. Consequently, the adaptive behaviour of these drivers results in almost no measurable improvement in road safety or decreased accident rates.

[1] “An exploration of the offset hypothesis using disaggregate data: The case of airbags and antilock brakes”, Winston, C., Maheshri, V. & Mannering F., Journal of Risk and Uncertainty, March 2006, Vol 32, pp 83-99

There is an interesting corollary to the Peltzman Effect. If safety measures result in adjusted behaviour that opposes the benefit of the safety measures, what would happen if safety measures were removed? This question was on the mind of a Dutch traffic engineer named Hans Monderman in the 1960s and 1970s. He posited that an individual’s behavior in traffic was more likely to be affected by the built environment rather than traffic regulating devices such as curbs, sidewalks, road surface markings, signs and signals. To this end, he proposed a design approach called “shared space”, where demarcation between the right of way for cars, bicycles, and pedestrians is entirely removed.

Shared Space

To understand how the shared space scheme works, it is useful to compare it to the traditional approach to traffic engineering. Queens Quay in Toronto, Canada, along the shore of Lake Ontario is a mixed-use neighborhood popular with residents and tourists alike, accessible by cars, public transit, bicycles, and foot traffic. At this particular intersection, you can see various road surface markings delineating the boundaries for the various modes of travel. In addition, there are no less than four sets of signal lights: one for cars, one for transit (streetcars), one for bicycles, and one for pedestrians. Finally, there are numerous road signs indicating no right turns on red lights, no entry to streetcar lanes, no parking etc. This is the standard approach to traffic engineering.

An example of the shared space scheme is the intersection of Exhibition Rd. and Imperial College Rd. in downtown London, England. Note the absence of traffic signals and road signs. Vehicular, bicycle and pedestrian traffic are permitted to mix and there is little in terms of demarcation between the rights of way. In fact, all types of traffic have the right of way at all times. The shared space scheme relies on the common sensibility of those sharing the space to optimize its use.

There are various such examples of the shared space design throughout the world today, and for the most part, in terms of benefits, the results are encouraging. In the Cheshire town of Poynton, England, a three-way intersection serving 26,000 vehicles per day was a cluttered signal-controlled junction that impeded traffic flow for cars and pedestrians alike. Attempts to ease traffic flow by signal timing had failed and the barrier to pedestrian traffic was choking local street-side businesses. The entire roadway was replaced with a shared space scheme in 2012. The results? Personal injury accidents fell from 4-7 incidents per three-year period before the change, to just one in the three years after the change. Traffic speed decreased to about 20 mph despite there being no changes in posted speed limits. Paradoxically, traffic throughput for both cars and pedestrians significantly improved, and local businesses began to thrive from increased foot traffic[1].

[1] “Poynton Town Centre: streetscape and place-making project”. Institution of Civil Engineers, UK (www.ice.org.uk) Retrieved May 29, 2017.

Poynton Town Center Before & After (Image Credit: Video “Poynton Regenerated” youtube.com June 3, 2017)

What if we took the cojustify even further? If removing safety measures makes people more careful, what would happen if we introduced danger-enhancing measures? This was the controversial suggestion of Gordon Tullock, an economist and professor of law at George Mason University. He suggested that if governments really wanted to make motorists drive safely, a spike should be installed in the center of all steering wheels, to be deployed during a collision in the fashion of an airbag, thus increasing the probability of fatal injury to the driver . The proposal is known as “Tullock’s Spike”.

The Duplessis Orphans

A final example of perverse consequences illustrates a failure to appraise the forces of human nature—where outcomes were driven by religious biases and greed. This is the story of the “Duplessis Orphans”, a group of impoverished Canadian children who were exploited and treated inhumanely in the 1940s and 50s—a group that has struggled for justice for decades, and continues to do so to this day.

The province of Quebec in Canada is a predominantly French culture strongly influenced by the catholic church. In the 1940s and 1950s there was widespread poverty in the province, and many social services, including the operation of orphanages and psychiatric asylums, were entrusted to the catholic church.

In Quebec, from 1944 to 1956, some 20,000 so-called “illegitimate” children were born to impoverished families and unmarried young women. As a result of poverty and the stigma associated with being an unwed mother, many of these children were given up to church-run orphanages.

At that time, the premier of the province was Maurice Duplessis, a strict catholic and a powerful ally of the church. With presumably good intentions, he managed to negotiate financial subsidies from Canada’s federal government to help Quebec’s poor. As part of the relief effort, government contributions were set at $1.25 per day for orphans and $2.75 per day for psychiatric patients[1]. This funding model would prove disastrous. Pursuing a desire to secure increased funding, the unwanted children were falsely diagnosed with various psychiatric disorders—declared “mentally deficient”—and placed in asylums where many suffered from sexual, mental, and physical abuse. Some were allegedly subject to illegal medical experiments, lobotomies, and electroshock, sometimes resulting in death.

[1] “Orphans of the 1950’s, Telling of Abuse, Sue Quebec”, Clyde H. Farnsworth, New York Times, Published: May 21, 1993

For many years, the Duplessis Orphans struggled for justice and launched various investigations to prove the wrongdoings of church and state. Finally, in 2001 the Quebec government apologized to the Duplessis Orphans and paid $25 million in compensation to some 1,100 survivors. In 2006, the Quebec government announced it would pay an additional $26 million to the remaining orphans on the condition that no further legal action would be taken. For its part, the church has continually deflected responsibility and has never apologized.[1]

[1] “Duplessis Orphans get $26M from Quebec”, Canadian Broadcasting Corporation (CBC) December 21, 2006

The Duplessis Orphans are a prime example of how ostensibly good intentions can pave the way to hellish outcomes.

Continue to Part 2