Bloodletting: Why Controlled Experiments are Important
Ron Kohavi
19 May 2008
Many times, the question of the value of controlled experiments comes up. We are often asked why experts can’t just “do the right thing” and why we encourage evaluating ideas using controlled experiments. Here is one of the best stories I know, which shows that even the best experts tend to latch on to conventional wisdom and correlations, sometimes for thousands of years. A (randomized) controlled experiment, or a randomized clinical trial (RCT), is the gold standard in drug approval because it can test whether the drug is causally effective.
Many times, the question of the value of controlled experiments comes up. We are often asked why experts can’t just “do the right thing” and why we encourage evaluating ideas using controlled experiments. Here is one of the best stories I know, which shows that even the best experts tend to latch on to conventional wisdom and correlations, sometimes for thousands of years. A (randomized) controlled experiment, or a randomized clinical trial (RCT), is the gold standard in drug approval because it can test whether the drug is causally effective.
The following is based on several sources: NY Times, May 5, 2002, Childbed Fever, A Physician Looks at the Death of Washington, and Wikipedia/Bloodletting.
Since the days of the ancient people, including Mesopotamians, the Egyptians, the Greeks, the Mayans, and the Aztecs, the prevailing conception of illness was that the sick were contaminated by some toxin or contagion. These conditions could be improved by opening a vein and letting the sickness run out – bloodletting. Once the toxins were gone, the patient immediately felt different, and often better. As anyone who has given blood can tell you, losing a pint or two can make you feel transported, transformed. Intuitively, it was satisfying to doctors that the procedure left the patient feeling drained – physically, emotionally and into the sink.
The practice was continued by surgeons and barber-surgeons. Though the bloodletting was often recommended by physicians, it was carried out by barbers. This division of labor led to the distinction between physicians and surgeons. The red-and-white-striped pole of the barbershop, still in use today, is derived from this practice: the red represents the blood being drawn, the white represents the tourniquet used, and the pole itself represents the stick squeezed in the patient's hand to dilate the veins.
Bloodletting was used to treat almost every disease. One British medical text recommended bloodletting for acne, asthma, cancer, cholera, coma, convulsions, diabetes, epilepsy, gangrene, gout, herpes, indigestion, insanity, jaundice, leprosy, ophthalmia, plague, pneumonia, scurvy, smallpox, stroke, tetanus, tuberculosis, and for some one hundred other diseases (Childbed Fever, p. 6). It was judged most effective to bleed patients while they were sitting upright or standing erect, and blood was often removed until the patient fainted.
Physicians often reported the simultaneous use of fifty or more leeches on a given patient. Through the 1830s the French imported about forty million leeches a year for medical purposes, and in the next decade, England imported six million leeches a year from France alone (Childbed Fever, p. 7).
On December 12, 1799, President George Washington, 68 years of age, rode his horse in heavy snowfall to inspect his plantation at Mount Vernon. It was about 30 degrees Fahrenheit, and he complained about a sore throat, yet rode again the day after. On December 14, he was in respiratory distress. Mr Albin Rawlins, the estate overseer, prepared a medicinal mixture of molasses, vinegar, and butter, and when Washington almost suffocated trying to swallow the concoction, decided on bloodletting and removed 12-14 ounces of blood. Dr. James Craik was brought in, and extracted another 20 ounces of blood, followed by yet another 20 ounces of blood. When a vinegar and hot water solution did not help, he extracted another 40 ounces of blood. In the afternoon, another doctor arrived, Dr. Dick, and he drew another 32 ounces of blood for a total of about 82 to 124 ounces, or 2.5 to 3.7 liters in ten hours. The total blood in George Washington’s body was estimated at 7 liters, so about 35% to over 50% was extracted, which inevitably led to preterminal anemia, hypovolemia, and hypotension. The fact that General Washington stopped struggling and appeared physically calm shortly before his death may have been due to profound hypotension and shock (A Physician Looks at the Death of Washington).
It is understood now that bloodletting only hastened the death of the ill.
We know that bloodletting is unhelpful because a Parisian doctor named Pierre Louis did an experiment in 1836 that is now recognized as one of the first clinical trials, or a randomized controlled experiment. He treated people with pneumonia either with early, aggressive bloodletting, or less aggressive measures; at the end of the experiment, Dr. Louis counted the bodies. They were stacked higher over by the bloodletting sink.
Next time someone tells you that they are sure their idea will work, consider running a prototype in a controlled experiment, and make a data driven decision!