Abstract
In spite of my personal belief in the benefits of artificial intelligence (AI), reading Cathy O'Neil's book “Weapons of Math Destruction” left me feeling unsettled.[1] She describes how flawed and unchecked algorithms are widely applied in areas that affect us all: hiring, credit scoring, access to education, and insurance pricing. In one example, a fixed percentage of teachers in a U.S. region was dismissed every year based on biased and opaque algorithms. The authors concluded that such algorithms act as “weapons of math destruction,” perpetuate and amplify societal biases, act unethically, and harm vulnerable populations. The question arises as to what happens when we apply these algorithms to medicine? How do we know whether we are giving our patients the correct diagnosis or prognosis? Are we still sure that patients are receiving the appropriate treatment? Would we notice if the algorithms were geared more toward the needs of companies (make a lot of money) or health insurance companies (spend as little as possible)? In fact, evidence of bias and inequality of algorithms in medicine is already available.[2] Due to these risks, some of my colleagues suggest that AI should be completely banned from medicine.
