Weapons of Math Destruction by Cathy O’ Neil

Recommended by Professor Musser

Cathy O’ Neil’s Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy warns us that the pervasive use of large collections of data can be dangerous. O’Neil, with a Ph.D. in mathematics (“algebraic number theory”), has taught at Barnard, and also has worked at a hedge fund (as a “quant”) and at an e-commerce start-up. Disturbed by the unexamined influence of large data on our policies and our lives, she decided to alert others to her concerns about big data. So she started a blog, MathBabe, and she wrote this book.

The book explains “the damage inflicted by WMDs [Weapons of Math Destruction] and the injustice they perpetuate” (13). It “explore[s] harmful examples that affect people at critical life moments: going to college, borrowing money, getting sentenced to prison, or finding and holding a job”—all “increasingly controlled by secret models wielding arbitrary punishments” (13).

The book is clear and persuasive, enlightening and disturbing. As O’Neil explains, large databases may seem to be objective and scientific, their findings or results determined by mathematical principles immune to human error and biases. However, their designers, like all human beings, are prone to error, misunderstanding, and ignorance. The databases “are constructed not just from data but from the choices we make about which data to pay attention to—and which to leave out. Those choices are not just about logistics, profits, and efficiency. They are fundamentally moral” (218). To minimize the damage, O’Neil argues that “We have to explicitly embed better values into our algorithms, creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit” (204).