Google does it. Amazon does it. Wal-Mart does it. And, as recent news reports made clear, the U.S. government does it.
Does what? Uses "big data" analysis of the swelling flood of data that is being generated and stored about virtually every aspect of our lives to identify patterns of behavior and make correlations and predictive assessments.
Amazon uses customer data to give us recommendations based on our previous purchases. Google uses our search data and other information it collects to sell ads and to fuel a host of other services and products.
The National Security Agency, a news article in The Guardian revealed recently, is collecting the phone records of millions of American customers of Verizon under a secret court order. And under another surveillance program called Prism, The Guardian and The Washington Post reported, the agency has been collecting data from emails, audio and video chats, photos, documents and logins from leading Internet companies like Microsoft, Yahoo, Google, Facebook and Apple, to track foreign targets.
In Big Data, their illuminating and very timely book, Viktor Mayer-Schönberger, a professor of Internet governance and regulation at the Oxford Internet Institute at Oxford University, and Kenneth Cukier, the data editor for The Economist, argue that the nature of surveillance has changed. "In the spirit of Google or Facebook," they write, "the new thinking is that people are the sum of their social relationships, online interactions and connections with content. To fully investigate an individual, analysts need to look at the widest possible penumbra of data that surrounds the person — not just whom they know, but whom those people know too, and so on."
Mayer-Schönberger and Cukier argue that big data analytics are revolutionizing the way we see and process the world — they even compare its consequences to those of the Gutenberg printing press. And in this volume they give readers a fascinating — and sometimes alarming — survey of big data's growing effect on just about everything. including the way we think. Notions of causality, they say, will increasingly give way to correlation as we try to make sense of patterns.
Data is growing incredibly fast — by one account, it is more than doubling every two years — and the authors argue that as storage costs plummet and algorithms improve, data-crunching techniques, once available only to spy agencies, research labs and gigantic companies, are becoming increasingly democratized.
Big data has given birth to an array of companies and has helped existing companies boost customer service and find new synergies. Before a hurricane, Wal-Mart learned, sales of Pop-Tarts increased, so stores began stocking boxes of Pop-Tarts next to the hurricane supplies "to make life easier for customers" while boosting sales. UPS, the authors report, has fitted its trucks with sensors and GPS so it can monitor employees, optimize route itineraries and know when to perform preventive vehicle maintenance.
Baseball teams like Billy Beane's Oakland A's (immortalized in Michael Lewis' best-seller Moneyball) have embraced number-crunching approaches to scouting players with remarkable success. The 2012 Obama campaign used sophisticated data analysis to build a formidable political machine for identifying supporters and getting out the vote.
In the years to come, Mayer-Schönberger and Cukier contend, big data will increasingly become "part of the solution to pressing global problems like addressing climate change, eradicating disease and fostering good governance and economic development."
There is, of course, a dark side to big data, and the authors provide an astute analysis of the dangers they foresee. Privacy has become much more difficult to protect, especially with old strategies — "individual notice and consent, opting out and anonymization" — losing effectiveness or becoming beside the point.
"The ability to capture personal data is often built deep into the tools we use every day, from websites to smartphone apps," the authors write. And given the myriad ways data can be reused, repurposed and sold to other companies, it's often impossible for users to give informed consent to "innovative secondary uses" that hadn't been imagined when the data was first collected.
The second danger they worry about sounds like a scenario from the sci-fi movie Minority Report, in which predictions seem so accurate that people can be arrested for crimes before they are committed. In the real near future, the authors suggest, big data analysis might bring about a situation "in which judgments of culpability are based on individualized predictions of future behavior."
Already, insurance companies and parole boards use predictive analytics to help tabulate risk, and a growing number of places in the United States, the authors of Big Data say, employ "predictive policing," crunching data "to select what streets, groups and individuals to subject to extra scrutiny, simply because an algorithm pointed to them as more likely to commit crime."
One problem with relying on predictions based on probabilities of behavior, Mayer-Schönberger and Cukier argue, is that it can negate "the very idea of the presumption of innocence."
"If we hold people responsible for predicted future acts, ones they may never commit," they write, "we also deny that humans have a capacity for moral choice."
To their credit, Cukier and Mayer-Schönberger recognize the limitations of numbers. Though their book leaves the reader with a keen appreciation of the tools that big data can provide to help us "quantify and understand the world," it also warns us about falling prey to the "dictatorship of data."
"We must guard against overreliance on data," they write, "rather than repeat the error of Icarus, who adored his technical power of flight but used it improperly and tumbled into the sea."
'Big Data: A Revolution That Will Transform How We Live, Work, and Think'
By Viktor Mayer-Schönberger and Kenneth Cukier
Eamon Dolan/Houghton Mifflin Harcourt. 242 pp. $27.
The country's book-buyers are reading up on being watched.
Sales for dystopian classics such as George Orwell's 1984 and Aldous Huxley's Brave New World have been strong since news broke recently that the U.S. government had vast surveillance programs targeting phones and Internet records.
Several editions of Orwell's 1984, about an all-seeing government, were among Amazon.com's top 100 sellers as of Thursday morning. Huxley's story of a mindless future ranked No. 242 and was out of stock.
A perennial favorite of futuristic horror, Ray Bradbury's Fahrenheit 451, was ranked No. 60.