Play Live Radio
Next Up:
0:00 0:00
Available On Air Stations
Donate today to give back in celebration of all that #PublicMediaGives. Your contribution will be matched $1 for $1.

Big Data Is The Steam Engine Of Our Time

Big Data: trying to make sense of the numbers
Kazuhiro Nogi
AFP/Getty Images
Big Data: trying to make sense of the numbers

We inhabit a world of blinding technological change. New devices, new programs and new infrastructure rise up, dominate discourse and pass away before we even have time to comprehend their intent. But for all the change we've experienced, the the most profound transformation of the digital era is really just getting started. Welcome to the era of Big Data.

For the sake of clarity, let's recall what Big Data means. At the intersection of the Internet, digital recording technologies and a profusion of small-scale wireless sensors comes an exponential increase in stored data. We're recording information of every kind: tweets, engine temperatures, Facebook photos, stock trades, grocery store purchases. These data points pile up, billions upon billions, trillions upon trillions, recording our lives and the life of the world we inhabit. They are very, very valuable to people with the ability to see patterns in, and extract insight from, the blizzard of information blowing out of the cloud.

Ultimately, the promise of Big Data is the ability to understand (and control) a seemingly chaotic world on levels never before imagined. The dangers of Big Data stem from that very same promise. Its impact on society will be akin to the transformative effect of past technological revolutions.

Consider thermodynamics, the all-embracing theory of energy and change that led us to ideas like entropy and the arrow of time. Thermodynamics traces its roots to the steam-power revolution of the 19th century. It was the rapidly expanding industries of power generation, of railroads and steamships, that gave birth to the new worlds of human behavior that emerged from the industrial revolution. In the tumble and tumult of all that change, scientists recognized new possibilities for describing and understanding the world and the science of thermodynamics was born. Soon that science grew beyond the realm of steam power to touch everything from cosmology to biology.

As strange as it may seem, Big Data may be the steam engine of our time.

Not everyone, however, is enchanted by the ever-burgeoning sets of numbers at our disposal. Emanuel Derman, one of the pioneers of Wall Street financial modeling, says this in an excellent piece for Frankfurter Allgemeine Zeitung:

Some talking heads like to say that computer-aided analysis of patterns will soon replace our traditional methods of discovering the truth, in medicine and the social sciences as well as in physics.

Derman then warns against our penchant to become "bewitched" by data, to believe the data can speak for itself. He argues that human intuition is a crucial ingredient in our quest to understand the world.

Big Data is useful, but is not a replacement for the classic ways of understanding the world. Data has no voice. There is no "raw" data. Choosing what data to collect takes insight; making good sense of it requires the classic methods: you still need a model, a theory, or intuition to find a cause.

While this point is well taken, I believe there is something real and powerful happening in the Big Data revolution. It's more than just a fad. It's the next link in the long chain connecting culture and technology to human history.

Now Big Data — seen and unseen — is hitting us in all corners of our lives, from the price of things to traffic patterns to who our social networks think we befriend.

Through new fields like data science and network theory, Big Data will not only change the world we move through as individuals, it will change the world we imagine through science.

Like it or not, Big Data will only get bigger (and bigger).

You can keep up with more of what Adam Frank is thinking on Facebook and on Twitter: @AdamFrank4

Copyright 2021 NPR. To see more, visit

Adam Frank was a contributor to the NPR blog 13.7: Cosmos & Culture. A professor at the University of Rochester, Frank is a theoretical/computational astrophysicist and currently heads a research group developing supercomputer code to study the formation and death of stars. Frank's research has also explored the evolution of newly born planets and the structure of clouds in the interstellar medium. Recently, he has begun work in the fields of astrobiology and network theory/data science. Frank also holds a joint appointment at the Laboratory for Laser Energetics, a Department of Energy fusion lab.

You make NHPR possible.

NHPR is nonprofit and independent. We rely on readers like you to support the local, national, and international coverage on this website. Your support makes this news available to everyone.

Give today. A monthly donation of $5 makes a real difference.