The age of Big Data is upon us, and mostly what we hear are the troubling and potentially terrifying consequences of business and government having easy access to all of our data. That's a real problem that we have to devote time and attention to dealing with.
Yet Big Data can do many other things, many of them very beneficial. The misnamed Department of Commerce collects enormous amounts of data about, among other things, the weather. Before the growth of the internet into its modern form, that data mostly sat on paper, and later on tape, and eventually some of it on servers, in the bowels of NOAA--the National Oceanic and Atmospheric Administration, inside the Commerce Dept. Then a grad student with the foresight to see how useful vast stores of data could be went looking for weather data to test out a theory for his research, and stumbled upon a hole in the Commerce Dept. systems that let him download that data and work with it.
