Flipping the Model of Scientific Enquiry, Step One: Engaging with Big Data

Joe Lamantia. The Language of Discovery: Designing Big Data Interactions.
http://www.slideshare.net/moJoe/the-language-of-discovery-designing-big-data-interactions

One of my recent soapboxes has focused on flipping the model of scientific inquiry. No longer is Hypothesis-First a valid approach for all science. I’m sure this classic model will always be necessary for some questions in some spaces, however this is changing. The purpose of a hypothesis first strategy made sense when quality data was scarce and the effort to generate data was the most expensive part of the research process. Of course, you would want to maximise the data generation process by making it as efficient and cost-effective as possible. Thus, we’ve tended to approach science with a methodology something like this: Hypothesis > Study Design > Data Generation & Gathering > Analysis & Publication (repeat).

What changes when data is cheap and readily available? An excellent question, and one being confronted in practice by researchers in fields across almost all of human endeavor. My distillation of what I’ve been observing and hearing is that when data is cheap, the data comes first. I expect the new model of scientific inquiry to be more like this: Data > Exploratory Analysis > Hypothesis > Confirmatory Analysis > Sharing Findings & Conversations (repeat).

These assumptions are all based on the idea of big data, with the big hurdle being that little tricky fiddly bit of the exploratory analysis. When we have all the data we could imagine, how do we know what questions to ask? Step one is how we engage with the data, how we explore it, what tools are available, how do those tools help us to make sense of the data. With that as a big part of my mental context right now, when I saw this presentation in Slideshare, I had to share it with you. This is still just the beginning, folks, but there is already more available to help us with this than we might imagine. Incredible resources to start learning to use, new competencies to develop.