Death. Many of the discussions around data revolve around death. These discussions are typically pretty mundane. Why do phone calls die? Why do banks go out of business? What makes servers and other network machinery fail? It’s sad, but sometimes servers die, phone calls are lost, and some long-standing banks suddenly come to an end.
When that happens we use data to discover where it all went wrong. More accurately, we try to use data to discover where it all went wrong, poring through customer accounts data, cell tower network logs, profit and loss statements. That’s the post-mortem, a Big Data autopsy.
Typically these are done using logs or selected data sets, where an analyst pulls some segment of the data for analysis. If that data has structure, there are many tools for querying and analyzing it. For unstructured data, the long strings of numbers and letters that is the language of machines, solutions have begun to emerge that index the data, making it possible for analysts to search for the signals that will lead to the right answer. In each case, analysts and data experts must comb through data for a cause.
Not too dissimilar is the actual autopsy, which is itself a search for cause. Coming to a definitive cause of death can require a knowledgeable physician, but regardless of the skills of the investigator, prior knowledge of the subject can be critical. Some autopsies are straightforward, but we could easily imagine a situation where there are many contributing factors, some combination of toxins and disease that lead to an untimely death.