Health / Pharma

Google’s flu fail shows the problem with big data

When people talk about ‘big data’, there is an oft-quoted example: a proposed public health tool called Google Flu Trends. It has become something of a pin-up for the big data movement, but it might not be as effective as many claim.

The idea behind big data is that large amount of information can help us do things which smaller volumes cannot. Google first outlined the Flu Trends approach in a 2008 paper in the journal Nature. Rather than relying on disease surveillance used by the US Centers for Disease Control and Prevention (CDC) – such as visits to doctors and lab tests – the authors suggested it would be possible to predict epidemics through Google searches. When suffering from flu, many Americans will search for information related to their condition.

The Google team collected more than 50 million potential search terms – all sorts of phrases, not just the word “flu” – and compared the frequency with which people searched for these words with the amount of reported influenza-like cases between 2003 and 2006. This data revealed that out of the millions of phrases, there were 45 that provided the best fit to the observed data. The team then tested their model against disease reports from the subsequent 2007 epidemic. The predictions appeared to be pretty close to real-life disease levels. Because Flu Trends would able to predict an increase in cases before the CDC, it was trumpeted as the arrival of the big data age.

Between 2003 and 2008, flu epidemics in the US had been strongly seasonal, appearing each winter. However, in 2009, the first cases (as reported by the CDC) started in Easter. Flu Trends had already made its predictions when the CDC data was published, but it turned out that the Google model didn’t match reality. It had substantially underestimated the size of the initial outbreak.

Read More

  1. g 8 months ago

    bookmarked!!, I really like your site!

  2. minecraft games 4 months ago

    These are in fact impressive ideas in concerning blogging.
    You have touched some nice things here. Any way keep up wrinting.

  3. minecraft games 4 months ago

    Hi there, I enjoy reading through your article. I like to write a little comment
    to support you.

  4. then coconut oil 2 months ago

    Hey! Would you mind if I share your blog with my zynga group?
    There’s a lot of folks that I think would really enjoy your content.

    Please let me know. Thank you

  5. coconut oil where 2 months ago

    When I originally commented I clicked the “Notify me when new comments are added” checkbox and now
    each time a comment is added I get four emails with the same comment.
    Is there any way you can remove me from that service?

  6. ps4 games 1 month ago

    Your style is so unique compared to other people I’ve read stuff from.

    Many thanks for posting when you have the opportunity, Guess I will just book mark this blog.

Leave a Comment

Your email address will not be published.

You may also like

Pin It on Pinterest