By: Eric Seigel

  LISTEN

  READ

We are surrounded by data. Lots of it. BIG data.  But what can we do with it? What can we learn from it? Most people are not interested in data. To them, it’s a vast, endless set of recorded facts and figures, made even more banal through endless tweets about what others have had for lunch. 

Yet in business, data collation is everywhere. As Seigel puts it, “Every medical procedure, credit application, Facebook post, movie recommendation, fraudulent act, spammy e-mail, and purchase of any kind—each positive or negative outcome, each successful or failed sales call, each incident, event, and transaction—is encoded as data and warehoused.”

We seem to have an information goldrush – but where’s the gold?  It isn’t the data.  The gold is discovered within: knowledge of things that haven’t happened yet. 

That’s what’s behind Eric Siegel’s book, “Predictive Analytics” and this summary.

According to Siegel, whilst we live in a predictive society, the best way to prosper in it is to understand the objectives, techniques, and limits of predictive models.  So let’s make ourselves aware of the constraints. I predict ten minutes well spent. 

PA 101

Prediction is power. Big business can secure a competitive advantage by predicting the future destiny and value of individual assets.  Predictive analysis is the process by which an organisation learns from the experience of all its team members and computer systems. It’s about forward thinking and as Sheldon from the Big Bang Theory points out: “The alternative would be to think backwards . . . and that’s just remembering.”

For a company to exploit predictive analytics it has to act on its predictions, applying what’s been learned and what’s been discovered within the data. This gives rise to what Seigel calls The Prediction Effect: as long as the predictions are better than guessing, predictive analytics is believable. 

Predictive Analytics can be split into in two parts: 

1. What’s predicted: the kind of behaviour (i.e., action, event, or happening) to predict for each individual, stock, or other kind of element.

2. What’s done about it: The decisions driven by prediction; the action taken by the organisation in response to or informed by each prediction. 

These are supported by a predictive model, a mechanism that takes a characteristic of the individual as input, crunches the numbers and provides a prediction as output. The higher the score, the more likely it is that the individual will exhibit the predicted behaviour.  This score is then used to drive an organisational decision, guiding which action to take.  

The Ethics of PA

Here we begin to get into the Future Crime Scenario of Minority Report. How do we safely harness a predictive machine that can foresee the future? Are civil liberties at risk? Prediction has the potential to snoop into our private future. This isn’t a case of mishandling, leaking, or stealing data. Rather, as Siegel puts it, it’s the generation of new data, the indirect discovery of unvolunteered truths about people. 

Data’s value is the very thing that also makes it sensitive. The more data, the more power. The more powerful, the more sensitive. 

As organisations we must secure good data governance to fully exploit the benefits of predictive analytics:

• We must decide what is stored and for how long.
• Which employees, types of personnel, or group members may retrieve and look at which data elements.
• What data may be disseminated to which parties within the organisation, and to what external organisations.
• What data elements may be brought together, aggregated, or connected.
• How may each data element be acted upon, determining an organisation’s response or other behaviour.

To make it even more complicated, add to each of these items “ . . . under which circumstances and for what type of intention or purpose.” 

Predictive Analytics produces new information that’s so powerful, it must be handled with a new kind of care. We’re in a new world in which systems not only divine new, potent information, but must carefully manage it as well. 

The Data Effect 

What’s exciting about data isn’t how much of it there is, but how quickly it is growing.  There’s always so much more today than yesterday.  Size doesn’t matter. It’s the rate of expansion. 

What guarantees that all this data noise holds value?  The answer is simple. Everything is connected to everything else—if only indirectly—and this is reflected in data.  Pull some data together and, although you can never be certain what you’ll find, you can be sure you’ll discover valuable connections by decoding the language it speaks. That’s The Data Effect in a nutshell. 

The Data Effect: Data is always predictive. 

But this carries an immediate risk. Correlation does not imply causation.  The discovery of a predictive relationship between A and B does not mean one causes the other, not even indirectly.

When applying predictive analysis, we normally don’t know about causation and don’t often care. What we want is to predict rather than explain.

The Ensemble Effect 

Netflix is a prime example of Predictive Analytics in action.  Why? They claim 70 percent of Netflix movie choices arise from its online recommendations. 

Seigel reminds us that the building block of predictive analytics is the predictor variable, that single value measured for each individual. But he also points out that, frequency—the number of times the individual exhibits the behaviour is a useful measure.  PA builds its power by combining dozens—or even hundreds—of predictors.  That’s why Netflix works.  Taking the advice of one person may not hold influence over us, but the common belief of many – that’s different.

Like a crowd of people, and the audience of the TV quiz show “Who Wants to Be a Millionaire?”, we get a  “collective intelligence” effect.  This makes up for any weakness in our predictive model where we can have both false positives and vice versa.  As with guesses made by people, the predictive scores produced by models are imperfect. Some will be too high and some too low. Averaging scores from a mix of models can wipe away much of the error. 

Siegel calls this The Ensemble Effect. By simply joining models together, we can increase our model’s structural complexity while retaining a critical ingredient: robustness against over assumption – the belief that our model will predict correctly. When joined in a collective group, predictive models compensate for one another’s limitations, so the ensemble as a whole is more likely to predict correctly than its component models are. 

So, predictive analysis is best performed collectively.  It needs positives and negatives – and many of them – to make a sound prediction. One small data set, one small analysis, one derived prediction… one big risk of error. 

It’s not what you ask, it’s how you react

Often, an organisation needs to decide what next action to take. It doesn’t just want to predict what individuals will do, it wants to know what to do about it.

Take the following example.  Your cell phone provider knows your contract is about to expire so they send you a brochure with their latest offerings.  Big mistake. The company just reminded you that your contracted commitment is ending and you’re free to defect.  The cell phone provider hopes you renew your contract. You on the other hand, having been reminded, may look at alternatives.

This unexpected behaviour brings up the question of what Predictive Analytics should be used to predict in the first place.  From the Cell phone providers perspective he may look to predictive analytics as follow:

Application: Customer Retention

1. What’s predicted: Which customers will leave.
2. What’s done about it: Retention efforts target at-risk customers.

However what they really need to analyse is the following:

Application: Marketing Impact

1. What’s predicted: How will customers react to the reminder brochure.
2. What’s done about it: Retention efforts target at-risk customers.

PA shifts substantially, from predicting a behaviour to predicting influence on behaviour. 

Predicting influence promises to boost predictive analytics value, since an organisation doesn’t just want to know what individuals will do—it wants to know what it can do about it.

But as Siegel points out, we can’t know everything about a human. In particular, we can’t know both of the things that we’d need to know in order to conclude that a person could be influenced; for example:

1. Will Bill purchase if we send him a brochure?
2. Will Bill purchase if we don’t send him a brochure? 

We can find out (1) by sending him a brochure. We can find out (2) by not sending him a brochure. But we can’t both contact and not contact Bill.

Instead of predicting an outright behaviour, we need a model that scores according to the likelihood of whether an individual’s behaviour can be influenced. We need what Seileg calls an Uplift Model: A predictive model that predicts the influence on an individual’s behaviour from applying one treatment over another. 

Future Predictions.

Predictive analytics has the ability to influence to the very heart of society: companies, governments, law-enforcement, charities, hospitals, or universities all undertake many millions of operational decisions in order to enact services. As Siegel proclaims, prediction is key to guiding these decisions and subject to an awareness of its limitations, predictive analysis is the means with which to improve the efficiency of these operations.