Consider a typical decision – you have just arrived in a new city after a 6 hour flight, it’s 6.30 PM and you want to get a decent vegetarian meal. What do you do?You fire up Google, search for good restaurants in that city, and filter them by type, price, location etc. You search the review sites and blogs, both international and local for reviews and ratings on what might be a decent choice. You ask the hotel concierge or ask some friends on Facebook, email or SMS for ideas. You whip out your mobile phone and look for stuff that is ‘nearby’.A good 30-60 minutes later, tired and weighed down by information overload, you do what most would in the coffee shop of the hotel you are staying at, or even order room service. The good meal you planned will have to wait till tomorrow.The challenge you face is not lack of information. The challenge is to filter the right information, for you, the decision-maker. To do so, you need to spend huge amount of effort and time to navigate the choices. But the one thing we do not have enough of today is time. So what does a Choice Engine do? And how does it solve this problem? [Excerpts from the previous post.]
In my last post, we discussed the growing need for a new kind of Choice Engine to solve the defining problem of our age: the misery of too much choice! Let’s now find out how exactly a Choice Engine works.
With the use of cutting-edge technology, a Choice Engine first tries to recognize the underlying factors behind how choices are actually made.
Choices are determined by a person’s taste, influence, context and behaviour. To form meaningful conclusions about one’s choice, the engine needs to combine these factors. It is a complex task. Because we cannot easily get access to the data relating to one’s taste, influence, context and behaviour. Even if we do, they may not be in a structured form for the machine to understand.
However, technology allows us today to arrive at a certain level of precision in understanding one’s tastes by analyzing one’s opinions and views that he/she scatters daily on the web in the form of reviews, likes, ratings, shares, comments etc about food, movies, books, restaurants, Pinterest, Facebook, Twitter etc. These are the testimonies of our ‘tastes’. Although there is a tendency among some experts to classify them as superficial or skin-deep, sheer volume of such a trail can be undoubtedly considered as a strong indication of our tastes.
Influence is someone/something that shapes our tastes or behaviour. This type of data can be gathered from what we read and whom we follow on the net and social media. However, influence can be inferred by the frequency and level of two-way communication between two parties. If I tend to share something on football with David and he responds or shares it, it is likely we have some influence on each other’s views on sports. (Nevertheless, it must be said that it is harder to infer who influences whom without substantial data).
Coming to behaviour, it is the best and most researched part of choice. In general, a behaviour in the past is considered to be the best predictor of a future behaviour, because if a behaviour in the past involves conscious decision making, it is likely to repeat in the future as well. Generally, enterprises like banks, retailers, telcos, airlines, hotels, consumer good companies etc, own large amounts of behaviour data of the customers.
Technology makes it easy today to crawl the web and collect tremendous amount of data about tastes and influences. Those data can also be collected by establishing data partnerships with aggregators and from individuals with use of explicit permissions.
Although data acquisition is relatively easy, combining data is a bigger challenge in terms of both scale and complexity. Perhaps, the best way to actually handle these data sets is to construct a Taste Graph that allows us to understand the linear and non-linear relationships between all the entities or products in these data sets, and the relative strength or affinity between each of them. Once built, it is a data monster. Just imagine a massive graph where you can see the relationships between any restaurant, any movie, book or any type of shirt, anywhere in a country.
This isn’t over yet. Adding Influence or the level of Trust is even harder. Essentially, we need to compute the likelihood that if person A likes something, B will like the same. Simple correlations based on exhibited common behaviours is one basic way. But experts like Dr Jennifer Goldbeck with her Tidal Trust algorithm have gone beyond and established ways in which we can measure the level of trust across many levels of separation.
Combining taste and Influence data with behaviour presents a lesser technical challenge, but a critical one, to make real the promise of Choice Engines. For years, the company you bank with or shop at has a deep view of your behaviour inside the organization, but a very limited view of you as a person.
Now, adding enterprise behaviour data to the Taste Graph will allow a more holistic view of a customer’s likely behaviour, tastes and influence, something so far available to very few companies like Amazon.
This resultant Choice Graph can be used to predict much smarter choices for each individual consumer. This graph will do two things better than anything available today.
1. It will enable the discovery of a wider set of tastes and influences for each consumer than are known today (based on the external Taste Graph).
2. It will allow for a deeper and more predictive personalisation for each such consumer, based on the deep enterprise behaviour data available to a company.
In essence, such a Choice Graph will allow a bank to not only know that an individual like Zara likes certain types of movies, food and books, it also allows them to conclude that she will mostly likely shop at a certain brand of fashion (though she may never have bought any of these before using a bank product).
Finally, let’s add an important layer of context. Context data comes from the sensors of every device we carry, such as phone, tablet, computer and wearable sensors. They can tell us about the location, time, date, weather and activity we indulge in. They can track the digital trail we leave behind including every site we visit, every click we make. Increasingly, with the proliferation of cameras, they can even capture every gesture or face we make.
By launching apps, placing cookies or using FB or Twitter pages, the enterprise can now use the context data that is automatically collected in such engines, to narrow down the choices that are relevant for Zara and present them to her as a simpler set of choices.
So, how does all these benefit Zara? Going back to my previous post… Zara has just arrived in a new city. She is hungry and looking for a place to eat. She fires up her Bank’s SimplerChoices app. It knows she is vegetarian, that she doesn’t like spicy food. The sensors tell it where she is. She looks at the screen and there are 4 choices of restaurants that serve vegetarian food – one recommended by a friend of hers who has been there recently, one reviewed well in her favourite newspaper, another that has rave reviews from other vegetarians, and a fourth that is really close by and affordable. She clicks on two of them to get to know more, and after 5 minutes, she’s made her choice.
Easy. Simple. Quick. And really useful.
For Zara, yes. For the bank, yes, because they have delivered a great experience to their customer.
Is such a Choice Engine within our reach today?
It is. It is tough to build. The challenges involved, though non-trivial, are mathematical and computational, but are very solvable. It has to do all of the math and computing, starting from ingesting massive data sets, to combining them intelligently to form Taste and Influence Graphs, to mixing them with enterprise data, to computing all possible and relevant choices for each customer and presenting a simpler set of prioritized choices based on the current context of the customer.
It is precisely because of this power that Choice Engines have a tremendous amount of promise. But a good choice engine needs to go beyond these to strike a balance between privacy and personalization, and allow enterprises to ensure their internal data is kept confidential. These are major issues worthy of a separate post.
I am sure you have many questions. Are Choice Engines the same as Recommendation Engines? How will you ensure my enterprise data is kept confidential? How will I benefit as a CMO or business leader? Keep watching this space for some answers.