eCommerce Marketing Blog

The Good Judgment Project

Posted by Paul Kaye on Jan 14, 2015 9:22:00 AM


I just read a fascinating article the other day on the website of my all-time favorite news source, NPR. It was about a 3-year study called The Good Judgment Project, conducted by 3 psychologists and a few other people within the intelligence community. Its purpose was to determine how accurately a group of regular citizens could predict the outcomes of important world events. The group consisted of 3,000 people, none of whom had any government security clearance or access to classified information, and they were all asked to make predictions on issues such as these:

Would there be a significant attack on an Israeli territory before spring of this year?

Would North Korea launch a new multi-stage missile prior to May 10, 2014?

The result of the study? This random group of 3,000 political neophytes predicted the outcomes of world events with incredibly high accuracy. In fact, they were so accurate that a smaller subset of this group was able to predict world affairs 30% more accurately than the CIA. No training, no special government clearance, no daily briefings. Yet they were better judges of political, military, and economic outcomes than the world’s most revered intelligence agency. How is this possible?

The answer lies in a theory founded by a British statistician named Francis Galton in 1906, called “The Wisdom of the Crowds.” The theory states that when large groups of people make a prediction about something, their averaged collective answer is more likely to be correct than when one person (or a few people) with a vast amount of knowledge on that subject makes the same prediction.

You can read the full article here:

So, what does this have to do with recommendations systems for online retailers? Well, to be honest, everything.

When a shopper visit a retailer’s website, there is a good chance they will be presented with specific product recommendations as they surf around from page to page – “People who Like That Product Also Like These Products,” and “We Think You’ll Love These Items” are common website merchandising blocks we have all seen on most stores’ sites. And, for the most part, the same assertion that the 3 psychologists in The Good Judgment Project made – that large groups of people can predict something better than a few people can, no matter how much or how little information they have – is used every time a shopper visits a retailer’s website.

In general, there are 3 things that a good product recommendation engine will consider when it determines the right products to offer a shopper: what products are popular today; what the specific customer has been doing on the website recently; and, most interestingly, the wisdom of the crowds.

Product popularity: Simply put, this is nothing but a counting mechanism. The logic behind it is that if a product is getting a lot of page views, then it means that it is popular; and if it is popular, then there is a really good chance that that is what other customers are going to want to see and buy as well. Everyday, the products that have been viewed or purchased the most are the products the retailer will promote. Of course, it’s also a self-perpetuating cycle, and popularity begets popularity.

Customer behavior: All good recommendations engines track customer behavior on the website they are visiting, and make product recommendations based on their behaviors over time. Did your customer take a look at the new Callaway graphite irons last time she visited, but ended up not buying them? Next time, show her the same irons and tempt her once again. Better yet, sweeten the offer, and show her the same irons, but give her free shipping if she buys them today. Recommendations engines do this all day, millions and millions of times per day.

Wisdom of the Crowds: This is the most interesting part of how product recommendations are made. Just as in The Good Judgment Project, most online retailers have their own CIA operatives: marketing and merchandising experts who know the store’s product lines better than anyone else in the world. They know what to cross-sell and what the upsell. They know that these shoes go with that dress; that this HDMI cable is the right one for that 47 inch Samsung HDTV, and which toy your 2-year old cockapoo, Lucy, is most likely not to be able to destroy in 5 minutes flat. They know their product lines well. However, they don’t know them nearly as well as the crowds do, and certainly not to the scale they do. Based on its tracking of the crowds – the mass data-gathering of what all customers have been looking at and buying over time -- a recommendation engine can instantly predict the best products to cross-sell and upsell to each of a store’s customers, and present them in a personalized manner, millions of times per day. And, given that many stores have hundreds of thousands or even millions of products, being able to cross-sell and upsell instantly is essential and can really only be done by machine.

In online retail, the result of the wisdom of the crowds is undeniable: a 10-30% sales lift immediately and consistently. For the operatives working at Langley: well, as our friends at NPR and the scientists of the Good Judgment Project have shown, it might be time for them to start updating their resumes.


Paul Kaye

Subscribe to Email Updates

Posts by Topic

see all