Recommendation Engines for Email Marketing
“Micro-targeting,” “hyper-personalization,” “individualized insights” and “one-to-one marketing” are some of the “buzziest” of the “big data” email marketing phrases, but with good merit.
Personalized emails have 6x higher transaction rates on average. A way to achieve this at scale for hundreds of subscribers is through the use of a Recommendation Engine. But how does it work and what’s important to look for when buying recommendation technology?
What you’ll learn from this article:
- The impact of relevance and personalization on email
- How Recommendation Engines work and the current state-of-the-art
- How to evaluate predictive recommender technology for your email program
This is interesting information for Chief Marketing Officers (CMOs), Agency executives responsible for client growth and all email marketers in general. For these groups will sooner or later be confronted with the topic of recommendation engines for email marketing.
The pressure is high for greater relevance in email
In the digital marketing mix, email has long been the leader in terms of revenue, and returns $38 for every dollar invested. It’s the top preferred channel for customers who wish to receive information from brands: 72% of thousands of adults surveyed, want brands to contact them through email.
However, information overload on every channel has caused many challenges for email marketers. One effect in particular is that the average person’s attention span has dropped by nearly half in the last ten years. The impact on email is an ongoing decline in click rates because people have less time, less patience, and more rapidly scan content when hunting for meaningful information.
Recommendation Engines optimize email results
One solution that filters out irrelevant content and provides a more personalized email approach is a Recommendation Engine.
Recommendation Engine technology delivers more relevant content in order to:
- Increase conversions
- Improve customer retention
- Foster referral marketing
- Extend the lifetime value of subscribers
- Increase the ROI for marketing efforts
The use of email personalization is by no means a new topic, however what can be done today with the latest recommender technology is much more exciting and impactful than adding a bit of custom text to a message.
For example, by allowing predictive recommendations to be added to any email, we report an overall average lift in CTOR (click-to-open rate) of 73% versus emails with no predictive content.
Recommendation Engine results by vertical are as follows:
- Electronics and gadgets: +283%
- Fashion and apparel: +49%
- Food and beverage: +399%
- Health and beauty: +296%
- Sports and recreation: +265%
Others have also reported on the beneficial impact of personalization – Experian has shown that subscribers who receive more relevant content have:
- 29% higher open rates
- 51% higher click rates
- 6x increase in transactions
In contrast, negative effects occur if personalization is ignored. Not only are open, click, and transaction rates less, according to Silverpop more than 50% of people unsubscribe from email lists because the content isn’t relevant, or it’s too frequent, or both. If an email doesn’t immediately grab the eye it’s doomed. Consumers have become desensitized and even blind to generic email marketing.
Subscribers will exchange data for relevance
Online consumers are rather educated on the basics behind personalization, thanks to the stellar stories of companies like Amazon and Netflix. They know that customized experiences don’t just happen out of thin air.
People believe in the benefit they can get when they are better understood, and they are willing to share their information – much like they share with a shop assistant so that he or she can offer a solution for their needs. For example:
- 80% of Americans who read promotional emails find it helpful when brands recommend products based on past purchases (sharing of transactional history).
- 71% want recommendations based on online browsing behavior (sharing of their data when visiting websites).
- 82% of consumers admitted they would buy more items via emails that had better personalization (sharing of profile information to get better offers).
- 82% said that if they were more relevant, more emails could be sent to them each week (making more time for real-time, relevant updates).
When it is used securely and legally, subscribers are willing to share more data, and they are willing to do so even when they know it will entice them to spend more money. Marketers rejoice! The perceived value of personalized email content is extremely high.
Automated relevance is the only scalable way
According to research by Venturebeat nearly 70% of marketers will use more email marketing in 2015 than in 2014. The value is high, therefore businesses everywhere are working to improve their emails and sending even more. There’s a firehose of content aimed at every inbox and only the best of the best will stand out.
The biggest concern marketers face today is the risk of subscribers drowning in a deluge of irrelevant content. The only solution is to achieve greater relevance. And that’s a problem addressed by a Recommendation Engine.
The goal of a Recommendation Engine is to power a customized one-to-one interaction that surprises and delights in a scalable way. It’s the value-added feeling, not the technology, that is more apparent to the end consumer.
Consider the significance of executing customized one-to-one experiences at scale. One email marketer can now reach 100,000 customers with the press of a single send button and each person will simultaneously feel like the interaction is all about themselves. It’s automated relevance – and it’s the key to running a great email marketing program.
Most online content can be emailed as a recommendation
Today’s Recommendation Engines are more advanced, more intelligent, and pushing new limits to keep up with the consumer demands for more and better content. The best way to begin understanding the technology is to start with the result.
Recommendation Engines generate a set of items (the recommendations) that are considered appropriate for a small, well-defined audience (a micro-cluster of customers). With the more advanced systems, the recommendations can be precise enough for an audience of one.
The recommendations themselves can be any kind of content found online. Recommendations can be as diverse as:
- Products in e-commerce shops
- Articles, infographics, and slide decks from brand publishers
- News articles from media outlets
- Brochures from different insurance types
- Online educational materials for university students and alumni
The recommended content is then delivered through a channel such as an email campaign or website display. Depending on the type of business, potential targets for the content produced by a Recommendation Engine can be users, consumers, visitors, prospects, subscribers, and customers — in short, people engaging with information and brands online.
Behaviors indicate likelihood of interest
The science of producing predictions about what people want is based on lots of data. For a Recommendation Engine, this data can be acquired in many different ways, such as manual upload, FTP batch data upload, or other connectors for getting data into the model (ex. an API).
A more lightweight, real-time technique to capture data is to add a bit of code to the header of your website. Google Tag Manager also offers great capabilities for handling this if there are reasons to not add the code directly. The code can capture all the personal behavioral and content information that’s needed in real-time, and continuously send it to the engine for analysis.
For example, things that are particularly interesting are items that have been seen in conjunction with other items, things downloaded in the same session, and content browsed before and after a purchase, to name a few. These relationships are known to the engine as interesting, because algorithms have been designed to look for them.
Predictive algorithms look for relationships
There are many algorithmic methods (math-based instructions for solving a problem) to generate recommendations, including:
- Item Hierarchy (You bought a set of golf clubs, therefore you also need golf balls)
- Attribute Based (You like action-packed, non-violent, science-fiction movies with a strong female hero)
- Collaborative Filtering with User-User Similarity (People like you who bought opinionated t-shirts also bought fashionable combat boots)
- Content-Based Filtering with Item-Item Similarity (“Kill Bill” is similar to “12 Monkeys” therefore you will like watching it)
- Social+Interest Graph Based (your friends like Angry Birds so you’ll like Angry Birds)
- Model Based (pattern recognition for implicit behaviors combined with machine learning)
Collaborative Filtering and Content-Based Filtering
The topic of predictive algorithms can get deep pretty fast, but having some technical knowledge is the basis for smarter business decisions and profitable investments, so let’s break it down into different practical business cases.
Collaborative filtering
Collaborative filtering, also known as behavioral clustering, is a method that builds a data model based on a person’s past behavior, as well as similar historical activities by other people. For example, items previously browsed, searched, clicked, “liked,” downloaded, purchased, and preference ratings given to those items.
A principle behind collaborative filtering assumes that consumers are likely to enjoy items similar to those they’ve already purchased or downloaded, etc. It then follows that they will also demonstrate similar patterns and take actions consistent with the people that they are “the most like.”
Let’s consider a simple example of collaborative filtering based on three visitors to an e-commerce shop:
Jake got a beach ball and sunglasses
Funmi got a bikini
Nick got a beach ball
What else would Nick want?
The next email that Nick gets from this store will include a pair of sunglasses as a recommendation. Based on the data available, Nick looks more similar to Jake than he does to Funmi. The Recommendation Engine does not need to know that Funmi is a girl and the other two are boys, the data patterns actually reveal the item that is best.
It looks pretty simple, so you might be wondering why a predictive and automated system is needed to produce the answer for smart marketers? This example is super basic for the purposes of conveying the logic. Adding just one more person-scenario significantly increases complexity:
We could guess that the next email Joanna gets will contain a bikini, because she and Funmi are both female and the patterns in the data may pick up on that. But it could also be that Joanna is more similar to Jake because they’re both from Australia, and according to the data from dozens of Australians that shop at the store, everyone Down Under wears green shades.
There can be infinitely more products in the store that have relationships that are relevant, and infinitely more customers and website visitors with patterns to take into account. Furthermore, this example only looks at an order when in reality much more online behavior is also quite important. Finally, all of this needs to be assessed and updated continuously and in real-time in order to be most relevant.
Collaborative filtering is an algorithmic method that can reverse engineer and understand your customers down to the individual level. The patterns in behavior of visitors to your website are a rich source of information about who your customers are and what they value from you. That’s collaborative filtering in a nutshell.
Content-based filtering
Another method, content-based filtering, also known as product clustering, is a method that uses a set of specific characteristics related to an item (tags, categories, pricing, and other less defined attributes) in order to identify and recommend additional items with similar properties.
For example: items categorized as male may have a higher likelihood of being recommended with other items categorized as male; clothes tagged as red will be more likely to show up with other red items; items priced at a premium will be grouped with other premium products.
Again, it sounds simple in theory but the complexity is in the scaling. Content-based filtering is especially important to ensure that the data model takes a big-picture view of your entire product or content “catalog.”
Both the collaborative filtering and content-based filtering methods have been around for a while, but they remain the most advanced ways to produce recommendations — and the real magic happens when the two are put together.
Hybrid Recommendation Engines offer better personalization
A good Recommendation Engine strives to provide as much personalization as possible. For example, any mix of items on a website should be able to be assembled as a set for one individual. To get closer to achieving this capability, “hybrids” combine multiple algorithmic methods.
Many of the methods work quite well in tandem to process the data streams they are receiving. In particular, collaborative filtering and content-based filtering are often paired-up. In addition, there are plenty of other conditions and variables that are layered into the algorithms that make the recommendations right for a particular vertical or business.
The mysterious “Black Box” of Recommendation Engines
Companies often call the variable part of their technology “the secret sauce,” and will not usually share specific details about how it works (a.k.a. black box technology). That means there’s much more to a good Recommendation Engine than accurate predictions alone — those other variables must be considered.
The reason why hybrids and secret sauce are needed is again information overload. Yes, content continues to penetrate every aspect of the consumer experience. But a new trend that is not always acknowledged is that the content has also grown in quality. Marketers everywhere are starting to get really good at content. The impact is that it has become increasingly difficult to precisely nail that perfect set of three recommendations for a person.
Today, there are 30 items that could be recommended and rather well-received. So, how do the ten companies that each made three recommendations differentiate from each other? The way to gain advantage is to blend the models together, add some “secret sauce,” and make the recommendations more relevant and therefore more performant.
That’s a key point to discuss with vendors of Recommendation Engine technology. Even if the black blox details are proprietary, marketers should feel comfortable asking about the industry-unique aspects that a data model has been designed to address.
Machine learning is essential for marketing sanity
Machine learning is an aspect of Recommendation Engine technology that is found in the best systems. It’s an advantage because when done well, it means there are feedback loops that helps the predictive model learn from how the subscriber reacts (or does not react) to the recommendations.
The reason why marketers should care about machine learning is because it’s the part of the recommender technology that automatically drives content optimization down to the individual level, at-scale. It’s advanced computing power means data analysis which used to take days can now be done in just a few seconds in the cloud. In contrast, think of all the AB tests your teams have run, and the drain on resources that can be.
Furthermore, machine learning in many cases also means that the data model benefits from information processed across many businesses as well as individual websites. It identifies which algorithms matter more or less for a specific scenario or vertical, and fine-tunes itself to become more brand-customized.
A Recommendation Engine that learns directly from businesses as well as customers across website and email touchpoints, results in a “customized algorithm” that is constantly getting smarter about what matters most for a company’s success.
Advice for buying Recommendation Engine technology
Recommendation Engines can provide a big boost to your campaign performance, and there are dozens of commercially available systems that email marketers can use to optimize their email programs.
Some have native functionality integrated directly into the email service platform along with a full set of technology for other marketing activities ($$$). Other email platforms and marketing automation companies work with partners who specialize in predictive and contextual intelligence and provide the technology as an add-on software service ($–$$). And for those who like to code and customize, there are open source recommender technologies that can be used in a variety of ways ($–$$$$).
Regardless of your budget and business needs, there are several questions marketers should ask related to data, model design, speed, onboarding, simplicity and industry relevance.
In particular, some important things to look for include:
- The use of implicit as well as explicit data as a basis for the predictions
- A hybrid use of algorithmic methods that take into account patterns in subscriber behavior as well as relationships between content items
- Real time data capture, continuously updated recommendations, and the speed of delivering fresh content into an email
- Manageable integration requirements in terms of resources and time as well as cost
- An intuitive and easy user interface with quick learning curve
- Case studies and data that demonstrate proven results in your vertical
Let’s keep marketing in the martech buying decisions
Recommendation Engines were developed to cut through the clutter and help people find exactly what they want. They were also designed to help marketers easily and quickly deliver a hassle-free and enjoyable content experience to all customers, no matter how many. This technology is in the early stages of uptake and will dramatically change email marketing on a broader scale in the next 12-24 months.
Of marketers surveyed as part of Econsultancy and Adestra’s Email Marketing Industry Census 2014, 78% predicted that within five years, all email will be integrated and personalized. The report also found that “one in three companies are already engaging in content personalization, a 27% increase from last year, with 37% planning to include this as part of their email marketing activities.”
Therefore, CMOs and email marketers should be thinking about email content personalization if they haven’t already, and how they will add a Recommendation Engine to their marketing programs in the next 12 months. This is no small task, but it doesn’t have to be onerous.
At first Recommendation Engines can seem daunting, especially to a less-technical marketing person. The math and analytics involved in making “big data” based systems work is pretty advanced even in its simpler forms. However, that complexity should not keep marketers out of the board room when important technology decisions are being made.
The business perspectives and knowledge of customers coming from the marketing team are critical inputs for the buying process when it comes to new email technologies. There are some important basics that this article has hopefully made easier to grasp, especially if you’re involved in evaluating or working with technology vendors for your company.
Do you still have more questions? No problem at all – simply ask in the comments below and I’d be happy to get back to you, or tweet me with your question – @startupsandy.
This article was written by a marketer for marketers, but it was only able to be completed thanks to several technical experts who gave great input and advice. Thank you very much to senior developers Kacper Bielecki and Christoph Bünte from AVARI, Parry Malm from Phrasee, Jordie van Rijn from EmailMonday, Paul Shriner from AudiencePoint and Jason Brett from Silverpop.
