Do you always have confidence and trust in your data to make important marketing decisions?

building trust in data to make marketing decisions

We are asking the question because we expect the key currency of the new post-COVID economy will be trust, and trust in data. 

Imagine you are amid your biggest campaign of the year, you are explaining the results to your leadership team, and are faced with questions like “how do you know”, “is the data right” or “why didn’t that campaign reach all of our intended audience”? I’m sure you have been in these types of scenario, which happen every day in the life of a marketer. 

SnapLogic [1] recently published an intriguing research report on how data distrust impacts analytics projects and decision making, which highlighted:

  • 77% of IT decision makers do not completely trust the data in their organisation for accurate, timely, business-critical decision making.
  • 76% of IT decision makers report that revenue opportunities have been missed due to a lack of data insights. 
  • 83% find data is not available at the time it is needed
  • 53% of mid-size companies suffer from too many disconnected data sources.

So, we would like to focus attention on some of the key data and insight issues faced by mid-size B-to-C companies in the UK and make suggestions around how they can be resolved.

Our experience is that these problems often have three separate causes:

  1. Customer data availability and quality
  2. Availability of skilled data analysts equipped with the right analytical tools
  3. A failure by the decision makers to frame the right questions for the analyst

 

Customer data availability and quality

The SnapLogic report reveals that 53% of mid-size companies have too many disconnected data sources, while 40% have poor integration of data sources meaning that data is missing or incomplete.

A typical B-to-C marketing department will often be looking at a distributed data situation with multiple silos like this:

data flow of different silos
Distributed data flow with multiple silos

The problem with this configuration is that there is no place for maintaining the overall customer picture, just pieces of the jigsaw in different places. So, it would be well-nigh impossible to answer questions like:

  • where am I acquiring my higher value customers from?
  • how is my latest email or catalogue campaign performing when most orders are placed without source codes via the website?
  • how do I know which of my dormant customers are worth trying to reactivate?
  • how many of my orders are coming from customers recruited this year, last year, and the years before?
  • how do I understand the ROI I am getting from each acquisition channel?

… and many more.

One solution to the data availability and quality problem is to introduce a customer data platform (CDP) that ingests data from all available online and offline sources and builds a single customer view. Marketers are increasingly focusing on first-party data to drive better customer experiences and marketing outcomes. More than half of marketers surveyed by Winterberry Group say cross-channel audience identification and matching is their highest priority. In fact, investment for identity resolution is projected to reach $2.6B in 2022, according to Forrester Consulting. So, it is no surprise that brands are taking this seriously and most want to create a single customer view.

A major part of what a CDP does is to undertake identity resolution; the process whereby data arriving from different sources is matched together using a range of different personal identifiers such as email, mobile, postal, cookie ID, customer number. The key consideration here is that the CDP needs to maintain for each customer a table of all known personal identifiers so that when a new one is introduced it can where possible be matched in.

The CDP then provides the single central source of truth about customer behaviour from which dashboards can run and analytics can be undertaken; it will also be used for activating multi-channel customer campaigns and for resolving GDPR questions.

 

Availability of skilled data analysts equipped with the right analytical tools

A large organisation like a bank will have upwards of 50 skilled data analysts, but with many smaller organisations it is often the case that they have one or none and rely on external resources to support them.

There are several reasons for this. Cost is a key factor and linked to that, the difficulty of putting a precise number on the value that a good data analyst can bring. Next the demand for analysis normally fluctuates, and a single analyst would always be facing feast or famine. Also, data analysts usually prefer to work in small teams so that they can discuss problems and learn off each other. Being the only data analyst in an organisation is a lonely position, and often they end up just cranking out reports and become dispirited.

A lot of the reporting can be resolved by introducing dashboarding technology like Tableau or Microsoft Power BI, but these tools still need to be configured to produce the right information.

However, dashboards and data visualisation tools can only take you so far. If you need some more complex analysis, or if for instance you want a propensity model to predict the next best offer to make to each of your customers, then a data analyst becomes essential.

To undertake more complex analysis the analyst will need good tools like SAS, SPSS, or R.

For the smaller organisations, the right solution could then be to outsource to an analysis company or to independent contractors, until demand has grown to a scale where the function can be brought inhouse.

 

A failure by the decision makers to frame the right questions for the analyst to answer

This issue is less frequently discussed but, in our opinion, not one to be brushed under the carpet.

A considerable amount of the work done by data analysts is wasted because someone does not spend sufficient time thinking about what the real problem is that the analyst should be trying to answer.

Einstein said…

“invention is not the product of logical thought, even though the final product is tied to a logical structure”.

Unravelling this statement in the context of customer marketing, we would suggest that the person who requests the analysis will succeed if they allow their imagination to fire up a range of conjectures that the logical analyst can then set about proving or disproving.

Some analysis is more mundane, but when for instance a business is contemplating several alternative strategic changes then the analyst should look at all the different scenarios that these would potentially deliver, and, as far as possible, provide the business with an understanding of their relative merits.

 

So, in conclusion…

From our experience it is fair to say that a large proportion, probably more than 50%, of medium size organisations involved in B-to-C marketing that we encounter have their customer data disconnected and spread across multiple systems. This is a problem that can be solved, and the costs are not frightening. A CDP will usually cost no more than 0.5% to 0.75% of sales.

However, setting up from scratch an internal insight and analytics department is challenging, and outsourcing will make economic sense until demand has grown. Also, the outsourced provider should have analysts with a very wide range of experience and skills.

And then how to ask the right questions of the analyst? We would recommend giving the analysts scope to try out different approaches, and to look at different angles to a question. Like this they are far more likely to land on an interesting and valuable solution.

[1] Data Distrust Report – the impact of data distrust on analytics projects and decision making published by SnapLogic in 2020, based on interviews with 300 US and 200 UK IT decision makers.

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed. Our ambition is to help our clients stay empowered and ahead in this challenging environment.

How does a customer data platform work?

how does a customer data platform work?

How exactly does a customer data platform work, and help marketers leverage data to gain a better, more accurate understanding of customer behaviour?

Ingesting and integrating data

The first element in understanding this is ingesting data. CDP’s ingest customer data from multiple sources. Typically, these will include website data, paid digital, transactional, direct mail, retail, email, and call centre. All data received by a CDP will relate in some shape or form to a customer. The data is usually sent to a CDP using an API or via an SFTP site.

Customers have multiple identifiers and these change over time, such as mobile phone number, email address, cookie ID, postal address, customer reference or landline number. This data is collected, and these identifiers are used to generate a single customer view also known as ‘Identify Resolution’. For example: if someone logs into your website with their current email, but with a different cookie ID, then the new cookie ID is added to that particular customer record on the assumption that they are using a new device. Equally, if a new transaction record is received with the same customer reference, but a new address, then a new address is added on the basis they have either moved or added an extra residence.

As new data is ingested, each record goes through what is called the ‘purning’ process. This is the stage at which the record’s personal identifier(s) are matched against all other customer records that are held in the CDP until a match is or is not found. At this point the data may be matched into an existing single customer view or a new one created. Each recognised customer is given a permanent unique record number or ‘purn’.

Identity resolution

Is at the heart of a CDP and is central to all the rest of its functionality. A good CDPs’ functionality is rooted in the knowledge that people have multiple identifiers, and that these identifiers can all change. Over time many or all of these identifiers are likely to change for an individual. The CDP should keep a history for every one of every version of these, although regarding the latest versions as most likely to be current. This collection of identifiers is what it calls on to build the single customer view.

The data in a CDP is held in what is called a schema. This is the way in which the data is organised. Every organisation using a CDP will need their own schema although within an industry, schemas will have a lot of similarities.

Engineering derived data

Engineered data is important for the value it provides for selecting specific customer groups for communications or developing customer insight. It can comprise any variable that can be calculated using an algorithm or other means from the raw data in the customer data platform.

Data engineering can take many forms, from simple examples like banding variables such as age, to more complex ones like keeping a counter on customer’s total historic value. A major use of engineered data is in developing and recording scores derived from algorithms such as propensity models.

An example of an engineered data field is where we want to know what each customer has contributed to a business after the cost of acquiring them. We can then:

  • Use historic purchase data for each individual in say their first and second year since recruitment
  • Deduct the cost of acquisition which can be derived the channel they came in from
  • Deduct the cost of communications sent to them in the same period which is held in the contact history area
  • Calculate an individual customer contribution

Engineered data is updated at an individual level every time a relevant event happens; so, each new home shopping purchase, eCommerce transaction or physical retail transaction can lead to a changed score in the engineered data section. A great benefit of engineered data is that it allows you to base axis for charts or selections for campaigns on these additional variables.

Analysing customer data

A CDP is essential for gaining a full and accurate understanding of customer behaviour. For instance, without a CDP that combines web browsing history with transactions, it would not be possible to understand the relationship between the two. Again, if individual contact history is not held against a customer record then the effectiveness of campaigns that are sent to the customer, and to which the customer may respond through different channels, cannot be accurately measured.

The CDP builds the single customer view, and it is against this that customer analysis can take place. It provides the dataset that becomes the one authoritative source of information about customer behaviour for an organisation. With this in place decision makers have a firm basis on which to proceed.

There are so many aspects to the analytical tools that can be used to analyse customer data that there is little merit in trying to list them all. Some are built into the CDP and others require data to be first extracted from the CDP and then transferred to them. What matters is that they have the best possible customer data set to analyse.

So, the results from customer analysis form the basis on which key decisions about customer marketing can be made. These include such areas as:

  • Customer acquisition (targeting and channel choice)
  • Digital planning
  • Product development
  • Customer relationship management
  • Salesforce management
  • Pricing

Even corporate mergers and company valuations.

Given how important these decisions are, it makes good sense when designing a CDP to first start with a list of the kind of results that will be required from customer analysis so that for instance data is held with sufficient granularity to make these possible.

Connectivity to external systems

The CDP can support other systems in their personalisation and management of customer communications. Typical examples are:

  • Providing customer selections for email marketing systems
  • Customer segmentations for web personalisation technology
  • Names and addresses for postal marketing
  • Target audiences for social media

So just as the CDP ingests data from multiple sources it also provides selected data to external systems. These connections are usually made via an API or via transfer of data to an SFTP site.

Delivering personalised customer experiences

Within the CDP we expect to find functionality for the selection of specific customer groups either on a one-off or on a recurring basis. These groups are usually selected for output to external systems that manage the actual communications. The selections themselves can be simple based on Boolean logic rules, or they may be more complex based on propensity scores applied within the engineered data. They can also be based on triggers, such as a new customer having just been recruited.

The CDP needs to enable these different types of selection, and crucially record what contacts each individual customer has been selected for. Functionality is also required for test and control, and for including source codes with the selection.

Associated with delivering personalised customer experiences needs to be functionality for measuring the results of campaigns. This is often automated within the design of the CDP and should always include the ability to attribute results such as orders back to campaigns, even if they respond through different channels.

What are the costs for a customer data platform?

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Our data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed, and our ambition is to help our clients stay empowered and ahead in this challenging environment.

What is a customer data platform?

So what exactly is a customer data platform?

As customers we generate massive volumes of data as we engage across multiple channels using different devices which makes it challenging to capture, integrate and activate this data effectively.

Data repositories are often siloed and not integrated with each other or allow easy transfer of data to marketing platforms. Let us now throw in some GDPR and updates from Apple with identifier for advertisers (IDFA) and the deprecation of third-party cookies by Google.

Today’s customers simply assume that your company knows and remembers who they are, what they have done, and what they want, always and across all channels. Their expectations are high, and tolerance is low. So it is not surprising to see that many marketers have made a unified customer experience their highest priority.

What’s the problem with data?

Not having a single customer view creates many challenges including:

  • Making it more tedious to activate campaigns to the right audience and report on them in a timely manner.
  • Degrades customer experiences.
  • Introduces privacy concerns.

Marketers and marketing technologists know that gathering and acting on unified customer information is not easy. In fact, only a small percentage of companies have achieved this and can truly operationalise their first party data. The rest are battling with technology, strategy, budgets, organisations, staff skills, and other obstacles to success.

Traditional methods for collecting that data into unified customer profiles, such as an enterprise data warehouse, have failed to solve the problem. Newer approaches, like “data lakes”, have collected the data but failed to organise it effectively and enable marketers to activate the data into owned and paid marketing channels.

The Customer Data Platform is an alternative approach that has had great success at pioneering companies. The process of collecting and unifying the data is known as identity resolution which is a core building block for enabling better customer experiences and optimised marketing effort. A CDP puts your marketing team in control of the data unification project, helping to ensure it is focused directly on marketing requirements.

CDPs apply specialised technologies and pre-built processes that are tailored precisely to meet marketing data needs. This allows a faster, more efficient solution than general purpose technologies that try to solve many problems at once.

Customer Data Platform Definition

“A Customer Data Platform is packaged software that creates a persistent, unified customer database that is accessible to other systems”.

This definition has three critical elements:

1. “packaged software”: the CDP is a prebuilt system that is configured to meet the needs of each client. Some technical resources will be required to set up and maintain the CDP, but it does not require the level of technical skill of a typical data warehouse project. This reduces the time, cost, and risk and gives business users more control over the system, even though they may still need some technical assistance.

2. “creates a persistent, unified customer database”: the CDP creates a comprehensive view of each customer by:

  • Capturing data from multiple systems.
  • Linking information related to the same customer.
  • Storing the information to track behaviour over time.

The CDP contains personal identifiers used to target marketing messages and track individual-level marketing results.

3. “accessible to other systems”: data stored in the CDP is then made available to other marketing systems for analysis and to manage customer interactions.

What should a customer data platform do?

In essence, a customer data platform combines all your customer data from online/offline sources and unifies this into a single customer view to enable cross-channel activation and personalisation.

A CDP should integrate into existing and future marketing/advertising technology enabling you to decide which channels to communicate with your customers.

It should enable automated reporting of activity on your key marketing metrics. And of course, it should support GDPR enabling you to check customer consent, action subject access requests and the right to be forgotten.

How does a customer data platform work?

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Our data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed, and our ambition is to help our clients stay empowered and ahead in this challenging environment.

The identity resolution process: are customers turning into chameleons?

It may be noticeable that, like chameleons, they are becoming harder and harder to identify. And there is a reason for this. They are constantly changing their personal identifiers, like email, mobile numbers, or cookie IDs. The process to properly identify individuals we call “identity resolution”, and failures in identity resolution may sometimes have quite negative consequences.

Customers actively dislike not being recognised, for instance being treated as a new recruit when in fact they have been buying from you for years, or being sent the same message twice, and in addition to that there is a cost for the organisation with added communications costs.

Lack of good identity resolution processes also makes a nonsense of trying to calculate customer lifetime value or undertaking forward business planning based around your expected rates of recruitment and attrition.

 

So what does a good identity resolution process consist of?

We see it as matching all available personal identifiers, from every one of your customers, to get the best possible chance of joining your customer data inputs from multiple sources into actual customer records.

This used to be a relatively straightforward task when the main personal identifier was the postal name and address, although that in itself posed some considerable challenges.

With the usual mix of badly typed addresses, varying address structures, and incorrect postcodes we often find there is a problem just within name and address matching. In a recent case we found 25% name and address duplicates.

But the postal address is just one of multiple personal identifiers, each of which can change at any time.

We have all become identity chameleons, changing our mobile numbers, emails, cookie IDs etc with great regularity.

There is however a relatively simple solution – just keep hold of all the personal identifiers you have been able to link to each individual since you first recognised them, so that you have the best possible chance of identifying them when the reappear.

This is exactly what our cloud-based customer data platform does with the data it ingests; as each individual item of customer data is taken in, its identifiers are matched across the entire customer base.
an example of cdp identity resolution

If you think that you may have an identity resolution problem with your customer data, we can offer you a very low-cost solution; we can trial match all your customer data sources together in UniFida, and report on the amount of duplication that exists between them.

This will tell you how many customers you actually have, and how many duplicates you are carrying.

 

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Our data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed, and our ambition is to help our clients stay empowered and ahead in this challenging environment.

CRM identity resolution: Why it’s not really what it claims to be!

On the surface, CRM systems give the appearance of covering most things you might need when you are looking for technology to help you manage relationships with customers. But the reality is that they have been designed to only deliver the basics, so CRM identity resolution often fails to deliver a full single customer view.

To understand why let’s take a deeper look into what CRM identity resolution does and doesn’t deliver.

  • CRM systems manage what they call ‘contacts’ but these are usually only matched together using email addresses. Customers in fact have many different identifiers like mobile phone numbers, cookie IDs, postal addresses, account numbers etc and there are usually multiple versions of each relating to a single customer. All versions of these personal identifiers need to be stored so that customers reaching you through multiple channels can be properly identified.
  • Next CRM systems only include a part of your customer data. Your customers browse your website, they visit specific pages, they arrive there via different forms of on-line advertising, and all this vital information needs to be included in your single customer view.
  • CRM systems may report sales at a company or salesman level but they don’t look at longer term customer value, or how different channels and different customer recruitment propositions bring different types of customers with different kinds of customer needs.
  • CRM systems may undertake bulk email campaigns, but these are not usually synchronised with other channels such as post or SMS; also given that they don’t generate any added value customer knowledge such as probability of response or likelihood of attrition, the selections they make can only be via quite simplistic filters.
  • CRM systems were born out of the need to enable sales forces to record their activities and results; they were not designed to ingest data from multiple online and offline sources, and deliver a complete customer view.

Over the last five years or so a new breed of technology, Customer Data Platforms or CDPs, have been developed to allow you to properly relate to customers, and to make all aspects of your marketing accountable so that for instance an online order can now be linked back to the several online and offline contacts that preceded it.

Find out more about what a customer data platform is.

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Our data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed, and our ambition is to help our clients stay empowered and ahead in this challenging environment.

Are you having a digital identity crisis?

Back in the ‘70s, it was de rigour for cool people to be undergoing an identity crisis, and this involved trying to fathom out who the hell one was, why one had been put on the planet in the first place, and what other people would make of one’s extraordinary persona. Roll forward to today and you are more likely to have a digital identity crisis which involves losing one’s user names and passwords, or worse still having someone else steal them.

The very concept of identity has mutated into a set of letters, numbers, and codes by which organisations can recognise us as unique individuals; so much for the soul searching of previous generations.

The recently enacted GDPR tacitly assumes that identity crises have been banished, that organisations and individuals can immediately recognise each other, and that there is never a problem in tying together all the personal information that lies behind different doors, accessed by recognising identifiers like email address, cookie ID, mobile phone number etc.

This clearly is not the case, and for it to change, organisations are going to have to put in place a much more sophisticated process for identity recognition; something that we call a digital passport.

As we frequently change many of our identifiers e.g. get a new mobile phone number, different email, new tablet etc. an organisation needs to maintain a historic record of all means by which we have previously communicated our persona to it. This historic set of identifiers can then give us the best chance of recognising an individual when they appear through one of our many communications channels.

Developing this kind of digital passport is something that we have made a key component in the design of UniFida, our cloud-based single customer view technology.

Are you having a digital identity crisis? Contact us if you’d like to find out more about how we can help your organisation.

UniFida logo

UniFida is the trading name of Marketing Planning Services Ltd, a London based technology and data science company set up in 2014. Our overall aim is to help organisations build more customer value at less marketing cost.

Our technology focus has been to develop UniFida. Our data science business comes both from existing users of UniFida, and from clients looking to us to solve their more complex data related marketing questions.

Marketing is changing at an explosive speed, and our ambition is to help our clients stay empowered and ahead in this challenging environment.