What is data enrichment?
‘Data enrichment’ is the process of adding new information to records in your database to make them more valuable — that is, enriching them.
The data appended does not always have to be purchased from an external party, although it often is. Sometimes, data enrichment processes are packaged as products or solutions, usually when they involve a proprietary, privileged or official data set, such as Australia Post’s National Change of Address (NCOA) product.
What’s the difference between data cleansing and data enrichment?
Data cleansing is the process of removing inaccuracies and merging duplicate records. It does not usually require appending new data. By contrast, data enrichment adds new data to an existing data set. However, these processes can interact with one another: because a data enrichment solution will usually attract a per-record cost, it can be more cost effective to cleanse the data to the greatest extent possible prior to enrichment.
How does data enrichment benefit businesses?
Data enrichment makes a business’s data more valuable by appending information that improves the outcomes of other activities, such as customer communications or business decision-making.
One common form of data enrichment with which most of us will be familiar is a credit check. A lender receives an application for a product. That application contains first party data like a name and licence number, and it enriches that application with third party data in the form of a credit check, which then helps the lender make a decision about the outcome of the application.
A second example of a data enrichment process, and one that’s common here at DCA, is the Australian Death Check. The Australian Death Check is a government initiative that uses data sourced from the registers of births, deaths and marriages across all states and territories to curate the only official source of death data for Australia. With an approved broker, an organisation can use the Australian Death Check to flag deceased contacts within their databases — useful for preventing fraud, meeting regulatory requirements and trimming customer engagement campaigns.
These are just two examples of how data enrichment processes benefit businesses, but data enrichment is used across marketing, sales, customer service, compliance and business intelligence activities. Use cases are virtually limitless.
Case study: ISUZU Australia
The utility of data enrichment is highly varied and applies equally across all sorts of businesses.
One example of a successful data enrichment project is the one DCA shared with ISUZU Australia. The gap that ISUZU found in its data was that it had limited information about who it sold to, so it couldn’t see what businesses or sectors the top ISUZU customers were in. This problem was fixed with an enrichment process that appended data like ABNs, business addresses and industry codes to ISUZU’s existing records.
How does data enrichment actually work?
Start with a goal
Every data project begins with defining a goal. Having clear goals helps contextualise every step that comes after, and in this case, it’s where the business finds out whether or not enrichment is the right solution. For example, an enrichment processes needed to support a bulk direct mail fundraising campaign will differ tremendously from one used to create technographical profiles of business customers.
Whatever the goal, it should be “SMART” — specific, measurable, achievable, relevant, and placed within a meaningful timeline. When defining goals, it can also help to check in with key users of the data to be enriched and get their perspectives, because they will often have valuable, if granular, views on realistic goals, timelines and performance indicators.
Find the gaps in the data
The data currently held by the business must be reviewed for gaps. At this step, the organisation examines its existing data assets and determines what kind and quantity of additional data should be appended in order to support its goal.
Source enrichment data
Once the business is aware of what its goals are and where the relevant gaps appear in its data, it’s time to think about were to source the information that will fill those gaps.
In some cases, data enrichment can be a process of matching disparate internal data sets over which the business already has ownership. But in other cases, an organisation might seek the services of a reputable data broker to enrich its data. Deciding on a source occurs on a case-by-case basis, and knowledge of both the desired outcome and where the gaps are will help determine which data source will be most appropriate.
Match up records
The process of data enrichment requires records to be matched based on attributes that are unique to that record, which can consistently identify them. In some cases, this is as simple as a phone number or an email address. In most cases, some of this matching process can be automated, based on clearly defined business rules, but it also usually requires some degree of human oversight for ruling on edge cases and quality assurance before merging.
Maintain the data
Data is rarely static. Any given record typically describes something that is subject to change in the real world: contacts move, SIM cards are lost and replaced, donors pass away, and even businesses close down or merge.
Data degrades at the rate at which the thing it describes changes. So, depending on the type of data the business holds and what it wishes to append, the rate of data decay differs. We may find that, for example, a person’s date of birth is never subject to change, a person’s name is only rarely subject to change, and someone’s business email address is relatively frequently subject to change. So, In addition to the collection of new data, the rate of decay is one of the most significant determinants of frequency of data enrichment.
To maintain complete and accurate data, the business must make an assessment about the kind, purpose, and relative changeability of the data it is using and how frequently it collects it.
Do you need data enrichment for AI?
The short answer is: yes. Artificial intelligence requires large amounts of complete, accurate data, and it also requires that this data be correctly labelled and described so that machine learning processes can “understand” it in context. If a business needs to keep its AI tools operating in a reliable and up-to-date way, then data enrichment is a valuable weapon in the armoury.
How often should you enrich data?
The answer to this question depends on what the data is and to what purpose it is put. An organisation could enrich its data at virtually any frequency and it could still be the right choice in context.
The following are all examples of reasonable enrichment exercises, which illustrate the wide range of appropriate frequencies:
- A company that conducts telemarketing knows it must check none of its prospects are on the Do Not Call Register at least every 30 days.
- A company wishes to segment its existing customers by the industry for the purposes of conducting a one-off campaign, and chooses to enrich its data with industry information just once.
- A non-profit organisation runs major fundraising campaigns at tax time and Christmas, and uses the Australian Death Check and National Change of Address enrichment processes every six months to reduce waste and raise ROI.
- A company’s CRM system is in constant use by a team of hard-working sales staff, so it chooses to integrate real-time enrichment of new leads that flow into the system.
