The data audit: what it is and why you need one.
Many organisations regard data as their most valuable asset. This is especially true for those that use their accrued data extensively to power AI agents or improve sales enablement.
And because data is such a valuable asset, it’s wise to make sure it’s fit for purpose. That’s where auditing comes in.
What is a data audit?
A data audit is an assessment that provides insight into how fit for purpose an organisation’s data actually is. During a data audit, you discover duplicates, track data flows, assess cleanliness and displacement, investigate security and process compliance and examine deliverability. The result is a bird’s-eye view of your organisation’s data landscape, which shows you where your strengths and weaknesses are.
Why would an organisation require a data audit?
Any organisation that captures and stores data should regularly audit the data it holds. The data could be used for purposes that range from knowing which employees still work at a company all the way through to supporting machine learning and artificial intelligence systems. Whatever the reason, a data audit is a way of finding out whether or not an organisation’s data is fit for purpose.
If the data is fit, that tells asset owners that their information is trustworthy, and so insights and outcomes derived from it are likely sound.
If it isn’t fit, the outcome of any task that relies upon that data is suspect, and remedies must be found and applied.
Data audits tell you about:
- The quality of the data overall, including where errors are found and how they’re introduced
- Compliance with regulatory requirements for the storage of the data
- Security, including how, where and why the data is shared with others and what parts of systems may be exposed
- Any problem areas in the data, so the organisation can avert complications before they happen
What do organisations risk by not undertaking regular data audits?
The major risk associated with failing to regularly audit data is simply not knowing.
When an organisation does not have complete oversight of the data it holds, that means the organisation can’t understand — and cannot account for — any of the risks that may be associated with that data. Risks about which you are unaware are risks with outsize potential to impact your operations.
For example:
- If there are data quality problems of which asset owners aren’t aware, the outcomes from using that data are likely to be poor. If problems with data quality aren’t fixed, errors get compounded when decisions are made based on the faulty data.
- If there are specific regulations that apply to the data stored (for example, personal or sensitive data), the organisation may not know whether or not its responsibilities are being met, and it may or may not, therefore, be exposed to compliance risk.
- Security risks about which the organisation is unaware cannot be addressed. Without auditing, it is more likely that awareness will arise in a situation in which personal data or valuable IP has already been exposed. It is far safer (and certainly cheaper) to discover and fix security holes than it is to risk an adverse cybersecurity event.
How is a data audit conducted?
A data audit is a process of systematically reviewing the current state of an organisation’s data.
Goals and purposes
The first step, as with any project, is to define the parameters of the task in accordance with the goals it is intended to achieve. In the case of a data audit, the questions to ask are ones like:
What is the purpose of the data that is stored?
In organisations that are very large, it can be challenging to maintain holistic oversight, so sometimes it’s hard to know all the uses internal stakeholders may actually have for data. Knowing why you collect data, and who eventually uses it for what, informs choices about how to manage that data. The very beginning of your data audit is a good time to make sure you understand the real uses of data in your organisation.
What standards does the data need to meet for it to be usable for the organisation?
We use data for all sorts of business activities. Forecasting, reporting, business intelligence and efficient resource allocation, regulatory compliance, service delivery, sales enablement and marketing campaigns are just a few among them. When you understand how your organisation uses its data, you can determine what standards it must meet to deliver on those uses.
What standards must it meet for compliance with wider regulations or laws?
This can be quite separate from the standards required for the data to be fit for its primary business purposes. For example, in Australia, the Privacy Act demands organisations over a certain size adhere to specific standards for the maintenance of personal data.
Data sources
A data source refers to where data is captured. For example, data may be passed on by a third party, generated automatically in response to a trigger, or manually entered into a system.
When auditing data sources, data taxonomy and validation rules are typically considered because both are applied at that point of capture.
Much like biological taxonomy, data taxonomy is the way data is grouped into hierarchical relationships to create structure in a database. A data audit will review whether or not the data taxonomy in place is consistently applied. It will also asses whether or not the taxonomy, as it exists, is appropriate for the goals and purposes of the organisation.
Data validation refers to assessment of whether or not records adhere to the structures and formatting rules set out for your data. For example, when you validate a list of emails, you know each of them will contain an “at” symbol (@). Anything called an email address that doesn’t include such a symbol may be flagged as invalid and earmarked for remedy. Adding an automatic validation layer can, as an example, reject birth dates that occur after a certain date or require correctly formatted contact information to promote standardisation.
Data flows
A data audit will examine data flows, which means it will track and review how data moves around the organisation. This includes information like who uses it, where they get it, what paths it takes and how it gets shared. These details have implications for how it may need to be organised and structured. It also offers insight into the security implications of current BAU data use.
Data quality
One of the key reasons to perform a data audit is for an organisation to examine the quality of its data. Common data quality factors include data accuracy and completeness, duplicate records and the recency of the data. Low quality data is not usually useful to an organisation.
Security
Data sources, data flows and data quality all influence data security. An audit reviews the way security operates within an organisation’s data systems. This is an opportunity to discover if the organisation’s data governance framework is being followed in day-to-day data management — in practice as well as in policy.
Security is always a trade-off against use, access and convenience, but a data audit also uncovers if there parts of your data flows that could be exposing your data unnecessarily.
Compliance
Data storage, use, maintenance and destruction may need to meet regulatory requirements.
Auditing helps organisations discover how current practices stand up to regulatory requirements, and if there’s any need to alter processes to ensure either current, or future, compliance.
Although it sounds counterintuitive, it may be wise to pay particular attention to data destruction when thinking about compliance activities. Organisations are often obligated to hold onto certain types of data for several years at a time. But, once that time has elapsed, it is good practice to avoid retaining data you don’t need. Destroying historical data can reduce exposure to risk.
Reporting
A data audit results in a comprehensive report on its findings. The report allows the people who have a stake in the organisation’s data assets to know what the state of the data is, how fit for purpose it is, and what recommendations might apply going forward.
The report also serves as a useful benchmark for further activities, as it provides a real measurement for data quality and usefulness to which you can refer when you perform your next audit or other data project.
Next steps
It is useful for an organisation to know what problems may exist in its data. But knowing about them is only the first step. Next steps will include reviewing what remedies exist for any of the problems that may have been uncovered.
This is also the time at which an organisation should be making plans for a repeat of the exercise. Although auditing can be laborious, its greater value is as an iterative process: by comparing reports, an organisation can see how its data quality and stewardship has improved over time, and it can review the longitudinal efficacy of any remedies that have been applied.
At DCA, data audits form an important part of the work we do for our clients. If you have questions, we have answers — feel free to contact us for a no-obligation chat about your data.
