Data Quality Trends for 2023

In its most recent Data Trust Survey, analyst firm IDC reports that just over a quarter (27%) of data practitioners fully trust the data with which they routinely work. As enterprises forge ahead with a host of new data initiatives, data quality remains a top concern among C-level data executives.

In its Data Integrity Trends report, Corinium found that 82% of respondents believe data quality concerns represent a barrier to their data integration projects. Over 50% reported that data quality is “very challenging” in their organizations, coming in at #1 on the list of data integrity concerns among the survey respondents.

Data quality remains top-of-mind today. Many companies invest heavily in artificial intelligence (AI) and advanced analytics, seeking a strategic advantage over their competition. Data democratization is receiving more attention than ever, and data analytics is becoming a central element in compliance, including ESG reporting. Data governance is going mainstream as well, prompting companies to focus more attention on managing data quality at scale.

Here are some of the most important trends in data quality for 2023:

1. High Volume Data Calls for a Scalable Approach to Data Quality

Cloud-based analytics platforms allow for large-scale computational processing at a level that would have been prohibitively expensive for most enterprises just a few years ago. At the same time, the digitization of business processes, the proliferation of mobile devices, and the advent of practical and affordable IoT sensors generate mountains of new data for analysis.

As the world grows more and more interconnected, the nature of data quality initiatives must adapt to scale. Two decades ago, most organizations might have struggled with a limited number of internal data sets. Duplicate records and decaying data quality – especially in customer databases – were the primary concerns for many. Today, the volume and variety of data quality challenges have exploded.

Today’s enterprises must approach data quality through a lens of scalability. The best data quality tools provide robust mechanisms for discovering, profiling, and cataloging data; and then building sophisticated business rules to define what good data quality looks like. Using that as a foundation, automation can bring potential data quality issues to the attention of line-of-business data owners to address problems as soon as they emerge.

2. Data Democratization Hinges on High Data Quality

Data democratization has the potential to help stakeholders throughout an organization make better business decisions. Doing so comes with some risks. If data quality is not up to par, democratization initiatives could lead to poor decisions, with potentially devastating consequences.

According to IDC’s most recent Data Trust Survey, trust in data quality tends to be lowest among front-line employees working in operations, human resources, and sales/marketing. Top executives, in contrast, are typically far more confident in the data they use to make important decisions.

This has important implications for executives aiming to increase data democratization within their organizations. When trust is low, utilization will likewise be diminished. Enterprises simply will not be taking full advantage of the data assets at their disposal, which amounts to a huge missed opportunity.

Proactive, scalable data quality programs offer two very important benefits. First, they increase overall confidence in data integrity throughout the organization. Second, they yield more accurate insights, which lead to better business decisions.

3. AI/ML Raises the Stakes for Data Quality

Enterprises are investing heavily in artificial intelligence and machine learning (AI/ML), but the algorithms that drive so many of today’s AI initiatives must be trained on clean, accurate datasets. Imagine, for example, that your customer database contains records that have addresses incorrectly formatted or missing incomes levels. Unless directed to do otherwise, a machine learning algorithm factors that data into its predictive models for purchasing behavior among various demographic segments. If your team is building demand forecasts with that data, the forecast simply won’t be accurate.

Although AI/ML has huge potential for creating business value, it also has the potential to go off the rails. Good data quality drives accurate results from AI, whereas poor data quality can create huge problems that may not be apparent until it’s too late.

4. Compliance Depends on High-Quality Data

Enterprises are spending increasing amounts of time, energy, and resources to comply with new reporting standards, contractual obligations, and government regulations.

Environmental, social, and governance (ESG) reporting is emerging as an increasingly important activity for many global enterprises, for example. Critics have pointed to what they call “greenwashing”, a skewed representation of ESG metrics that has largely been enabled by a lack of common reporting standards. Enterprises must strive to overcome those kinds of accusations, even when they’re unwarranted.

Government regulators, likewise, continue to ratchet up requirements for timely, accurate reporting to meet a host of new mandates. At the same time, large customers are demanding that their vendors need increasingly stringent standards, supported by routine reporting of key metrics.

In addition to helping with compliance, improved data quality is essential to minimize reputational and financial risk.

5. Data Quality Is Key for Effective Data Governance

Most enterprises today recognize that data governance is no longer a “nice to have” feature. For 2023 and beyond, it has become a de facto requirement for any enterprise that intends to use data to generate strategic and tactical business value.

Business data is growing more complex. The digitization of many business processes and digital consumer interactions as well as a wealth of geospatial information and demographic data add up to big opportunities. At the same time, increasing regulatory complexity brings new challenges, requiring that businesses of all sizes understand and control the information entrusted to them.

If data governance is to succeed in its mission of bringing order, trust, and compliance to a company’s data, then data quality is a non-negotiable first step. Moreover, data quality is not a “one and done” proposition, so the ability to manage data quality on an ongoing basis, at scale, is essential.

These five trends are driving more and more enterprises to find better ways of managing data quality proactively and routinely for high volumes of data.  Read our eBook 4 Keys to Improving Data Quality which informs you as to how to overcome the root problems of data quality, not just identifying specific types and patterns of quality problems, but also clarifying the roles of data quality management and data governance in resolving them.