All or Nothing?

By David Loshin One of the most frequently referenced dimensions of data quality is completeness. At a formal level, completeness implies rules specifying mandatory assignment of values to particular data elements. In layman's terms, that specifies rules to make sure critical attributes are populated with values. Now there are a few things to think about here regarding the critical nature…

Continue Reading

Get it Right the First Time

By Elliot King People generally think of data quality as a remedial exercise. During the ongoing course of business, for a variety of reasons, companies find themselves with incorrect data. The goal of a data quality program is to identify the incorrect data and fix it. And while data errors inevitably do occur, an essential element of a data quality…

Continue Reading

Normalizing Structure Using Data Standardization for Improved Matching

By David Loshin In my last few posts, I discussed how structural differences impact the ability to search and match records across different data sets. Fortunately, most data quality tool suites use integrated parsing and standardization algorithms to map structures together. As long as there is some standard representation, we should be able to come up with a set of…

Continue Reading

Data Quality Management Mistakes to Avoid

By Elliot King Everyone wants high quality data and it seems that goal should not be so hard to achieve. The need seems obvious and there are plenty of good tools that can be put to work in the effort. Unfortunately, it is just not that easy to set up a successful, ongoing data quality program. The first mistake companies…

Continue Reading

Structural Differences and Data Matching

By David Loshin Data matching is easy when the values are exact, but there are different types of variation that complicate matters. Let's start at the foundation: structural differences in the ways that two data sets represent the same concepts. For example, early application systems used data files that were relatively "wide," capturing a lot of information in each record,…

Continue Reading

Moving to Action

By Elliot King The first step in a data quality program is to assess your data. Whether you opt for data profiling or some other assessment mechanism, this part of the process consists of systematically identifying exactly where the problems can be found in your data sets. While assessment is obviously the first step, it should be just as obvious…

Continue Reading

Non-technical Employees Need to Understand Data Quality

By Elliot King Too often, data quality is seen as a technical issue. You know the drill: profile, measure, remediate, integrate, augment, control and so on. In many ways, a company's data is like an ecosystem, and the data quality team is analogous to the environmental specialists. They conceptualize an ideal state for data, measure the actual state, and then…

Continue Reading

Data Cleansing and Simple Business Rules

By David Loshin Having worked as a data quality tool software developer, rules developer, and consultant, I am relatively familiar with some of the idiosyncrasies associated with building an effective business rules set for data standardization and particularly, data cleansing. At first blush, the process seems relatively straightforward: I have a data value in a character string that I believe…

Continue Reading

In a Global Economy, a Global Solution is Vital

By Patrick Bayne Data Quality Tools Software Engineer Heidelberglaan 8 3584 CS Utrecht If you were given this address, how would you know it was valid? Is it formatted correctly? How long will it take you to verify? For years businesses have understood a need for address validating solutions, because clean, accurate data is essential. Without accurate and consistent data,…

Continue Reading

Communication is a Key to Data Quality

By Elliot King Too often, data quality is seen as a strictly technical issue. Data quality problems must be identified, assessed and then rectified, and that process is best managed by experts using the right tools. But communication may be the most important element in a data quality program. A data quality program can only succeed if all the stakeholders…

Continue Reading