Structural Differences and Data Matching

Blog Administrator | Address Quality, Address Standardization, Data Cleansing, Data Enhancement, Data Enrichment, Data Governance, Data Integration, Data Management, Data Matching, Data Quality, Duplicate Elimination, Fuzzy Matching | , , ,

By David Loshin

Data matching is easy when the values are exact, but there are different types of variation that complicate matters. Let’s start at the foundation: structural differences in the ways that two data sets represent the same concepts. For example, early application systems used data files that were relatively “wide,” capturing a lot of information in each record, but with a lot of duplication.
Read More

Moving to Action

Blog Administrator | Address Quality, Analyzing Data, Analyzing Data Quality, Data Cleansing, Data Quality, Data Quality Assessment | , , , , , , ,

By Elliot King

The first step in a data quality program is to assess your data. Whether you opt for data profiling or some other assessment mechanism, this part of the process consists of systematically identifying exactly where the problems can be found in your data sets.
Read More

Improving Identity Resolution and Matching via Structure, Standards, and Content

Blog Administrator | Analyzing Data, Analyzing Data Quality, Data Cleansing, Data Enhancement, Data Enrichment, Data Integration, Data Management, Data Profiling, Data Quality, Duplicate Elimination, Record Linkage | , , , ,

By David Loshin

One of the most frequently-performed activities associated with customer data is searching – given a customer’s name (and perhaps some other information), looking that customer’s records up in databases. And this leads to an enduring challenge for data quality management, which supports finding the right data through record matching, especially when you don’t have all the data values, or if the values are incorrect.
Read More

Designing More Effective Data Cleansing Rules

Blog Administrator | Analyzing Data Quality, Data Cleansing, Data Management, Data Quality | , , , ,

By David Loshin

In my last post, we looked at a simple data transformation and cleansing rule that was to be used to standardize a representation of a street type. We found that an uncontrolled application of the rule made changes where we didn’t really want a change to happen.
Read More

Data Cleansing and Simple Business Rules

Blog Administrator | Address Quality, Analyzing Data, Data Cleansing, Data Quality | , , ,

By David Loshin

Having worked as a data quality tool software developer, rules developer, and consultant, I am relatively familiar with some of the idiosyncrasies associated with building an effective business rules set for data standardization and particularly, data cleansing. At first blush, the process seems relatively straightforward: I have a data value in a character string that I believe to be incorrect and I want to use the automated transformative capability of a business rule to correct that incorrect string into a correct one.
Read More