More About Data Quality Assessment

By David Loshin In our last series of blog entries, I shared some thoughts about data quality assessment and the use of data profiling techniques for analyzing how column value distribution and population corresponded to expectations for data quality. Reviewing the frequency distribution allowed an analyst to draw conclusions about column value completeness, the validity of data values, and compliance…

Continue Reading

Get Used to It: Inconsistent Data is the New Normal

By Elliot King Nobody is perfect and neither is corporate data. Indeed, data errors are intrinsic to IT's DNA. Data inevitably decays. Errors can be caused when data from outside sources are merged into a system. And then, of course, the humans that interact with the system are, well, human. Unfortunately, despite the best efforts of data quality professionals, the…

Continue Reading

A Revolutionary Approach to Full Contact Data Quality

To stay competitive, cut costs, and drive growth, companies are always looking for ways to attain the most relevant, most complete and up-to-date customer information available. Now there's technology that moves beyond the mere validation of contact data to bring your data to a higher level of accuracy - a term we call, "data quality uplift." Melissa Data's new Personator…

Continue Reading

Where Do You Fit In?

By Elliot King Too often, those of us with our noses to the grindstone have no time to look up. We are so busy putting out fires, monitoring and maintaining what we have, or trying to launch new initiatives that we never look around to see how other organizations are dealing with similar issues. This may be particularly true in…

Continue Reading

Using Data Quality Tools for Classification

By David Loshin Hierarchical classification schemes are great for scanning through unstructured text for identifying critical pieces of information that can be mapped to an organized analytical profile. To enable this scanning capability, you will need two pieces of technology. The first involves a text analysis methodology for scanning text and determining which character strings and phrases are meaningful and…

Continue Reading

Understanding Hierarchies

By David Loshin Defining standards for group classification helps in reducing confusion due to inconsistencies across generated reports and analyses. In the automobile classification example we have been using for the past few posts, we might pick the NHTSA values (mini passenger cars, light passenger cars, compact passenger cars, medium passenger cars, heavy passenger cars, sport utility vehicles, pickup trucks,…

Continue Reading

Standardizing Classifications

By David Loshin In the most recent post, we posed a straightforward problem: if we have a reporting or analytical objective that depends on using a dimension for classification, what happens when two different value domains are presumed to map to the same conceptual domain? More concretely, the example we used was mapping individuals to their car purchase preferences, but…

Continue Reading

Managing Customer Connectivity

By David Loshin At the end of our last entry, we had come to the conclusion that standardization of potentially variant data values was a key activator for evaluating record similarity when looking to group customer records together based on any set of characteristic attributes. From an operational standpoint, this activity is supported using data quality tools that can parse…

Continue Reading

What Can Health Care Teach Us About Data Quality?

By Elliot King Data quality issues are more acute in health care than in perhaps any other industry sector. According to a seminal study by the Institute of Medicine, (IOM) preventable medical errors are responsible for nearly 100,000 deaths annually, making it the sixth leading cause of death in the United States. These errors cost $98 billion annually. Of course,…

Continue Reading

Customer Centricity and Connections: Establishing the Link

By David Loshin In my last post, we began to look at the value proposition for grouping individual customers into logical groupings. We began by looking at a grouping that generally appears naturally, namely the traditional residential household. We talked about householding in a previous blog posting, but it is worth reviewing the basic approaches used for determining that a…

Continue Reading