Data Quality Assessment: Value and Pattern Frequency

Blog Administrator | Analyzing Data, Analyzing Data Quality, Data Profiling, Data Quality, Data Quality Assessment | , , , , , ,

By David Loshin

Once we have started our data quality assessment process by performing column value analysis, we can reach out beyond the scope of the types of null value analysis we discussed in the previous blog post. Since our column analysis effectively tallies the number of each value that appears in the column, we can use this frequency distribution of values to identify additional potential data flaws by considering a number of different aspects of value frequency (as well as lexicographic ordering), including:

  • Range Analysis, which looks at the values, and allows the analyst to consider whether they can be ordered so as to determine whether the values are constrained within a well-defined range.
… Read More

Data Quality Assessment: Column Value Analysis

Blog Administrator | Analyzing Data, Analyzing Data Quality, Data Cleansing, Data Enrichment, Data Profiling, Data Quality, Data Quality Assessment | , , , , ,

By David Loshin

In recent blog series, I have shared some thoughts about methods used for data quality and data correction/cleansing. This month, I’d like to share some thoughts about data quality assessment, and the techniques that analysts use to review potential anomalies that present themselves.

The place to start, though is not with the assessment task per se, but the context in which the data quality analyst will find him/herself when asked to identify potential data quality flaws.… Read More