Data quality: the unglamorous step that makes analytics work

Every analytics project eventually discovers that the data is messier than expected. Fixing quality at the source is cheaper than cleaning the same problems in every report.

Common problems

• The same customer, patient or product recorded under several spellings.
• Free-text fields used where a fixed list was needed.
• Dates and amounts in inconsistent formats.
• Required information left blank because the system allowed it.

Fix it where data is created

Use drop-down lists for categories, validate formats on entry, and check for likely duplicates before saving a new record. Small changes to forms prevent large cleaning jobs later.

Define the key terms

Agree what "active customer", "completed order" or "admission" means, write it down, and use the same definition everywhere. Many conflicting reports are really conflicting definitions.

Measure quality

Track a few indicators — percentage of records with missing fields, number of duplicates found — and review them regularly. What gets measured improves.

Make someone responsible

Assign an owner for each key dataset who decides on definitions and approves changes.

See how we help with data engineering and analytics.