Businesses find themselves in an increasingly competitive market. Not only are they battling with traditional businesses they are now faced with the digital revolution. New start-ups are taking market share from those that fail to adapt and adopt new technologies. Customer buying habits are changing. Faced with such competition imagine marketing campaigns targeting the wrong people, sales initiatives focused on the wrong opportunities, finance billing the wrong amount. It’s the last thing a business needs and at the root of all these erroneous activities is poor quality data.
To stay competitive businesses need an accurate up to date view of their customers whether it’s in a CRM system, or other application for a very simple reason; to enable them to offer the highest service level for the lowest cost. Bad data can help spread a negative message very quickly, customers go to Twitter to air their grievances and staff have to check & recheck information to ensure its correct. They may need to understand exactly who they are dealing with by using information stored across multiple systems in a variety of formats.
Contact data is subject to constant decay, for example people change jobs and businesses merge. The rate of decay in areas such as job title can be as high as 50% change a year. Any CRM database without a customer data renewal strategy will become near useless in a few short years.
All of this can cost your organisation money, and perhaps more importantly failure to manage the relationship with your customers properly. Poor data quality is not just costly from a monetary point of view it can cause frustration amongst employees as they battle with poor marketing results, increased return of goods due to ‘Gone Always’ and wasted time adding post codes etc. to existing data sets.
It would be a lot simpler if organisations had just a single database containing information on their membership, their customers or their supporters, but in most instances that is not the reality. Organisations tend to hold data in multiple systems, in multiple back office applications, or just on plain old spreadsheets. One of the biggest tasks facing businesses especially if they are implementing new technologies like BI tools or CRM systems is how to clean and de-duplicate this information prior to it being loaded into the new application, and once there how to maintain its quality.
Stage One – Data Discovery
Just how bad is the state of your Data? Is it a problem worth worrying about? The first stage is to understand the size and nature of the problem and its impact on your organisation and future plans.
Is it something that can be ignored at the moment and fixed over a longer, or does it need urgent attention? What effect is it having on aspects of your business, customers, suppliers and staff? Where are the data issues found? What type of data issues do you have?
CertifyForce use leading software and domain experience to run your data through a comprehensive investigation process and provide a detailed report quantifying the problems you face e.g. 15% of your contacts are duplicate, 10% of your addresses are missing postcodes, 10% of your contact details are different between IT systems. This analysis provides the first step in helping you to formulate a plan to rectify your Data Quality issues in priority order.
The approach first identifies the data stored, from a high-level down to the “Data Dictionary”. Once the data landscape is understood a battery of data quality checks are run to establish the magnitude and location of issues.
Stage Two – Action planning
With a clear understanding of the issues identified in Stage 1 both short & long term actions plans can be formed and agreed. There are three main ways of addressing issues.
Firstly prevention and stopping the issue outright from occurring again. This is generally achieved by changing the business rules and other fixes to IT systems and data loading between systems. CertifyForce can provide guidance on the business requirements needed to prevent data issues but rely on IT teams to deliver enhancements of existing systems.
Secondly in changing operational procedures to prevent data issues. People through their use of IT systems can generate issues. A great example of this being duplicate accounts & contacts. We provide guidance and design of operational procedures and identification of simple improvements that can be made e.g. reduction in number of options in a pick list.
Thirdly by fixing the data issues as and when they appear. Data is always decaying and having a continual data refresh approach should be part of any data quality strategy.
This multi-pronged approach looking at immediate remedy to material issues and long-term preventative measures forms a solid basis for improvement work. These proposed plans need approval, resources and commitment to have an effect and deliver benefits. CertifyForce can assist in the development of the business case to demonstrate the cost of poor quality and areas of opportunity.
Stage Three – Data Cleaning
There are a number of approaches to data cleaning and the specific cleaning instigated for a business depends on the work conducted in stages 1 & 2 which understands and prioritises the issues to address. This section covers primary cleaning activities usually undertaken but is not meant to be comprehensive.
- Address Cleaning – Address data is matched against the Royal Mail Postcode Address File to find unknown addresses and validate correct address.
- Deduplication – The process of contact & account deduplication involves identification of potential duplicates, assessment to determine if they duplicate, selection of primary record to retain and merging of data together to remove the duplicate.
- Contact verification – There are times when only by asking the customer can the right answer be found. This is achieved initially through an automated process of email contact to request they update their details online. Where this is not effective an outbound telephone campaign can be deployed.
- System reconciliation – Where information is passed between systems an ongoing reconciliation and process to identify the missing or corrupted records for correction is put in place.
Through any data cleaning process two aspects must always be considered:
- Data security and ensuring critical customer data is kept safe. Clearly defined security procedures must be drawn-up and agreed.
- Audit trail should data need to be recovered and as proof of the cleaning work conducted
Stage Four – Ongoing Maintenance
Once the initial cleaning and de-duplication process has been completed and the revitalised data is in a single database it is time to maintain those gains and not let things decay. This is often a collaborative process with other departments working to prevent new data corruptions and continually checking and refreshing data. These and other elements come together to form the basis of an ongoing data quality strategy.
On an ongoing basis simple data quality tasks should be run, these include:
- Change of address register – run by the Royal Mail for people and businesses this provides the most up-to-date address available. In addition new telephone numbers and email addresses can be provided
- Annual customer verification –reaching out and asking customers to verify and correct their details online is a great way to keep data clean.
- Data enrichment – clean data can be enriched with marketing information such as lifestyle, behavioural, and demographic information. This allows for highly targeted sales & marketing efforts.