Data cleansing can provide many short and long-term benefits when implemented properly, but it is no surprise that we often run into companies who have experienced a failed data cleansing project and no longer believe in the benefits. It’s unfortunate to hear about companies who have invested thousands of dollars in data cleansing projects, only to end up going back and correcting the data afterwards. When meeting with project teams in these situations, we typically discover a few common reasons why their previous cleansing projects failed.
Today let’s talk about why some projects fail, how to avoid failure, and what to do if it does happen.
What is the point of a data cleansing project?
We’ve talked before about How Material Master Data Drives Efficiency, and that’s what data cleansing projects really boil down to. If you have good, actionable data, you can improve the efficiency of your company in a number of ways–from better procurement strategies to increased optimization to quicker maintenance and better searchability. The way to get that actionable data is going through a data cleansing initiative.
What is involved?
When you start a Material Master data cleansing project, the objective is generally to clean all of your existing data to a given schema. (What that schema is depends on your company’s needs and the service you use for data cleansing.) Typically, this involves first capturing all of the data in the same source and then going through the cleansing process to standardize all of it. Afterwards, there is a need to continue standardizing data to make sure all future entries are still being entered or cleaned to the schema decided on.
Why do projects fail?
One of the most common reasons for project failure is that the previous service provider simply used automated software to rapidly extract and classify thousands of existing items without human review. While the speed and efficiency of this method may be impressive, the end result is not.
In these cases, data is returned to the customer with incorrectly classified items, inconsistent descriptions, and often, inadequate information. Although the quality of these automated software applications has come a very long way and is continuously improving, the truth is there is no software application that can reliably transform large files of unstructured data into accurately standardised, enhanced, and structured descriptions without human intervention.
Another common reason why data cleansing projects fail is due to a lack of flexibility to accommodate customer requirements and an unclearly defined Standard Operating Procedure. Many data cleansing companies are very rigid and will only cleanse and format data to their own standards.This can become a significant issue as every company is unique and has different business requirements when it comes to format, standards, abbreviations, and project timeline. If data is not standardised and structured according to customer requirements, it not only defeats the purpose of implementing a data cleansing project, but also requires a significant amount of time and effort for the customer IT department to re-work and prepare the data before uploading.
Project timeline is also critical as data cleansing is often part of a larger ERP implementation. If the data cleansing deliverable is not completed on time and within scope, the entire project will be delayed, costing the company valuable time and money.
The final common reason why data cleansing projects fail is due to the absence of a long-term strategy to maintain ongoing data quality as items are added, modified, and suspended within the catalogue. If a catalogue management process is not implemented after the cleansing project is complete, the data will quickly revert to its previously corrupt state. Once again, this common mistake defeats the purpose of investing thousands of dollars into a data cleansing project.
How do you avoid failed projects?
Although every data cleansing company uses software to a certain extent, the best results can only be achieved through a combination of software and human intervention. When considering a data cleansing project, it is well worth the time and effort to research various service providers to understand their cleansing methodology and ability to meet company specific requirements. After all, data is the foundation for business decisions and if the foundation isn’t constructed properly, the entire investment will come crumbling down.
So before you start a project, consider the following:
Do you have a schema that already works for you? (We talk more about schemas in our last blog post.)
Can the data management company adapt to your schema?
What schema does the data management company use or suggest?
How does the data management company clean data? Is it automated or are humans involved?
Can they give you a demo with your data? Does it meet your expectations?
What is the long term plan for data governance after the cleansing project ends?
Does the data management company offer support during and after the project?
How do you fix a failed project?
If you are one of the unlucky companies who have invested thousands of dollars into a failed data cleansing project, don’t feel bad–you’re not alone. You also shouldn’t give up. Just because your first attempt didn’t succeed as you hoped doesn’t mean there is no value in data cleansing.
On the road to fixing your failed data cleansing, we recommend starting at the beginning:
Reassess what you want out of the project. What are your priorities? What is your timeline?
Do a post-mortem on the project. Where did things go wrong? What about the results is the problem?
Research new data cleansing options and compile a list of questions and expectations for the new companies. Make sure you assemble a team internally, as well, and get everyone on the same page!
As much as a failed project is troublesome, you can use the experience to help you get better results in the future. Every failure teaches us something!
Why work with IMA
As a results-oriented company, IMA Ltd. is dedicated to providing the most accurate, consistent, and reliable data available in the industry. We continuously develop and improve our solutions based on the changing market and feedback of customers in order to most effectively meet your needs.
At IMA, we don’t believe “one size fits all.” We work hard to be the most customizable solution in the industry. We’re committed to meeting your needs, including adapting to use your schema as needed and providing the right amount of governance support for you.
Although many competitors have chosen to sell software and services based solely on speed, IMA believes that quality remains the most important factor when dealing with critical inventory data. The IMA theory suggests that a balanced combination of technology and human intervention is required to achieve the highest level of data quality.
For more information on our services, check out our services page or contact us at info@imaltd.com. We look forward to hearing from you!
Comments