top of page
Writer's pictureAnna Davidson

The Great Debate: Internal vs. External Data Cleansing

Before we discuss the pros and cons of an internal vs. external project approach, let us first applaud you and your organization on recognizing the need for data cleansing. Regardless of which direction you go from here, you’ve taken the first step to improve data quality and achieve maintenance and procurement savings.


Now that you’ve made the decision to pursue a data cleansing initiative, you might be debating whether to perform the work internally or outsource it to an external service provider. As a strategic sourcing or procurement professional within a business, your responsibility is to identify and compare the various options and vendors, in order to ultimately select the best suited solution for your company. 


With data cleansing, there are many factors that must be taken into consideration when evaluating the options. For starters, let’s talk about cost, timeline, scope of work, technical requirements, resource requirements, methodology, and tools.


Factors to consider

The cost of your project will largely dictate what is possible and what method makes sense for you. Before you go into the decision process, consider what your budget is. Does it change based on other factors? For example, is a higher price worth the savings of fewer employee hours in the long run?


Your timeline also affects what you can reasonably get done, or the project cost. Do you have a timeline in mind for project completion? How would that timeline change depending on what resources you can allot? What timeline can an external company offer? 


A larger scope of work (ex. multiple warehouses vs a single warehouse) will extend your project. How does your timeline fit with the amount of work you have to complete? Do you plan to address everything at once or split it into stages?


Technical requirements may include the ERP you’re using or implementing. Consider, what specific needs do your ERP or other tools have? Will it affect how you need to clean your data? Do the external cleansing companies you’re considering have experience cleaning for your ERP or other systems you use?


The scope and timeline will largely determine what resources you need, specifically your staffing needs. A larger scope of work requires more resources and, depending on what you have available, it may extend your timeline. Consider what resources you can realistically provide, what the costs will be (ex. hiring extra staff if you are doing an internal cleansing project), and how these needs will change your ideal timeline.


Methodology dictates how you clean your data. Do you already have a methodology in mind? Is your current methodology working or do you need to replace it? What could an external company offer you?


Finally, what tools do you have at your disposal? If you’re considering an internal cleansing project, you’ll need to make sure your staff have appropriate tools to complete the work, whereas external cleansing companies will come with their own tools.

Teamwork, time management, shipping

So you’re thinking of doing an internal cleansing project…

From an initial perspective, internal data cleansing may appear as the most effective approach. In theory, your own resources should know your business and its data better than anyone, right? Not so fast. Your resources may be considered experts in their field, be it maintenance, procurement, supply chain, or asset management, but:

  • How much data cleansing experience do they possess and what tools do they have at their disposal? 

  • Will you pull them away from their regular role for the cleansing project, leaving their current workload to be spread across other employees? 

  • Will you hire new resources on a contract basis until the project is complete? 

  • Or will you create a new department, adding permanent resources and further increasing labour costs? 

  • Once you have the resource situation figured out, will you be forced to purchase an expensive data cleansing software package? 

  • Will your cleansing resources be able to get the job done within the desired timeframe? And how will they maintain the data moving forward? 


These are just a few of the questions you may want to ask yourself and your team before taking the plunge into an internal data cleansing project. From a cost and timeline perspective, internal cleansing isn’t actually as economical as you may think either. 


What does the project entail? 

Below are some average industry statistics that we’ve compiled to illustrate the cost and time requirement for a basic 25,000 SKU material master data cleansing project.


Example: 25,000 SKU Project

Average Time Spent Per Part (SKU): 10-15 Minutes

Average Hourly Burden Rate: $30-40

Hourly Cleansing Capacity: 6 SKUs

Daily Cleansing Capacity: 48 SKUs

Weekly Cleansing Capacity: 240 SKUs

Total Minutes Required: 500,000 Minutes

Total Hours Required: 8334 Hours

Man Weeks Required: 208 Weeks

Man Years Required 4 Years: (Or 4 people for 1 year)

Total Internal Cost: $250,020

Average Per SKU Cost: $10


Internal data cleansing

Based on industry statistics, project experience, and feedback from customers who have previously gone this route, internal data cleansing costs (on average) 50-65% more than outsourcing to an external service provider and can take up to 80% longer depending on the number of allocated resources, competence, and technology. Not to mention, the quality of cleansing significantly varies and often fails to meet expectations.


You’re probably wondering how this can be true as these numbers seem astronomical. You’re right, they do. However, you must remember that data cleansing service providers specialize in their field for a reason, just like doctors specialize in medicine and lawyers specialize in law. 


IMA and other service providers possess the experience, expertise, technology, and resources, which enable us to excel within the field of material master data cleansing. Not only do we utilize proprietary software that allows us to programmatically cleanse and maintain mass datasets within a short period of time, we also possess valuable subject matter expertise when it comes to implementing best practice data standards, formatting data to enterprise system configuration requirements, and performing critical quality control review tasks. In addition, most service providers have a growing internal library (catalogue) of MRO and OEM spare parts that they frequently reference to enhance and validate materials data.


Without the support of cleansing software, reference databases, and capable resources, the cleansing process can be incredibly laborious and time consuming. Each cleansing task must be performed manually, including consistently standardizing manufacturer names, part numbers, naming convention, and descriptions, while individually researching each item online or within manufacturer catalogues to validate and gather additional attribute information. And don’t forget duplicate identification, which is often a project of its own. Then, the data must be properly formatted to the enterprise system, which is a complex task that often requires IT support.


While it certainly is not impossible to perform a data cleansing project internally, it may be more costly and time-consuming than you had anticipated.


What can IMA do for you? 

IMA is the leader in Material Master data management; no one has more experience than us! If you’re ready to take on a data cleansing project, we’re here to talk through your options. Our services offer the most flexibility to meet your needs, whether that’s full or partial support. You can learn more about our services here or send us a message at info@imaltd.com. And don’t forget to take advantage of our no cost, no obligation Data Evaluation, to see actual cleansing samples and a detailed ROI projection!

Comments


bottom of page