Most MDM implementations will be based on a commercial software product. By relying on a packaged solution, the architectural considerations have been addressed by the software vendor, allowing the project team to focus on the data management considerations. However, there are cases when a custom MDM solution may be needed. When proceeding down the path of custom development for MDM, we have some advice for you: avoid these common mistakes.
The Case for Multi-Data-Domain MDM
The most common data domains addressed by MDM solutions include customer, financial, products, partners, employees and locations. Most organizations begin with a single data domain, typically focused on customer. Often additional data domains such as product or partners are created as separate MDM instances, managed by separate departments. As the customer data matures, it becomes apparent that an integration or federation between the customer and the product domain is necessary to get an accurate, consistent and complete picture of what each customer has purchased or considered. The same relationships may be needed between product and partner. The existence of multiple single data domain MDM solutions within an organization creates siloes of data that complicate the governance process and present challenges for data analytics and data quality. Many organizations realize they need a multi-data-domain approach to master data management.
Packaged Software vs. Custom Development
For these organizations, there are two options: extend a software product or custom develop a solution. Many software vendors offer specialized solutions focused on a single data domain. Extending a software product that was designed for a specific single data domain into a multi-data-domain model can be risky, complicated and expensive. This is a complex decision and every organization has to decide which option is best for their unique requirements. Many organizations build a custom solution to address multi-data-domain master data management requirements. At Sullexis, we want to share several common mistakes that you should avoid when building a custom MDM solution.
Mistake #1: Separate your Master Data from your Transactional Data
Separating master data from transactional data should be a simple process. But it often isn’t.
- Are Contracts master data?
- Is a Purchase Order transactional data?
- Are Cities, States and Countries master data?
Our guiding rule is that master data is data that does not change frequently and is at the core of the business. Transactional data is more dynamic, changing frequently in status and content; it is something that happens at a certain point in time.
Is Contract data dynamic? Well, it depends. Massive retail companies have a high volume of contracts that are constantly changing. Because of the number and volatility, it would be incredibly difficult to manage this data within a master data domain. We would consider these contracts transactional data, and not recommend inclusion in a master data domain.
Alternatively, for an organization such as a reinsurance company (the insurer of insurance companies), the contract (or policy) is a core component of the business and is fairly static, at least during the term of the contract. For reinsurers, these contracts are a good candidate for being treated as master data.
Purchase Orders are transactional in nature and not usually considered master data. However, in some organizations, particularly in the software industry, it is common to issue blanket POs to major vendors. These POs can span a long time period, be static in nature, be central to the operation of the business and represent significant financial commitments. In fact, they are similar to a contract. These organizations may need to consider these POs as master data.
Cities, States and Countries information almost never changes, is generally important for a business and should be centrally maintained. Should it be master data? In our experience, this data is most often treated as reference data rather than master data. Reference data is static data that is infrequently modified and is governed by standards bodies. For example, the list of countries is an ISO standard (3166-1). Data that is managed by external standards bodies typically does not need to go through the governance process of other master data.
However, Cities, States and Countries could be master data if the core information is enriched with dynamic data created within the organization such as alternate names, synonyms or custom classifications. Alternatively, if the organization operates in zones of the world with more dynamic changes due to geo-political conflicts, this data may need to be treated as master data.
What if I include transactional data in my MDM solution?
Including transactional data in the MDM realm can cause many problems.
- Processing transactional data will become a never-ending task, as every day new transactions will be created and this data will be added into the master domain.
- Data stewards will be over loaded by the amount of work in validating all this dynamic data via the governance process. In the worst case, they may abandon their responsibilities as stewards of the data, requesting custom development within the ETL process to automate data reviews. This has the effect of forcing the responsibility for the data onto IT rather than the business.
- The load on the server infrastructure will increase as the volume of master data increases with every new transaction added. Every new element must go through the standard ETL process that includes fuzzy logic, deduplication, matching, etc. These processes will need to run against an ever-growing amount of data.
- The number of duplicates within the master data may increase. Transactional data often includes real master data objects: customers, business units, vendors, and products. Adding redundant versions of master data to the system can increase the likelihood of failure of the deduplication algorithms, thereby introducing duplicates into your master data.
Evaluate your data. Don’t make assumptions.
While we have shared guidelines based on our experience and best practices, it is necessary for you to work within your organization to decide how best to classify your data. Avoid making assumptions based on what you have read or your experiences on prior projects. Discuss every single data element with the data stewards and business owners within your organization. Challenge each other’s point of view until everyone agrees which data is deemed worthy to be classified as master data.