The Sullexis Blog
Wrestling The Data Quality Bull: Using Informatica IDQ so Upstream Business Users Can Grab Data Quality by the Horns and Wrestle it to Submission
Learn how Sullexis & Informatica helped an Oil and Gas Super Major develop a business-led, self-service, and automated data quality management solution that enabled business processes to operate efficiently, reducing manpower and exception management, while laying a foundation for good data to power future automation and machine learning.
We’ve all seen the pictures of the Running of the Bulls event each year in Pamplona, Spain. While most of us look at that with a sense of shock, awe, and think “No way am I doing that!”, there is something to be said about the preverbal grabbing the bull by the horns. How many times have we been told to face our fears, tackle the problem head-on, and just do it? Well, the same can be said about data quality, but this particular bull we no longer need to fear.
With the increased adoption of big data platforms and cloud-based storage, many of our clients are recognizing how critical data quality is becoming to their day-to-day operations. They are seeing the need for a modern-day data architecture to help maintain and support good data quality across the various upstream functions. By following a straightforward data stewardship based framework, combined with Informatica Intelligent Data Quality (IDQ), you can align your people, processes, and technology, to quickly derive value from your data.
Self-Service Data Quality Framework & Solution
Sullexis, partnering with Informatica, has taken a data quality framework and put the control and ownership into the hands of the business end user. By applying this framework, the business users can address data quality issues in an efficient manner by setting up their own data quality rules and management without having to make requests to IT. IT, in turn, can focus on the infrastructure and technical capability enablement without having to devote resources to solution development.
Using this framework and approach, Sullexis & Informatica helped an Upstream Super Major develop a business-led, self-service, automated SAP PM data quality management solution for the client’s upstream Reliability and Maintenance organization utilizing Sullexis’ Data Quality Methodology and Informatica’s IDQ solution.
The solution focused on providing end users the ability to create and maintain a set of data quality rules which allow data assessment for non-conformity against maintenance data standards, prior to loading into the SAP PM production. In addition, these same rules could be applied to assess the quality of existing data within SAP PM. This data sustain solution was comprised of the following:
- Sullexis’ Enterprise Data Management methodology
- IDQ’s Self-service data rules management, assessment, profiling, and quality scorecard functionality for work management objects across all global regions.
- Informatica’s PowerCenter for data loading
- Informatica’s SAP PM direct connector
- In addition, client’s cloud-based Data Lake (Hadoop source) ingests IDQ results for inclusion for broader data analysis efforts.
Sullexis’ case study on this solution can be found in the Case Study section of Sullexis’ website here: Case Study #1: Informatica Data Quality Management Solution for Oil and Gas Super Major
IDQ Data Sustain Solution Results
With this type of solution deployed, your users can now focus on understanding their data quality performance by leveraging the IDQ profiling capability to assess data content, structure, and anomalies. This is an opportunity for business analysts, data stewards, and end-user developers to collaborate on data profiling. Once you establish your data profile, you can then monitor your data quality by building out a scorecard based on the profile set. By capturing and analyzing your data quality trends you accomplish the following:
- Completeness: What data is missing or unusable?
- Conformity: What data is stored in non-standard formats?
- Consistency: What data values give conflicting information?
- Duplicates: What data records or attributes are redundant?
- Integrity: What data is not referenced or otherwise compromised?
- Accuracy: What data is incorrect or out of date?
How Can Sullexis Help You
Sullexis and Informatica can partner with your company to duplicate the success of good data quality and give you a solid upstream data quality framework that will grow with your increasing data needs. This approach will provide a strategy to execute against, a remediation approach to monitor and track, and a set of business and technical processes to align users. Tame that wild bull called data quality and provide your people and your key operational systems with a set of data that is accessible, reliable, and trustworthy. Please contact us for more information or to set up a discussion.