The Challenge
Sullexis helped an Oilfield Services company develop a leading edge Enterprise Data Lake and Analytics to provide Executive Dashboards and Operational Analytics by utilizing Sullexis’ Big Data Analytics framework and the Azure Data Lake Platform.
- This leader in oilfield pressure pumping has experienced aggressive growth across all major basins in North America.
- The client has gone through multiple enterprise system transformation projects and established a core set of cloud-based applications to run the business.
- Executive management is pursuing a knowledge management and metrics driven business strategy to perfect operational efficiency, reliability, and safety in oilfield services.
- To enable this metrics driven strategy, the client needs interactive KPI dashboards and ad-hoc analytics capabilities based on aggregated data from key systems such as ERP, Opportunity-to-Cash Lifecycle, Work Management, EHS, HCM, and Operational equipment.
- Initial attempts at creating these KPI dashboards and ad-hoc analytics capabilities had limited success due to the lack of data integration and the strain placed on the live transactional systems, which impacted end users in the business functions.
The Solution
The client partnered with Sullexis to bring its relevant domain expertise; ability to quickly lay-down the technology foundation needed for enterprise analytics; know-how to create a fully functional set of Operations KPIs; and experience defining a roadmap to implement advanced analytic needs.
The initial project included custom ingestion routines for aggregating data from the cloud-hosted ERP, EHS, Work Management and SharePoint collaboration systems; an integrated data model for the Data Lake including materialized views; and utilizing the Microsoft Azure technology stack for Data Lake storage and Analytics Dashboards, including Azure Event Hubs for streaming IoT data from various sensors.
Key components of the solution include:
- Sullexis Enterprise Data Engineering methodology, Enterprise Data Lake architecture and Big Data principles
- Azure Data Lake Storage (ADLS Gen2) with BLOB Containers
- Custom data ingestion pipelines using Apache NiFi, Azure SQL Server, OData, OAuth and SharePoint REST APIs for extracting, loading and staging data into custom Delta Lake storage
- Microsoft Event Hubs for streaming near real-time sensor data from various operational equipment, to enable Data Science team apply AI/ML analyses for trends and better forecasts
- Microsoft Azure PowerBI for KPI dashboards and analytics visualization for various senior management, operations and external customer audiences
The Result
The solution delivered cross-system analytics and dashboard reporting to support the organization’s increased use of KPIs, scorecards and related metrics that are driving departmental management decision making as well as shaping executive business strategy. This was achieved by harnessing the power of cloud Data Lake with aggregation results in materialized views for specific metrics, measures, and historical trend analytics providing mobility-enabled lean and fast-loading reports with cloud-based, rich and interactive KPI dashboards.
