o be successful with a data migration project, whether migrating data from one system to another or even a version upgrade, you’ll need to adhere to best practices that are well documented for such a project, e.g. source system exploration, data assessment and profiling, migration design and build, etc. (This SAS white paper provides a good data migration overview: Enhancing Your Chance for Successful Data Migration.).
Typically, data migration is not just dependent on Extract, Transform and Load (ETL, but also on data cleansing which can be challenging in itself. And recently, I’ve been involved in a data migration where data didn’t just need to be cleansed prior to transformation and load, but new data needed to be created – a lot of new data. In this particular migration, new data had to be created to support a number of requirements:
- Government Reporting – new value lists were created to highlight and classify activities that needed to be reported to regulatory agencies
- Improve Analytics – new master data elements were added to objects to help classify operational usage and support new KPIs
- Enable New Process – entirely new data objects were deployed that had to be populated and cross referenced to existing data objects
- Improve Operational Efficiency – new data correlations were required between existing data objects to support process automation
- Enhance Cost Reporting – elimination of manual financial coding with the replacement of new account code lists
The big challenge with the creation of new data is that this activity does not fall within the expertise of the data migration team. It needs the end users to be actively involved in the creation and validation of these new data elements – people who already have day jobs. If you are managing a data migration project you are going to see that this is a huge red flag in your ability to deliver on time, as this is an activity set outside of your control but your success is dependent on this deliverable.
Identifying this new data requirement early and establishing ownership of the task is key to ultimate success – work with the Project Management Team to have this activity called out and managed separately to your data migration effort. Then, work with the end users on their engagement and ownership of the task, and sell them on the benefit that this activity will give them, i.e. early exposure to the system and its requirements, and an understanding of the increased data management responsibilities that comes with the new system (might as well get prepared for it sooner rather than later).
During this particular engagement, we didn’t recognize the need to create all the data elements in the initial deployment so go-live was particularly painful. Fortunately, the larger migration effort was still several months away so we quickly got in front of the users to share with them the opportunity and the need to eliminate the potential pain of not having enough data to operate the system effectively. It proved to be a very successful migration and the productivity gains of the new solution, driven by the new data, was evident within a matter of weeks.