Data Migrators

Articles

Accelerated Design

We've designed world class Data Migration solutions for some of Australia's largest Utilities clients. Performance, Auditing, Reporting, Maintainability, and Efficiency are baked into our solutions, so you just need to focus on defining functional requirements.

You want a data migration partner who's done this before.  One who has spent years addressing common data migration challenges for organisations like yours.  We've solved many challenges along the way, and have encapsulated this knowledge into a set of processes and assets from which our clients benefit on day one.

Let's take a look at the features of a typical migration solution which populates Oracle's CC&B application.  Here's an overview, presented using beautiful open source icons ....

Traditional CC&B Migration Architecture

A quick summary:

  • Source represents the source data spread across multiple source systems.
  • The Extract and Transform processes read and process this source data, before storing their results in a temporary staging database.
  • The Load process populates CC&B's 'STGADM' schema, which is an identical representation of CC&B's operational schema used as an input to 'Validation'.
  • Oracle's Load solution for CC&B acts as a data quality firewall by propagating data from STGADM to CISADM (the operational data store), validating the staged data against a set of built-in business rules as it goes.
  • Once completely populate, the CC&B application servicing the CISADM schema can be started.

Note that this classic arrangement has a number of opportunities for improvement.  Here's a couple ...

  • The ETL stages to a relational database.  Relational tables don't allow us to retain the ordering of partitioning of data so this information is lost, and downstream jobs must waste resources re-sorting and re-partitioning data.  Using a datastore which retains sorting and partitioning information would remove this redundancy.
  • The performance of Oracle's load solution for CC&B is questionable at best.  A prime candidate for reducing end-to-end execution windows is to remove the artificial delay imposed by waiting for the validation to complete before you can begin your post-migration application shake-out on the CISADM schema. By populating STGADM and CISADM in parallel you can perform your validation and shake-out simultaneously, saving enormous amounts of precious cutover time.

An improved CC&B Migration Design

Quick Contact