The Staging Approach to Data Integration

by Olga Belokurskaya April 15, 2009

Enterprise data warehouse architecture (Image credit)

I’ve recently come across an article by David Linthicum at ebizQ. There he elaborates on a staging approach to data integration.

According to him, not many consider the use of a staging area when looking at data integration. However it’s a great solution when support is needed for more complex and valuable data integration operations, including support for many large data sets and data operations that are more complex and of higher value. Using a staging area helps perform complex operations on data, which are, normally, difficult to do using direct integration approaches.

David provides benefits of a staging approach to data integration:

  • The ability to perform more complex operations on data, including complete transformation of semantics and the data content using any number of dimensions since, in essence, you operate on an intermediary database that you control completely.
  • The ability to leverage more coarse grained and complex data sets that may not always repeat.
  • Informational focused, supporting valuable information externalization approaches, including business intelligence.
  • More flexibility around business cycles, data processing cycles, widely disbursed systems, and hardware and network limitations, where it may not be feasible to extract all operational databases at the same time.
  • The ability to better support complex database functions, including replication, cleansing, and aggregation.

Use it or not, it’s up to you. But it seems helpful, no doubt.