Data Extraction Methods (data warehousing) -
08-18-2003
, 04:40 PM
This is probably and repeated post, I apologize for that if true.
Hello,
The sys admin of the AS/400 (our online system) suggested to prepare
file(s) that contains all the data required by the data warehouse, the
file(s) is then downloaded nightly instead of downloading from
individual tables like the customer, product tables etc..
I personally feel reluctant to go with the prepared file(s) approach,
I think it is redundant plus it adds one extra step that needs to
happen before the downloiad and also makes me so dependent on people
whom aren't in the same office. I also didn't see this approch in any
of the text books about data warehousing which makes me more reluctant
to go with it.
Your comments and suggestions on this matter are very important.
Another question:
Are there best practices for the Data Extracton process, is it simply
initial download of the required tables/fields to the staging are from
the source system, then the rest is nighltly downloads to trace
changes
and do the transformation and loading to the actual DW ?
Thanks, |