Re: ETL Best Practice

#1

Thanks a lot Luts - som really good arguments.

Are text files really slower than odbc ? With the text files you can make
the bulk insert ... can you do a similar action with odbc ...

Of course with text files the host system has to save it to the network
which also takes some time.

However - think you're right about the XML ...

\Michael V.

"Lutz morrien" <lutz.morrien (AT) ocb (DOT) com> wrote

Quote:

Hi Michael,
i can only give you my opinion on whether to use text
files or odbc for the ETL process.

Both ways are a valid approach. if you use odbc, I think
you are trading flexibility for speed. Transferring data
from server A to server B directly will usually be faster.

On the other hand, if you use text files (with any
delimiter or xml), you have less speed. But you have a
backup data file in case something goes wrong ... and you
have a format any customer will be able to provide. It
might save you lots of time and trouble, if you tell the
customer that he has to provide a text file in a precise
format... instead of implementing a database interface.
If your client changes his database or application,
chances are good, he will still be able to produce a
text/xml file. But he might not have an odbc source any
more.

In future I would go with xml.

HTH
Lutz
So I personally

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Re: ETL Best Practice

microsoft.public.sqlserver.olap microsoft.public.sqlserver.olap

Re: ETL Best Practice - 06-29-2003 , 05:32 AM