Re: Rants. Difficulty to learn ETL tools? -
04-23-2007
, 04:22 AM
Hi DBA,
a couple of other comments....
1. Skills are not particularly transferrable between the leading ETL
tools like Informatica and DataStage....and worse, when people learn
one or the other they become 'religiously convinced' that the one they
learned is the best and tend to be worse than a newbie if they are
forced to transfer to the other.....just human nature...
2. The ETL tools are evolving rapidly and so there is significant need
to stay up to date.....just like any other piece of complex software.
3. ETL tools make the 'simple things simpler' and the 'harder things
just about impossible'. A few years back I was working on a project
where the ETL tool forced changes to the data model because the tool
could not load the data model as designed. If that is not the cart
before the horse I do now know what is.
4. By making the 'simple things simpler' it has been possible to
outsouce more development to lower skilled people. And a lot of that
has gone to India. There are some things workable and some unworkable
about this.....if you are outsourcing your ETL development then you
need at least one good ETL person on site as an employee so you know
you are getting 'good stuff' back.
5. The ETL tools are generic general purpose tools that are then in
some way 'coded'. They exhibit the exact same properties of other
source code environments...it makes no difference that the 'source
code' is presented as squares joined by lines in a GUI...it is still
source code of a fashion...it still needs to be tested every time
something changes....
And just as an amusing aside....I recently did a project where we used
our ETL tools to build the prototype and then recoded everything into
Informatica. We hired one of the leading Indian IT companies to do the
conversion of our prototype ETL to Informatica.....their lead person
looked at what we were giving him and the first question out of his
mouth was 'Why do you need us? You already have a working ETL
subsystem?' to which the answer was 'Well, the corporate standard is
Informatica, and your job is to convert this into Informatica. And you
get paid for doing so.'....and they were happy campers.... ;-)
Best Regards
Peter
www.peternolan.com |