Hi all,

I'm a Bachelor in Computer Engineering, and going to study Masters
(major in Knowledge-Based systems). I'm quite fascinated by the concept
of data-mining and knowledge-based systems, and so I'd like to pursue
my career in this field. However, I'm not too sure about the
opportunities available in the field. Apart from research, what else is
(commonly) available? I'd be most interested in developing
knowledge-based software (e.g. using neural networks), but I'd still be
very interested in any of the computational side of things in this
field.

Another question I have is, because my Masters degree will be
coursework-based (plus a minor thesis -- using neural net to compose
music), I'm still thinking about what courses to take. Should I take
computer-based courses only (e.g. machine learning, data mining, data
warehousing, DB development, e-commerce (?) )? Or, should I take some
courses on statistics also (e.g. statistical inference)? Any other
advices will be much appreciated.


Cheers,
Michael

Data mining lies in an area of overlap among A.I. (especially the
machine learning and pattern recognition end of A.I.), statistics (and
related applied math fields such as O.R.) and computer science. I
would suggest studying inferential statistics. If you're interested in
learning more about career opportunities in data mining, see the job
listing on KDnuggets:

http://www.kdnuggets.com/jobs/index.html

I'm not sure why you asked about this in a database group- data miners
are more likely to congregate in newsgroups such as comp.ai.neural-nets
or sci.stat.math.

-Will Dwinnell
http://will.dwinnell.com

Yep, but on the other hand the first step in data mining is typically
collection, integration, and cleansing. Additionally, most
organizations need more basic analytics first - hyperlinked olap
reporting, etc: since most organizations have problems that they know
about (and just need more info on) - that's where the low-hanging fruit
is. Finding problems that they don't know about yet is great, but
should wait until you've got the known problems fixed and the
foundation set.

Further, in my experience the value of additional depth of analytics is
roughly equivilent to additional breadth of data. And the breadth of
data can usually be solved more reliably and cost effectively via data
warehousing than the depth can be via data mining.

So, if you know data warehousing and BI you're in a great position to
deliver 80% of the analytics most organizations need today - plus
deliver the foundational components also needed by most data mining
activities.

Unfortunately, most organizations won't pay for that last 20%, but I'd
say that in the meanwhile, BI is more fun than unemployement.

Predictor wrote:
"I'm not sure why you asked about this in a database group- data miners
are more likely to congregate in newsgroups such as comp.ai.neural-nets
or sci.stat.math."

bucknuggets (AT) yahoo (DOT) com responded:
"Yep, but on the other hand the first step in data mining is typically
collection, integration, and cleansing."


The amount of these tasks performed by the data miner varies:
Depending on the circumstances, it may be neccessary to understand a
relational database and formulate an appropriate query, or it may
sufficient to recieve a prepared flat file. Regardless, the data miner
is always responsible for the statistics. If one wants to become a
data miner, I think it makes more sense to study to become a
statistician than a DBA.


-Will Dwinnell
http://will.dwinnell.com

Agreed. I'm not recommending that someone interested in data mining
pick up database administration.

But a statistician that can consolidate & cleanse data, possibly
provide some contextual reporting, possibly provide a scoring solution
- will be in a better position than one that can't.

working with a set of statisticians right now limited by data
logistics...

buck

bucknuggets (AT) yahoo (DOT) com wrote:

I think a DBA with good data mining skills would be an extremely
employable person in the next few years (maybe 5). It's my contention
that there is a huge data explosion on the way, and somebody that could
practically seperate the wheat from the chaff will be very, very valued.

Mark Townsend wrote:
"I think a DBA with good data mining skills would be an extremely
employable person in the next few years (maybe 5)."

Perhaps, but I would think that would be due to opportunities which
required one or the other of these skills, not often both.


Mark Townsend wrote:
"It's my contention that there is a huge data explosion on the way, and
somebody that could practically seperate the wheat from the chaff will
be very, very valued."

I'd say the data explosion has already been under way for at least 10
years, but the point is that most of a DBA's skills would be wasted as
a data miner. This will vary by the data mining task, naturally, but I
would think that a database report writer would have more than
sufficient skill required in the overwhelming majority of data mining
projects. I've worked on a number of projects in which I was simply
handed a flat file.

Starting from scratch, becoming a capable DBA or data miner takes time.
There are only 24 hours in a day. My recommendation is to concentrate
on the math and statistics.
-Will Dwinnell
http://will.dwinnell.com