I almost hate to ask this in such an open forum, but I'm looking for
the right OLAP solution to replace one that we already have.

Our application has Planning functions where users work on subsets of
a cube in a workbook and later commit their changes to the original
cube. It also has Forecasting and Allocation functions that generate
dense forecasts based on sparse data.

It isn't purely a 'read-only' OLAP (whose source might be a relational
database), it is also updatable.

The factual data typically ends up being on the order of 1 TB (not
counting user workbooks, derived values, or "materialized"
calculations).

The solution needs to be fast (user workbooks are on the order of 50
MB and typically have changes to 50% of the data which needs to be
committed) and able to handle concurrent updates from potentially
thousands of users (though more typically hundreds); updates are on
data that is unique per user (in other words, one user doesn't update
the same data as another).

It also needs to be able to handle batch-like operations quickly where
substantial parts of the multidimensional space change based on
forecasts.

And I haven't yet mentioned that the measures are not static. The
time dimension always changes (obviously) but other measures (like the
number of locations) also very with time.

So, with all of these requirements is there any *one* OLAP system that
can meet our needs? I've had a look at a variety of options and
Oracle OLAP (under 9.2) appears to be the best bet...but the
traditions of that product seem to be in data mining and not
"interactive, updateable" applications. It seemingly has that
capability, I'm just how sure it performs well.

There seems to be a difference between the planning space and the
forecasting space. Engines are geared for one or the other but not
both...but this is based only on research that I've done on the
web...so I could be missing a gem!

Please contribute your ideas to what I'm sure will be a lively thread!

Thanks,
:-Phil

Phil

Have a look at www.o2olap.com. O2OLAP for Excel works with Microsoft's
Analysis Services. Signup for one of our online real-time demonstrations.

Planning, forecasting, allocations, batch processes and "interactive,
updateable" with optimization.

Regards
Howard


"Phil" <google (AT) philgra (DOT) org> wrote

Phil

We do not have customers with this sort of volume of data but I will try to
find out from MS and some other OLAP vendors if they have referencable
clients with the volumes concerned.

Howard
www.o2olap.com


"Phil" <google (AT) philgra (DOT) org> wrote

As a matter of interest, what solution are you using currently? It sounds a
bit like OFA.

I'm not sure that it could cope with the data volumes, but Cognos Planning
(the former Adaytum) probably has the right set of functionality for you.
It's always worth checking the extent to which your large data volumes are a
consequence of the technology, rather than representing the real business
need. For example, I find it hard to believe that end user are typing in or
changing 25Mb of data in their workbooks.

Incidentally, Oracle 9i OLAP doesn't have anything to do with data mining.
It's the modern version of Express Server that's (more) integrated with the
RDBMS. Oracle's data mining option is based on another product that Oracle
acquired (Darwin).

Nigel Pendse
OLAP Solutions
http://www.olapreport.com


"Phil" <google (AT) philgra (DOT) org> wrote

Hi Phil,

as the other posters, I also work for a company offering these kind of
SW/services, not our policy to advertise here so I won't, but there's
something in your post that made me curious, I'll be glad if you take
the time to clarify.

Now, note that I've been specifically into planning/budgeting apps for
three years, and relating to

Almost every customer keeps asking things like this, when you get
deeper into the analysis you discover it's more usually like 10 to 50
users entering/updating data. I got the impression that this is simply
because of organizational issues: controlling a process in which
hundreds/thousands users have their voice in the budgeting cycle is
simply unmanageable (if only because the probability that someone
completely screws up things grows dangerously close to 1), unless
there's a way to decouple users some way. I may be wrong, obviously.

Does this match your situation, are you simply keeping the number of
users large to allow for future scalability or do you already have
such an extended process in place? What kind of industry? Retail? Are
we talking about Sales/Cost/Capital budgeting or something else?

Changes to 50% of the data per user means a 25 MB change, roughly 3
millions figures are changed (assuming an eight byte float), not an
easy manual task for a human being I would say. Is there anything
automated in the process of generating changed data, can you please
explain?

There are vendors (sufficiently) strong in both fields and able to
deal with significant amounts of data, I think. Given the complexity
you're exposing I would say an out of the box solution won't solve
your problem and you'll need significant personalization, I would look
for a vendor with smart, experienced consultants and try to talk with
the people that actually deliver the results ASAP (read: as little
salesman BS as possible) keep searching

Just my point of view...

Regards,
H.

"Phil" <google (AT) philgra (DOT) org> wrote

Phil,

The 64-bit version of Microsoft Analysis Services can operate with very
large apps on large servers, though you won't find many production sites
using it as it only shipped very recently. A single app can span multiple
servers, too. It would certainly handle your data volumes, probably better
than any other MOLAP (smart software doesn't always need big iron).
Ironically, one case study I've seen for large apps on this version is IRI,
the former owners of Express, who continued to use it long after selling it
to Oracle.

But Analysis Services isn't a very good solution for large planning apps as
it doesn't have efficient data write-back in the current version. It also
does too much on-the-fly calculations, which hurts query performance. And it
doesn't have good top-down planning functionality. So beware of people
telling you that it's the best product for every application -- no product
is.

Nigel Pendse
OLAP Solutions
http://www.olapreport.com

Phil

I have been in touch with the right people within Microsoft and have been
sent some information, together with a reference case that involves over 100
TB of data. As the content involves various document types can you contact
me direct. I think this is the information you are looking for.

With respects to planning, forecasting and allocation functionality within
your application, we can demo this to you over the internet and can explain
how to work within the Microsoft environment. Understanding how the
Microsoft environment works, together with tested techniques and
functionality, will help you to understand how you can work within this
environment. We can help you put a prototype together and would also bring
Microsoft into this.

Regards

Howard
Howard.Taylor (AT) o2olap (DOT) com
www.o2olap.com


"Howard Taylor [O2OLAP]" <Howard.Taylor (AT) o2olap (DOT) com> wrote