dbTalk Databases Forums  

To Process or not to Process, that is the Question..

microsoft.public.sqlserver.olap microsoft.public.sqlserver.olap


Discuss To Process or not to Process, that is the Question.. in the microsoft.public.sqlserver.olap forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
SmartGuy
 
Posts: n/a

Default To Process or not to Process, that is the Question.. - 01-11-2006 , 04:37 PM






Like most users I have a database with a few dimension tables and one fact
table. I add new data to both the fact table and dimension tables as the day
goes on. As soon as data is added to the fact and dimension tables, the
users pick up the scent of new data and want to see it ASAP.

I have a SSIS package that creates a new partition for the new data and sets
the filter appropriately.

So what exactly do I need to process? Do I need to process the partition
AND the dimensions? Can I just process the dimensions since they seem to
process the partitions anyway? Is there an easier way?

A few facts: Data is only ever added to the fact and dimension tables -
never deleted or udpated. I'm not married to the idea of creating a new
partition with every new chunk of data that I get.

Thanks,

SmartGuy

Reply With Quote
  #2  
Old   
Akshai Mirchandani [MS]
 
Posts: n/a

Default Re: To Process or not to Process, that is the Question.. - 01-11-2006 , 05:30 PM






Normally, you would ProcessAdd or ProcessUpdate the affected dimension(s)
and you would ProcessAdd to the last partition. On a regular basis you may
create a new partition so that the older data goes into a static partition.

It's not so easy to do ProcessAdd on a dimension (setting it up can be
tricky) -- but if your dimensions aren't tremendously large it's usually
fine to do ProcessUpdate. ProcessUpdate will scan all your dimension data
again, but it lets you keep the partition data without reprocessing.

HTH,
Akshai
--
This posting is provided "AS IS" with no warranties, and confers no rights
Please do not send email directly to this alias. This alias is for newsgroup
purposes only.

"SmartGuy" <SmartGuy (AT) discussions (DOT) microsoft.com> wrote

Quote:
Like most users I have a database with a few dimension tables and one fact
table. I add new data to both the fact table and dimension tables as the
day
goes on. As soon as data is added to the fact and dimension tables, the
users pick up the scent of new data and want to see it ASAP.

I have a SSIS package that creates a new partition for the new data and
sets
the filter appropriately.

So what exactly do I need to process? Do I need to process the partition
AND the dimensions? Can I just process the dimensions since they seem to
process the partitions anyway? Is there an easier way?

A few facts: Data is only ever added to the fact and dimension tables -
never deleted or udpated. I'm not married to the idea of creating a new
partition with every new chunk of data that I get.

Thanks,

SmartGuy



Reply With Quote
  #3  
Old   
SmartGuy
 
Posts: n/a

Default Re: To Process or not to Process, that is the Question.. - 01-11-2006 , 05:40 PM



Thanks for the info. Our dimension are very large - it takes 30 mins to
process them, so ProcessAdd may be what I need to do. Where can I read up on
ProcessAdd and/or find code samples?
--
SmartGuy


"Akshai Mirchandani [MS]" wrote:

Quote:
Normally, you would ProcessAdd or ProcessUpdate the affected dimension(s)
and you would ProcessAdd to the last partition. On a regular basis you may
create a new partition so that the older data goes into a static partition.

It's not so easy to do ProcessAdd on a dimension (setting it up can be
tricky) -- but if your dimensions aren't tremendously large it's usually
fine to do ProcessUpdate. ProcessUpdate will scan all your dimension data
again, but it lets you keep the partition data without reprocessing.

HTH,
Akshai
--
This posting is provided "AS IS" with no warranties, and confers no rights
Please do not send email directly to this alias. This alias is for newsgroup
purposes only.

"SmartGuy" <SmartGuy (AT) discussions (DOT) microsoft.com> wrote in message
news:C1E371E1-6348-46BB-9AB9-54CCBA9A50F0 (AT) microsoft (DOT) com...
Like most users I have a database with a few dimension tables and one fact
table. I add new data to both the fact table and dimension tables as the
day
goes on. As soon as data is added to the fact and dimension tables, the
users pick up the scent of new data and want to see it ASAP.

I have a SSIS package that creates a new partition for the new data and
sets
the filter appropriately.

So what exactly do I need to process? Do I need to process the partition
AND the dimensions? Can I just process the dimensions since they seem to
process the partitions anyway? Is there an easier way?

A few facts: Data is only ever added to the fact and dimension tables -
never deleted or udpated. I'm not married to the idea of creating a new
partition with every new chunk of data that I get.

Thanks,

SmartGuy




Reply With Quote
  #4  
Old   
Akshai Mirchandani [MS]
 
Posts: n/a

Default Re: To Process or not to Process, that is the Question.. - 01-12-2006 , 05:21 PM



There is a nice whitepaper on this Processing here:

http://msdn.microsoft.com/library/de...asprocarch.asp

You should look at the section on "Out of Line Bindings" and ProcessAdd.

Thanks,
Akshai
--
This posting is provided "AS IS" with no warranties, and confers no rights
Please do not send email directly to this alias. This alias is for newsgroup
purposes only.

"SmartGuy" <SmartGuy (AT) discussions (DOT) microsoft.com> wrote

Quote:
Thanks for the info. Our dimension are very large - it takes 30 mins to
process them, so ProcessAdd may be what I need to do. Where can I read up
on
ProcessAdd and/or find code samples?
--
SmartGuy


"Akshai Mirchandani [MS]" wrote:

Normally, you would ProcessAdd or ProcessUpdate the affected dimension(s)
and you would ProcessAdd to the last partition. On a regular basis you
may
create a new partition so that the older data goes into a static
partition.

It's not so easy to do ProcessAdd on a dimension (setting it up can be
tricky) -- but if your dimensions aren't tremendously large it's usually
fine to do ProcessUpdate. ProcessUpdate will scan all your dimension data
again, but it lets you keep the partition data without reprocessing.

HTH,
Akshai
--
This posting is provided "AS IS" with no warranties, and confers no
rights
Please do not send email directly to this alias. This alias is for
newsgroup
purposes only.

"SmartGuy" <SmartGuy (AT) discussions (DOT) microsoft.com> wrote in message
news:C1E371E1-6348-46BB-9AB9-54CCBA9A50F0 (AT) microsoft (DOT) com...
Like most users I have a database with a few dimension tables and one
fact
table. I add new data to both the fact table and dimension tables as
the
day
goes on. As soon as data is added to the fact and dimension tables,
the
users pick up the scent of new data and want to see it ASAP.

I have a SSIS package that creates a new partition for the new data and
sets
the filter appropriately.

So what exactly do I need to process? Do I need to process the
partition
AND the dimensions? Can I just process the dimensions since they seem
to
process the partitions anyway? Is there an easier way?

A few facts: Data is only ever added to the fact and dimension
tables -
never deleted or udpated. I'm not married to the idea of creating a
new
partition with every new chunk of data that I get.

Thanks,

SmartGuy






Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.