dbTalk Databases Forums  

Replacing a set of data in cube

microsoft.public.sqlserver.olap microsoft.public.sqlserver.olap


Discuss Replacing a set of data in cube in the microsoft.public.sqlserver.olap forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Shirish Nair
 
Posts: n/a

Default Replacing a set of data in cube - 10-27-2005 , 02:25 AM






Hi,

Is there a way to replace a set of data in the cube, without going for a
full process ?
Curently I am facing a situation where the a data on a particualr date was
incorrect and was processed into the cube.
The data has been corrected and posted into the data warehouse but it needs
to be corrected in the AS cube. If I do an incremental update then I
understand that the data would get duplicated and the count would be doubled.

One way is that I could do a full process for the cube which would be quite
difficult as I would need to restore from my archives and process the old
data.

I am sure some of you might have faced a similiar situtation. PLease guide
me on this.

Thank you,

Shirish

Reply With Quote
  #2  
Old   
Darren Gosbell
 
Posts: n/a

Default Re: Replacing a set of data in cube - 10-27-2005 , 04:40 AM






Can identify all the changes? If so you should be able to generate
"delta" records that just reflect the movements in the values and
process these as an incremental update. Below is a very simplistic
example of what I am talking about.

eg.

Original Values
================
Product, Month, SalesAmount
Product A, Oct 2005, 100
Product B, Oct 2005, 90

New values
===============
Product, Month, SalesAmount
Product A, Oct 2005, 110
Product B, Oct 2005, 70


would generate "delta" records of:

Product, Month, SalesAmount
Product A, Oct 2005, 10
Product B, Oct 2005, -20

You should be able to use these delta to incrementally process your cube
to adjust the values
--
Regards
Darren Gosbell [MCSD]
<dgosbell_at_yahoo_dot_com>
Blog: http://www.geekswithblogs.net/darrengosbell

In article <D8B1BFD1-65DE-4B57-B97F-F1B2EE50F916 (AT) microsoft (DOT) com>,
ShirishNair (AT) discussions (DOT) microsoft.com says...
Quote:
Hi,

Is there a way to replace a set of data in the cube, without going for a
full process ?
Curently I am facing a situation where the a data on a particualr date was
incorrect and was processed into the cube.
The data has been corrected and posted into the data warehouse but it needs
to be corrected in the AS cube. If I do an incremental update then I
understand that the data would get duplicated and the count would be doubled.

One way is that I could do a full process for the cube which would be quite
difficult as I would need to restore from my archives and process the old
data.

I am sure some of you might have faced a similiar situtation. PLease guide
me on this.

Thank you,

Shirish



Reply With Quote
  #3  
Old   
Shirish Nair
 
Posts: n/a

Default Re: Replacing a set of data in cube - 10-28-2005 , 12:56 AM



Thank you Darren,

Your approach would work, a small doubt though, if I have a meassure which
is count of records i.e. (count of sales trasanctions) then the count would
get doubled up.

One workaround is that, I should never use the count measure instead use sum
which could be on a field in the table which would have value of 1 and 0. Is
this the only way ? it would mean changing all the cubes

shirish

"Darren Gosbell" wrote:

Quote:
Can identify all the changes? If so you should be able to generate
"delta" records that just reflect the movements in the values and
process these as an incremental update. Below is a very simplistic
example of what I am talking about.

eg.

Original Values
================
Product, Month, SalesAmount
Product A, Oct 2005, 100
Product B, Oct 2005, 90

New values
===============
Product, Month, SalesAmount
Product A, Oct 2005, 110
Product B, Oct 2005, 70


would generate "delta" records of:

Product, Month, SalesAmount
Product A, Oct 2005, 10
Product B, Oct 2005, -20

You should be able to use these delta to incrementally process your cube
to adjust the values
--
Regards
Darren Gosbell [MCSD]
dgosbell_at_yahoo_dot_com
Blog: http://www.geekswithblogs.net/darrengosbell

In article <D8B1BFD1-65DE-4B57-B97F-F1B2EE50F916 (AT) microsoft (DOT) com>,
ShirishNair (AT) discussions (DOT) microsoft.com says...
Hi,

Is there a way to replace a set of data in the cube, without going for a
full process ?
Curently I am facing a situation where the a data on a particualr date was
incorrect and was processed into the cube.
The data has been corrected and posted into the data warehouse but it needs
to be corrected in the AS cube. If I do an incremental update then I
understand that the data would get duplicated and the count would be doubled.

One way is that I could do a full process for the cube which would be quite
difficult as I would need to restore from my archives and process the old
data.

I am sure some of you might have faced a similiar situtation. PLease guide
me on this.

Thank you,

Shirish




Reply With Quote
  #4  
Old   
Darren Gosbell
 
Posts: n/a

Default Re: Replacing a set of data in cube - 10-28-2005 , 02:23 AM



You are right, the "delta" approach will only work with measures that
are aggregated using SUM, counts will double count any adjusted records.

You are right that changing the counts to sum a column that just had 1
and 0 would work, but as you say this would mean changing all the cubes,
which would mean restoring the source data from your archives anyway.

One thing you might want to look into in future is setting up partitions
in your cubes (if you have the Enterprise Edition). I would look into
setting up one (or more) partitions for recent data and one (or more)
partitions for older data for which the source data had been archived.
That way you could re-process just the recent data if you need to make
corrections in future, leaving the older partition alone.

--
Regards
Darren Gosbell [MCSD]
<dgosbell_at_yahoo_dot_com>
Blog: http://www.geekswithblogs.net/darrengosbell

In article <AF67D07C-593E-4F86-8410-7A666AA690AF (AT) microsoft (DOT) com>,
ShirishNair (AT) discussions (DOT) microsoft.com says...
Quote:
Thank you Darren,

Your approach would work, a small doubt though, if I have a meassure which
is count of records i.e. (count of sales trasanctions) then the count would
get doubled up.

One workaround is that, I should never use the count measure instead use sum
which could be on a field in the table which would have value of 1 and 0. Is
this the only way ? it would mean changing all the cubes

shirish

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.