![]() | |
![]() |
| | Thread Tools | Display Modes |
#1
| |||
| |||
|
#2
| |||
| |||
|
|
AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
#3
| |||
| |||
|
|
can you send the XML/A command used to process the cubes? how many CPU have you? do you use the lazy aggregation option? "akornilovski (AT) gmail (DOT) com" wrote: AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
#4
| |||
| |||
|
|
- No lazy aggregations - 8 CPUs - As far as XML/A command - it is "normal" parallel processing comand from BIDS i.e. selecting several partitions and choosing process in parallel option: Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" Parallel Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q3</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q4</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process /Parallel /Batch Jerome P wrote: can you send the XML/A command used to process the cubes? how many CPU have you? do you use the lazy aggregation option? "akornilovski (AT) gmail (DOT) com" wrote: AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
#5
| |||
| |||
|
|
have you try to force the number of parrallel process? i have some partitions in my cubes, and everything is evaluated in parrallel with the default options. So... I can't help you more. you have nothing wrong from my point of view. "akornilovski (AT) gmail (DOT) com" wrote: - No lazy aggregations - 8 CPUs - As far as XML/A command - it is "normal" parallel processing comand from BIDS i.e. selecting several partitions and choosing process in parallel option: Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" Parallel Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q3</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q4</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process /Parallel /Batch Jerome P wrote: can you send the XML/A command used to process the cubes? how many CPU have you? do you use the lazy aggregation option? "akornilovski (AT) gmail (DOT) com" wrote: AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
#6
| |||
| |||
|
|
Thankls Jerome, Anybody else have more info on the subject? Jerome P wrote: have you try to force the number of parrallel process? i have some partitions in my cubes, and everything is evaluated in parrallel with the default options. So... I can't help you more. you have nothing wrong from my point of view. "akornilovski (AT) gmail (DOT) com" wrote: - No lazy aggregations - 8 CPUs - As far as XML/A command - it is "normal" parallel processing comand from BIDS i.e. selecting several partitions and choosing process in parallel option: Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" Parallel Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q3</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q4</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process /Parallel /Batch Jerome P wrote: can you send the XML/A command used to process the cubes? how many CPU have you? do you use the lazy aggregation option? "akornilovski (AT) gmail (DOT) com" wrote: AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
#7
| |||
| |||
|
|
I read here: http://support.microsoft.com/kb/914595/ that SSAS incorrectly estimates the memory required for processing dimensions. This could be the case for processing measure groups as well. This would explain why SSAS is asking for 22Gigs but your memory usage never exceeds 10Gigs...just a hypothesis... In any case, where can you view the "Quota KB" parameter? I am running into errors where SSAS is asking for 17 Gigs to process a partition when I only have 13 Gigs available. I am trying to determine the impact of removing partitions on this memory quota estimate. Any advice would be much appreciated. Thanks. "akornilovski (AT) gmail (DOT) com" wrote: Thankls Jerome, Anybody else have more info on the subject? Jerome P wrote: have you try to force the number of parrallel process? i have some partitions in my cubes, and everything is evaluated in parrallel with the default options. So... I can't help you more. you have nothing wrong from my point of view. "akornilovski (AT) gmail (DOT) com" wrote: - No lazy aggregations - 8 CPUs - As far as XML/A command - it is "normal" parallel processing comand from BIDS i.e. selecting several partitions and choosing process in parallel option: Batch xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" Parallel Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q3</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process Process xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Object DatabaseID>MIC_BU</DatabaseID CubeID>SubPrime</CubeID MeasureGroupID>MICSubPrime</MeasureGroupID PartitionID>P2001Q4</PartitionID /Object Type>ProcessFull</Type WriteBackTableCreation>UseExisting</WriteBackTableCreation /Process /Parallel /Batch Jerome P wrote: can you send the XML/A command used to process the cubes? how many CPU have you? do you use the lazy aggregation option? "akornilovski (AT) gmail (DOT) com" wrote: AS2005 x64: When doing parallel processing of my partitions I've noticed that AS2005 reads data from the source in parallel however when it switches to the Aggregate phase it is doing it one by one and not in parallel. I have 32GB of memory and my memory usage is below 10GB all the time during processing, however "Quota KB" parameter is showing that AS is asking for 22GB therefore blocking all but one Aggregation process. Anybody knows why there is such a discrepancy and what can be done about it? Thanks! |
![]() |
| Thread Tools | |
| Display Modes | |
| |