dbTalk Databases Forums  

Performance of processing

microsoft.public.sqlserver.olap microsoft.public.sqlserver.olap


Discuss Performance of processing in the microsoft.public.sqlserver.olap forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
herman
 
Posts: n/a

Default Performance of processing - 09-08-2006 , 03:48 AM






Hi



Is there any place that I can find some benchmarks, tests regarding
performance o professing cubes and ETL process.

I have to calculate how long will the whole process (ETL and processing the
cubes) will take on a particular hardware.

I also need some information how can I determine the appropriate hardware
setup to accomplish processing cubes in a particular time.



Regards

Radek







Reply With Quote
  #2  
Old   
Jeje
 
Posts: n/a

Default Re: Performance of processing - 09-08-2006 , 06:59 AM






there is no benchmark like this.
the number of components is too large to provide a performance / hardware
formula:
speed of the source database(s) (your OLTP source databases or flat files)
speed of the destination disks
speed of the disk controller (and memory cache)
type of Raid
speed and amount of memory
type and complexity of your ETLs transformations
number of columns in the tables
number of indexes in the tables
level of aggregations in the cubes
number and type of partitions
number of dimensions in the cubes
number of processes in parrallel
separates or shared server for database and cubes
speed of the network
speed and number of CPUs
IO architecture (Intel vs AMD)
and others...

again and again, there is no miracle: test and growth!

I have setup a lot of servers for DW & OLAP usage and on the paper the
servers should works fine or 1 better then the other, but in fact its not
always true due to too many parameters.
But from my experience:
use direct attached storage and the SAS technology, ignore SCSI drives and
controllers for DW usage they limit you
During sequential access using 15Krpm or 10Krpm drive provide no
differences. SAS controllers provide better throughput and are more scalable
then SCSI controllers (take a look at the TPC.org site)

the latest Intel CPU are incredible, but Opteron CPUs provides a constant
performance (there is less IO issues)
I have an Intel server where all individual benchmarks (disk, memory, cpu)
results are excellent compared to the Opteron server, but during real usage
where the 3 components works at the same time the Opteron server is finally
better, because there is no IO bottleneck!!!
take the highest Ghz you can instead of more CPU with lower frequency.

good luck in your setup :-)

Jerome.


"herman" <rhermaszewski (AT) o2 (DOT) pl> wrote

Quote:
Hi



Is there any place that I can find some benchmarks, tests regarding
performance o professing cubes and ETL process.

I have to calculate how long will the whole process (ETL and processing
the cubes) will take on a particular hardware.

I also need some information how can I determine the appropriate hardware
setup to accomplish processing cubes in a particular time.



Regards

Radek









Reply With Quote
  #3  
Old   
herman
 
Posts: n/a

Default Re: Performance of processing - 09-11-2006 , 05:02 AM



OK, thank you for your help.


Użytkownik "Jeje" <willgart (AT) hotmail (DOT) com> napisał w wiadomości
news:uDwsG5z0GHA.3464 (AT) TK2MSFTNGP03 (DOT) phx.gbl...
Quote:
there is no benchmark like this.
the number of components is too large to provide a performance / hardware
formula:
speed of the source database(s) (your OLTP source databases or flat files)
speed of the destination disks
speed of the disk controller (and memory cache)
type of Raid
speed and amount of memory
type and complexity of your ETLs transformations
number of columns in the tables
number of indexes in the tables
level of aggregations in the cubes
number and type of partitions
number of dimensions in the cubes
number of processes in parrallel
separates or shared server for database and cubes
speed of the network
speed and number of CPUs
IO architecture (Intel vs AMD)
and others...

again and again, there is no miracle: test and growth!

I have setup a lot of servers for DW & OLAP usage and on the paper the
servers should works fine or 1 better then the other, but in fact its not
always true due to too many parameters.
But from my experience:
use direct attached storage and the SAS technology, ignore SCSI drives and
controllers for DW usage they limit you
During sequential access using 15Krpm or 10Krpm drive provide no
differences. SAS controllers provide better throughput and are more
scalable then SCSI controllers (take a look at the TPC.org site)

the latest Intel CPU are incredible, but Opteron CPUs provides a constant
performance (there is less IO issues)
I have an Intel server where all individual benchmarks (disk, memory, cpu)
results are excellent compared to the Opteron server, but during real
usage where the 3 components works at the same time the Opteron server is
finally better, because there is no IO bottleneck!!!
take the highest Ghz you can instead of more CPU with lower frequency.

good luck in your setup :-)

Jerome.


"herman" <rhermaszewski (AT) o2 (DOT) pl> wrote in message
news:edrapb$av$1 (AT) atlantis (DOT) news.tpi.pl...
Hi



Is there any place that I can find some benchmarks, tests regarding
performance o professing cubes and ETL process.

I have to calculate how long will the whole process (ETL and processing
the cubes) will take on a particular hardware.

I also need some information how can I determine the appropriate hardware
setup to accomplish processing cubes in a particular time.



Regards

Radek











Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.