dbTalk Databases Forums  

General SAN advice needed for 2TB 9.2.0.8 database on HP-UX

comp.databases.oracle.server comp.databases.oracle.server


Discuss General SAN advice needed for 2TB 9.2.0.8 database on HP-UX in the comp.databases.oracle.server forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
DG problem
 
Posts: n/a

Default General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-23-2010 , 01:39 AM






Running 9.2.0.8 (still) on HP-UX 11.11
2TB database Used for OLTP and reporting
New SAN is a Hitachi 5TB AMS2100

It seems that these days the simple guidelines for configuring a large
Oracle database on a SAN is to simply configure the SAN with a one
megabyte stripe width across all disks (2 x 8 disks plus one spare).
And, to use a cache block size which is the same size as the Oracle
DB_BLOCK_SIZE which is 8192 (8kB)?

All redo logs, archive logs, datafiles and tempfiles would be on the
same mountpoint across the 1MB stripe.

We also plan to use RAID 1+0

Is this configuration in the right ball park?

Any suggestions are welcome.


I took the 1MB stripe width size from the following document.
Deploying, Managing, and Administering the Oracle Internet Platform
Optimal Storage Configuration Made Easy
Juan Loaiza, Oracle Corporation

Which states:
1) Stripe all files across all disks using a one megabyte stripe
width.
2) Mirror Data for high availability.

Reply With Quote
  #2  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-23-2010 , 05:00 AM






On 06/23/2010 07:39 AM, DG problem wrote:
Quote:
Running 9.2.0.8 (still) on HP-UX 11.11
2TB database Used for OLTP and reporting
New SAN is a Hitachi 5TB AMS2100

It seems that these days the simple guidelines for configuring a large
Oracle database on a SAN is to simply configure the SAN with a one
megabyte stripe width across all disks (2 x 8 disks plus one spare).
And, to use a cache block size which is the same size as the Oracle
DB_BLOCK_SIZE which is 8192 (8kB)?

All redo logs, archive logs, datafiles and tempfiles would be on the
same mountpoint across the 1MB stripe.

We also plan to use RAID 1+0

Is this configuration in the right ball park?

Any suggestions are welcome.


I took the 1MB stripe width size from the following document.
Deploying, Managing, and Administering the Oracle Internet Platform
Optimal Storage Configuration Made Easy
Juan Loaiza, Oracle Corporation

Which states:
1) Stripe all files across all disks using a one megabyte stripe
width.
2) Mirror Data for high availability.
Make it all SAME: Stripe and Mirror Everything.
Don't know about stripe sizes - what do you think to gain by
playing with these? Did you look at the release date
of aforementioned document? If you want to deviate from
the vendors recommendations of stripe sizes, make sure
to test, Test and TEST.

If anything else than vendor recommendation comes
from that, the ng would appreciate feedback. Did
I mention testing?

--

Regards,

Frank van Bortel (who's a member of BAARF)

Reply With Quote
  #3  
Old   
Mladen Gogala
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-23-2010 , 10:07 AM



On Wed, 23 Jun 2010 11:00:11 +0200, Frank van Bortel wrote:

Quote:
Make it all SAME: Stripe and Mirror Everything.
You're in marketing? SAME == RAID 1+0.

Quote:
Don't know about stripe sizes -
That was the OP's question.

Quote:
what do you think to gain by playing with these?
Speed.

And yes, stripe size of 1MB is good. Allegedly, 11G will work well with
4M, but I haven't had an opportunity to try that.




--
http://mgogala.byethost5.com

Reply With Quote
  #4  
Old   
DG problem
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-24-2010 , 02:47 AM



On Jun 24, 12:07*am, Mladen Gogala <gogala.mla... (AT) gmail (DOT) com> wrote:
Quote:
On Wed, 23 Jun 2010 11:00:11 +0200, Frank van Bortel wrote:
Make it all SAME: Stripe and Mirror Everything.

You're in marketing? SAME == RAID 1+0.

Don't know about stripe sizes -

That was the OP's question.

what do you think to gain by playing with these?

Speed.

And yes, stripe size of 1MB is good. Allegedly, 11G will work well with
4M, but I haven't had an opportunity to try that.

--http://mgogala.byethost5.com
Just some more notes to confirm that I have this correct.


Stripe depth is the size of the stripe, sometimes called stripe unit.
Stripe width is the product of the stripe depth and the number of
drives in the striped set.
http://download.oracle.com/docs/cd/B...ign.htm#i28412

segment size = stripe depth
1MB stripe width = eight drives * stripe depth = 8 * 128KB

This SAN parameter is dynamic apparently (guess it depends on brand)
SAN Cache block size = SAN cache page size = DB_BLOCK_SIZE = 8KB
DB_BLOCK_SIZE should = file block size which should = cache block
size
http://www.emc.com/collateral/hardwa...tor-sys-wp.pdf

We probably don't need to worry about making the log files non-cached
as the cache is battery backed up?

Reply With Quote
  #5  
Old   
John Hurley
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-24-2010 , 09:23 AM



DG:

Quote:
Just some more notes to confirm that I have this correct.

Stripe depth is the size of the stripe, sometimes called stripe unit.
Stripe width is the product of the stripe depth and the number of
drives in the striped set.http://download.oracle.com/docs/cd/B.../b28274/iodesi...

segment size = stripe depth
1MB stripe width = eight drives * stripe depth = 8 * 128KB

This SAN parameter is dynamic apparently (guess it depends on brand)
SAN Cache block size = SAN cache page size = DB_BLOCK_SIZE = 8KB
DB_BLOCK_SIZE should = file block size which should = cache block
sizehttp://www.emc.com/collateral/hardware/white-papers/h796-implementing...

We probably don't need to worry about making the log files non-cached
as the cache is battery backed up?
Your messed up if you can't depend on the cache of your storage array.

Most of my experience with Oracle systems has been setting up OLTP
( not big datawarehouse ) systems.

I almost always separate out the disks for online logs from the rest
of the system. For commit happy systems I really want to have average
log file sync times at 1 ms or under if I can.

These days my systems are using ASM but I still setup separate
diskgroups for the online log files and separate the disks and that
stuff out.

Reply With Quote
  #6  
Old   
DG problem
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-25-2010 , 01:45 AM



Quote:
I almost always separate out the disks for online logs from the rest
of the system. *For commit happy systems I really want to have average
log file sync times at 1 ms or under if I can.
So even though Oracle recommends putting everything on the same
stripe, you still separate the log files? Have you noticed any
performance gains from doing this from your testing?

The thing is that being able to write out log files across an 8 disk
stripe means much faster write times if the IO isn't being chugged
down by other processes.

Reply With Quote
  #7  
Old   
John Hurley
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-25-2010 , 06:37 AM



DG:

Quote:
So even though Oracle recommends putting everything on the same
stripe, you still separate the log files? Have you noticed any
performance gains from doing this from your testing?

The thing is that being able to write out log files across an 8 disk
stripe means much faster write times if the IO isn't being chugged
down by other processes.
You have to test in your own environment and measure what different
setups work better or not so much. As I noted my systems are OLTP
databases with some very commit happy applications ( commit too
much ... like on every row often ).

The one size fits all recommendations are generic ones that may or may
not make sense. Like I noted I get an average of less than 1 ms for
log file sync ... that is the wait event I tune for on the online log
files.

I am also using for my most important prod systems RAID 1 for the LUNs
for the log files and RAID 1+0 for the rest of the database portions
( again using ASM for both ). Any competent storage array buffers the
write activity in cache.

For test databases and our datawarehouse systems we are using RAID
5 ... not my prod OLTP systems.

Yes you have to do a lot of work to try different setups and write
different things to simulate different kind of workloads. Custom
designed tests take a lot of time but things that go after your
important tables in ways that ( somewhat ) simulate your real
applications give better and ( hopefully ) more reliable performance
numbers than generic testing tools that are available.

Reply With Quote
  #8  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-29-2010 , 03:26 PM



On 06/23/2010 04:07 PM, Mladen Gogala wrote:
Quote:
On Wed, 23 Jun 2010 11:00:11 +0200, Frank van Bortel wrote:

Make it all SAME: Stripe and Mirror Everything.

You're in marketing? SAME == RAID 1+0.
No; and I prefer RAID 0+1.
Not in storage, I seem to recall striping was RAID-0, mirroring
was RAID-1. Same would be RAID-0+1, then, eh?
Quote:
Don't know about stripe sizes -

That was the OP's question.

what do you think to gain by playing with these?

Speed.

Still, no one provided any hard numbers.

Everybody seems convinced about gains in playing
with these figures, but the question remains:
- is 1MB stripe any different from what the
vendor recommended?
- if so, what gains in speed were measured
ordered by stripe size?


--

Regards,

Frank van Bortel

Reply With Quote
  #9  
Old   
joel garry
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 06-29-2010 , 04:17 PM



On Jun 29, 12:26*pm, Frank van Bortel <fbor... (AT) home (DOT) nl> wrote:
Quote:
On 06/23/2010 04:07 PM, Mladen Gogala wrote:

On Wed, 23 Jun 2010 11:00:11 +0200, Frank van Bortel wrote:

Make it all SAME: Stripe and Mirror Everything.

You're in marketing? SAME == RAID 1+0.

No; and I prefer RAID 0+1.
Not in storage, I seem to recall striping was RAID-0, mirroring
was RAID-1. Same would be RAID-0+1, then, eh?
No, see what the faq says, and that 1+0 is better (for fault
tolerance): http://www.orafaq.com/wiki/RAID

http://jonathanlewis.wordpress.com/2...aster-stripes/ has
some good thoughts.

Quote:


Don't know about stripe sizes -

That was the OP's question.

what do you think to gain by playing with these?

Speed.

Still, no one provided any hard numbers.

Everybody seems convinced about gains in playing
with these figures, but the question remains:
- is 1MB stripe any different from what the
* *vendor recommended?
- if so, what gains in speed were measured
* *ordered by stripe size?
That's the gist: measure what the real load will be doing. I'd agree
with John about online logs, but most of my experience over the last
decade has been with RAID-5, so I didn't say anything 'cause I really
haven't looked. And yes, I'm a baarf member.

This one seems to say go coarse on the stripes:
http://www.hds.com/assets/pdf/best-p...c-with-hds.pdf
I haven't looked at it too closely to see if it makes sense, and
haven't checked whether it applies to the OP hardware. "The default
stripe depth for an ASM group is 1MB, which is too low to make
efficient use of the underlying RAID group stripe size. This should be
set to 8MB at ASM group creation time by specifying the AU_SIZE
attribute..."

jg
--
@home.com is bogus.
"I believe it’s horribly bad in SQL Server, but it’s doubly horribly
bad in Oracle." - Tom Kyte on triggers

Reply With Quote
  #10  
Old   
Frank van Bortel
 
Posts: n/a

Default Re: General SAN advice needed for 2TB 9.2.0.8 database on HP-UX - 07-03-2010 , 09:41 AM



On 06/29/2010 10:17 PM, joel garry wrote:
Quote:
On Jun 29, 12:26 pm, Frank van Bortel<fbor... (AT) home (DOT) nl> wrote:
On 06/23/2010 04:07 PM, Mladen Gogala wrote:

On Wed, 23 Jun 2010 11:00:11 +0200, Frank van Bortel wrote:

Make it all SAME: Stripe and Mirror Everything.

You're in marketing? SAME == RAID 1+0.

No; and I prefer RAID 0+1.
Not in storage, I seem to recall striping was RAID-0, mirroring
was RAID-1. Same would be RAID-0+1, then, eh?

No, see what the faq says, and that 1+0 is better (for fault
tolerance): http://www.orafaq.com/wiki/RAID
1+0 it is.
Quote:
http://jonathanlewis.wordpress.com/2...aster-stripes/ has
some good thoughts.




Don't know about stripe sizes -

That was the OP's question.

what do you think to gain by playing with these?

Speed.

Still, no one provided any hard numbers.

Everybody seems convinced about gains in playing
with these figures, but the question remains:
- is 1MB stripe any different from what the
vendor recommended?
- if so, what gains in speed were measured
ordered by stripe size?

That's the gist: measure what the real load will be doing. I'd agree
with John about online logs, but most of my experience over the last
decade has been with RAID-5, so I didn't say anything 'cause I really
haven't looked. And yes, I'm a baarf member.

This one seems to say go coarse on the stripes:
http://www.hds.com/assets/pdf/best-p...c-with-hds.pdf
I haven't looked at it too closely to see if it makes sense, and
haven't checked whether it applies to the OP hardware. "The default
stripe depth for an ASM group is 1MB, which is too low to make
efficient use of the underlying RAID group stripe size. This should be
set to 8MB at ASM group creation time by specifying the AU_SIZE
attribute..."

jg
--
@home.com is bogus.
"I believe it’s horribly bad in SQL Server, but it’s doubly horribly
bad in Oracle." - Tom Kyte on triggers

Will check it out - any hard numbers?

--

Regards,

Frank van Bortel

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.