dbTalk Databases Forums  

SDS and SAN Storage

comp.databases.informix comp.databases.informix


Discuss SDS and SAN Storage in the comp.databases.informix forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
rpa
 
Posts: n/a

Default SDS and SAN Storage - 08-12-2010 , 02:21 PM






Hi all

what is possible and what is the best way to create chunks on a SAN-
Storage (NetApp) for SDS.
We are planning for IDS 11.50 database installations with SDS feature.

OS will be LINUX (SLES or UBUNTU).

As Storage we have a NetApp SAN.

In case of cooked files (DIRECT_IO=1), which Filesytem has to be
choosen ?

How can we use raw devices over Fiber Channel ?
Present a lun to all SDS-servers and use it as raw device ?
Is it so simple or is something missing ?

What is best practice to use NetApp-SAN-Storage over FiberChannel for
Informix SDS-cluster ?

TIA
Ruediger Papke

Reply With Quote
  #2  
Old   
Davorin Kremenjas
 
Posts: n/a

Default Re: SDS and SAN Storage - 08-13-2010 , 05:10 AM






On Aug 12, 8:21*pm, rpa <ruediger.pa... (AT) t-online (DOT) de> wrote:
Quote:
Hi all

what is possible and what is the best way to create chunks on a SAN-
Storage (NetApp) for SDS.
We are planning for IDS 11.50 database installations with SDS feature.

OS *will be LINUX (SLES * or UBUNTU).

As Storage we have a NetApp SAN.

In case of cooked files (DIRECT_IO=1), which Filesytem has to be
choosen ?

How can we use raw devices over Fiber Channel ?
Present a lun to all SDS-servers and use it as raw device ?
Is it so simple or is something missing ?

What is best practice to use NetApp-SAN-Storage over FiberChannel for
Informix SDS-cluster ?
Hi Ruediger,

Hard to say which is _the best_ way, you'll probably get as many
different advices as the follow-up posts to your question
Here is my take - yes, it is quite simple as you've already guessed:

1. Forget filesystems, go raw (Fibre Channel doesn't care)
2. Use RAID10 on your SAN, you'll get both performance and reliability
3. Assuming this will be OLTP engine, use 16kb (Linux = 2kb pages x8
a.k.a. big buffers) segment/stripe sizes on your SAN while configuring
RAID10; for DSS go as big as you can
4. try to use all SAN controllers and HBA's on your host in active
mode with alternating preferred paths for LUNs to achieve max
throughput
5. carve out raw logical volumes on the host for Informix chunks; how
many chunks? - highly debatable, we try to stick to the rule of thumb
- 10 times more than physical disk drives in our SAN
6. ALWAYS use symbolic links for your chunks when adding them to
Informix
7. don't complicate things with separate data and index dbspaces -
RAID10 will spread the load for you; however, do follow the usual
advices for rootdbs, physical and logical logs separation, makes life
easier afterwards
8. enjoy the fact you're installing 11.50, you're not going to suffer
long checkpoints problems even if you get something wrong

Can't comment on SDS requirements, haven't been using SDS nodes yet.

HTH

Davorin

Reply With Quote
  #3  
Old   
Brian Amstutz
 
Posts: n/a

Default TAPEDEV or -t STDIO? - 08-13-2010 , 10:16 AM



HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO > /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO, my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO > <somefile> take significantly longer than
setting TAPEDEV to <somefile>?

Brian

Reply With Quote
  #4  
Old   
Goldrick, Jim
 
Posts: n/a

Default RE: TAPEDEV or -t STDIO? - 08-13-2010 , 10:18 AM



I have always suspected, though I don't know and never looked, that
when TAPEDEV is set to /dev/null, the archive is never really done.
Just some internal processing to make it seem so.

Thank you,

Jim Goldrick
Judson University
573-332-7739
http://www.judsonu.edu
jgoldrick (AT) judsonu (DOT) edu



-----Original Message-----
From: informix-list-bounces (AT) iiug (DOT) org
[mailto:informix-list-bounces (AT) iiug (DOT) org] On Behalf Of Brian Amstutz
Sent: Friday, August 13, 2010 10:16 AM
To: informix-list (AT) iiug (DOT) org
Subject: TAPEDEV or -t STDIO?

HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO > /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO,
my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO > <somefile> take significantly longer than
setting TAPEDEV to <somefile>?

Brian
_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Reply With Quote
  #5  
Old   
Clive Eisen
 
Posts: n/a

Default Re: TAPEDEV or -t STDIO? - 08-13-2010 , 10:38 AM



On 13/08/2010 16:16, Brian Amstutz wrote:
Quote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO> /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO, my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO> <somefile> take significantly longer than
setting TAPEDEV to<somefile>?
er RTFM?

http://publib.boulder.ibm.com/infoce...ds_bar_455.htm

specifically

When you set TAPEDEV to /dev/null and request a backup, the database
server bypasses the backup but still updates the dbspaces with the new
backup time stamps.

--
Clive

--
This message has been scanned for viruses and
dangerous content by OpenProtect(http://www.openprotect.com), and is
believed to be clean.

Reply With Quote
  #6  
Old   
Art Kagel
 
Posts: n/a

Default Re: TAPEDEV or -t STDIO? - 08-13-2010 , 10:48 AM



The engine recognizes /dev/null as a throw away and doesn't actually perform
any archive at all. It just updates the records to say that one was
performed, unless you specify -f in which case nothing happens at all.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.



On Fri, Aug 13, 2010 at 11:16 AM, Brian Amstutz <
brian.amstutz (AT) asburyseminary (DOT) edu> wrote:

Quote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO > /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO, my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO > <somefile> take significantly longer than
setting TAPEDEV to <somefile>?

Brian
_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Reply With Quote
  #7  
Old   
Brian Amstutz
 
Posts: n/a

Default RE: TAPEDEV or -t STDIO? - 08-13-2010 , 03:32 PM



I understand that /dev/null bypasses the backup and doesn't actually
perform any archive at all, but the manual also says that:

"The -t STDIO option overrides the value of the TAPEDEV configuration
parameter for the current backup."

http://publib.boulder.ibm.com/infoce...jsp?topic=/com.
ibm.bar.doc/barmst282.htm

** BTW this link is for v10 (as specified in my original post), not v115
**

So, if using -t STDIO causes the server to OVERRIDE the value of TAPEDEV
and I'm using -t STDIO > /dev/null, shouldn't either method,
theoretically, do the same thing and perform the same way?

However, my question still remains (aside from /dev/null) - when using
ontape to perform a backup to a file, is there any difference
(specifically performance-wise) between specifying the filename in TAPEDEV
and using -t STDIO filename?

Thanks!

Brian


Quote:
-----Original Message-----
From: Clive Eisen [mailto:clive (AT) serendipita (DOT) com]
Sent: Friday, August 13, 2010 11:39 AM
To: Brian Amstutz; informix-list (AT) iiug (DOT) org
Subject: Re: TAPEDEV or -t STDIO?

On 13/08/2010 16:16, Brian Amstutz wrote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO> /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO,
my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO> <somefile> take significantly longer than
setting TAPEDEV to<somefile>?

er RTFM?

http://publib.boulder.ibm.com/infoce.../com.ibm.bar.d
oc/ids_bar_455
.htm

specifically

When you set TAPEDEV to /dev/null and request a backup, the database
server bypasses the backup but still updates the dbspaces with the new
backup time stamps.

--
Clive

--
This message has been scanned for viruses and
dangerous content by OpenProtect(http://www.openprotect.com), and is
believed to be clean.

Reply With Quote
  #8  
Old   
Art Kagel
 
Posts: n/a

Default Re: TAPEDEV or -t STDIO? - 08-13-2010 , 03:47 PM



Overrides, not replaces in memory. Ontape and the engine do NOT know that
the shell has redirected STDIO to /dev/null (yes, ontape COULD stat the file
stdout and discover that it is connected to a device special file but tape
drives and RAW disks are also device special files), so then ontape is
behaving exactly like it was backing up to a "real" file and performs an
actual backup. Backups using TAPEDEV, whether it is set to a filename path
or to a directory name path will take the same time and MAY be slightly
faster than backing up to a file using STDIO, but only very slightly. All
in all, once the shell redirects stdout to a file, the actual IO is exactly
like writing to any other file. The application saves the overhead of
opening the file, but then the shell will have had the same overhead so
that's a wash. The 'driver' for the C/UNIX special file handling within the
C library has some small overhead, but it is very small and you won't be
able to distinguish it from random speed differences between any two runs in
general.

The ONLY reason the two runs you tested were so different was that TAPEDEV
set to /dev/null is handled differently, as has alrady been pointed out.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.



On Fri, Aug 13, 2010 at 4:32 PM, Brian Amstutz <
brian.amstutz (AT) asburyseminary (DOT) edu> wrote:

Quote:
I understand that /dev/null bypasses the backup and doesn't actually
perform any archive at all, but the manual also says that:

"The -t STDIO option overrides the value of the TAPEDEV configuration
parameter for the current backup."

http://publib.boulder.ibm.com/infoce...jsp?topic=/com.
ibm.bar.doc/barmst282.htm

** BTW this link is for v10 (as specified in my original post), not v115
**

So, if using -t STDIO causes the server to OVERRIDE the value of TAPEDEV
and I'm using -t STDIO > /dev/null, shouldn't either method,
theoretically, do the same thing and perform the same way?

However, my question still remains (aside from /dev/null) - when using
ontape to perform a backup to a file, is there any difference
(specifically performance-wise) between specifying the filename in TAPEDEV
and using -t STDIO filename?

Thanks!

Brian


-----Original Message-----
From: Clive Eisen [mailto:clive (AT) serendipita (DOT) com]
Sent: Friday, August 13, 2010 11:39 AM
To: Brian Amstutz; informix-list (AT) iiug (DOT) org
Subject: Re: TAPEDEV or -t STDIO?

On 13/08/2010 16:16, Brian Amstutz wrote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO> /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO,
my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO> <somefile> take significantly longer than
setting TAPEDEV to<somefile>?

er RTFM?

http://publib.boulder.ibm.com/infoce.../com.ibm.bar.d
oc/ids_bar_455<http://publib.boulder.ibm.com/infoce...oc/ids_bar_455
.htm

specifically

When you set TAPEDEV to /dev/null and request a backup, the database
server bypasses the backup but still updates the dbspaces with the new
backup time stamps.

--
Clive

--
This message has been scanned for viruses and
dangerous content by OpenProtect(http://www.openprotect.com), and is
believed to be clean.

_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Reply With Quote
  #9  
Old   
mpruet
 
Posts: n/a

Default Re: TAPEDEV or -t STDIO? - 08-14-2010 , 08:53 PM



On Aug 13, 3:32*pm, "Brian Amstutz" <brian.amst... (AT) asburyseminary (DOT) edu>
wrote:
Quote:
I understand that /dev/null bypasses the backup and doesn't actually
perform any archive at all, but the manual also says that:

"The -t STDIO option overrides the value of the TAPEDEV configuration
parameter for the current backup."

http://publib.boulder.ibm.com/infoce...dex.jsp?topic=.....
ibm.bar.doc/barmst282.htm

** BTW this link is for v10 (as specified in my original post), not v115
**

So, if using -t STDIO causes the server to OVERRIDE the value of TAPEDEV
and I'm using -t STDIO > /dev/null, shouldn't either method,
theoretically, do the same thing and perform the same way?

However, my question still remains (aside from /dev/null) - when using
ontape to perform a backup to a file, is there any difference
(specifically performance-wise) between specifying the filename in TAPEDEV
and using -t STDIO filename?
Brian,

When you set TAPEDEV to /dev/null, then ontape is able to read the
onconfig file, and thus knows that the backup is a NULL backup.

When you use -t STDIO > /dev/null, then ontape is not aware of the
final disposition of the backup. It only knows that it was requested
to pass the backup data to an output file via stdout. It does not no
that stdout is being directed to /dev/null. Only the shell knows
that.


M.P.
Quote:
Thanks!

Brian



-----Original Message-----
From: Clive Eisen [mailto:cl... (AT) serendipita (DOT) com]
Sent: Friday, August 13, 2010 11:39 AM
To: Brian Amstutz; informix-l... (AT) iiug (DOT) org
Subject: Re: TAPEDEV or -t STDIO?

On 13/08/2010 16:16, Brian Amstutz wrote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO> */dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO,
my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO> *<somefile> *take significantly longer than
setting TAPEDEV to<somefile>?

er RTFM?

http://publib.boulder.ibm.com/infoce...opic/com.ibm.b...
oc/ids_bar_455
.htm

specifically

When you set TAPEDEV to /dev/null and request a backup, the database
server bypasses the backup but still updates the dbspaces with the new
backup time stamps.

--
Clive

--
This message has been scanned for viruses and
dangerous content by OpenProtect(http://www.openprotect.com), and is
believed to be clean.

Reply With Quote
  #10  
Old   
Brian Amstutz
 
Posts: n/a

Default RE: TAPEDEV or -t STDIO? - 08-16-2010 , 07:36 AM



Got it! Thanks to all for your replies.

Brian

_____

From: Art Kagel [mailto:art.kagel (AT) gmail (DOT) com]
Sent: Friday, August 13, 2010 4:47 PM
To: Brian Amstutz
Cc: informix-list (AT) iiug (DOT) org
Subject: Re: TAPEDEV or -t STDIO?



Overrides, not replaces in memory. Ontape and the engine do NOT know that
the shell has redirected STDIO to /dev/null (yes, ontape COULD stat the
file stdout and discover that it is connected to a device special file but
tape drives and RAW disks are also device special files), so then ontape
is behaving exactly like it was backing up to a "real" file and performs
an actual backup. Backups using TAPEDEV, whether it is set to a filename
path or to a directory name path will take the same time and MAY be
slightly faster than backing up to a file using STDIO, but only very
slightly. All in all, once the shell redirects stdout to a file, the
actual IO is exactly like writing to any other file. The application
saves the overhead of opening the file, but then the shell will have had
the same overhead so that's a wash. The 'driver' for the C/UNIX special
file handling within the C library has some small overhead, but it is very
small and you won't be able to distinguish it from random speed
differences between any two runs in general.

The ONLY reason the two runs you tested were so different was that TAPEDEV
set to /dev/null is handled differently, as has alrady been pointed out.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly,
implicitly, or by inference. Neither do those opinions reflect those of
other individuals affiliated with any entity with which I am affiliated
nor those of the entities themselves.




On Fri, Aug 13, 2010 at 4:32 PM, Brian Amstutz
<brian.amstutz (AT) asburyseminary (DOT) edu> wrote:

I understand that /dev/null bypasses the backup and doesn't actually
perform any archive at all, but the manual also says that:

"The -t STDIO option overrides the value of the TAPEDEV configuration
parameter for the current backup."

http://publib.boulder.ibm.com/infoce...jsp?topic=/com.
ibm.bar.doc/barmst282.htm

** BTW this link is for v10 (as specified in my original post), not v115
**

So, if using -t STDIO causes the server to OVERRIDE the value of TAPEDEV
and I'm using -t STDIO > /dev/null, shouldn't either method,
theoretically, do the same thing and perform the same way?

However, my question still remains (aside from /dev/null) - when using
ontape to perform a backup to a file, is there any difference
(specifically performance-wise) between specifying the filename in TAPEDEV
and using -t STDIO filename?

Thanks!

Brian


Quote:
-----Original Message-----
From: Clive Eisen [mailto:clive (AT) serendipita (DOT) com]
Sent: Friday, August 13, 2010 11:39 AM
To: Brian Amstutz; informix-list (AT) iiug (DOT) org
Subject: Re: TAPEDEV or -t STDIO?


On 13/08/2010 16:16, Brian Amstutz wrote:
HP-UX 11i v2
Informix 10.00.FC9

ontape -s -L 0 with TAPEDEV set to /dev/null takes 2 seconds or less
ontape -s -L 0 -t STDIO> /dev/null still running after 30 minutes

Besides the question of "why" is it so much longer when using -t STDIO,
my
real question is if this archive were to a "real" file (instead of
/dev/null), would the time difference be significantly longer as well -
i.e. would using -t STDIO> <somefile> take significantly longer than
setting TAPEDEV to<somefile>?


er RTFM?

http://publib.boulder.ibm.com/infoce.../com.ibm.bar.d
http://publib.boulder.ibm.com/infoce.../com.ibm.bar.d
%0Aoc/ids_bar_455>
oc/ids_bar_455
Quote:
.htm

specifically

When you set TAPEDEV to /dev/null and request a backup, the database
server bypasses the backup but still updates the dbspaces with the new
backup time stamps.

--
Clive

--
This message has been scanned for viruses and
dangerous content by OpenProtect(http://www.openprotect.com), and is
believed to be clean.

_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.