I'm hoping here that maybe someone could settle a debate that I'm having with my coworkers here about d3 FILE-SAVE versus AIX mksysb tapes.

We have a very large d3 (7.4.2) system here running on AIX 5.3

We have over 100GB on our system. We used to run full FILE-SAVE backups every night. Eventually those started taking longer and longer until we could no longer get file save done over night. If there is no activity on the system, a file-save plus verify takes 24 hours. But we have jobs that are scheduled to run at night so a file-save during the week probalby will take several days.

We tried incrementals, but those took just as long.

So we implemented transaction logging to tape and it's a nightmare....it slows down our end users with all the locks, but we keep it running during the week and do full file-saves on fridays.

Anyways, the purpose of this post is to find out whether it's possible to to a mksysb image backup at the UNIX level and have it restorable to a usable state as far as d3 database is concerned. I realize we would not have item/file restore capability, but we're just concerned about disaster recovery at this point. We could still do weekly pick file-saves.

I've been trying to tell people that I didn't believe that a unix level backup of the appropriate volume group would not result in a usable d3 database if it was restored. I think it has something to do with the pick partitions being raw partitions or something like that. At my previous job, we needed to replace a failing hard drive.......we elected to do a mksysb backup instead of a pick file-save due to the faster speed. After restoring it, all our pick database gone and we lost a whole days work.

So what's the consensus, can we backup the whole system with mksysb, restore it and have a usable d3 system, or do we HAVE to do pick file-save?

thanks
Chris

Well,

If this is art of a disaster recovery plan, why not just restore onto
the DR machine & check? ... there is a DR machine, isn't there?

Not that it helps with your current confg, but as well as using RAW
partitions, IIRC you can also configure D3 to use "real" file systems
(at least under D3/Linux) - Doug D is the "expert" in this field.

Also, I assume that with a system with a DB this large you have D3
under mantenance with RD - why not call & ask them?

<spectrum note>
One of the things that caught my eye @ Spectrum was the improvements
made to the backup processor by the Northgate/Reality crew - 30x speed
improvement, and the ability to take a "snapshot" backup from a running
system. Impressive stuff when you consider that your full backup would
likely come down to around a hour [assuming backup speeds were on par
initially]. This is an area I think RD need to address for their larger
AIX sites, otherwise they are likely to start to see a "drift" to
products like Reality
</spectrum note>

Ross Ferris
Stamina Software
Visage > Better by Design!

Goo'day, Ross...

On 22 Mar 2006 15:14:30 -0800, "Ross Ferris" <rossf (AT) stamina (DOT) com.au>
wrote:

<snip>
Didn't look too hard at the others, did you?

Although, to be brutally honest, OpenQM didn't release this until
Monday 20th.....

Yep! It is, very, on OpenQM.... even with no Vegemite, or as Martin
would probably want it, Marmite....<g>
<snip>

Regards,

Bruce Nichol
Talon Computer Services
ALBURY NSW Australia

http://www.taloncs.com.au

If it ain't broke, fix it until it is....

Also backing up to Disk vs tape much faster, then you can backup the backup
, after the fact - works quite well
"chrismac2" <chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au> wrote in
message news:chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au...

Chris;

It's been a while since I've run D3 on aix but yes, you're right. If you
have D3 in a raw partition then a mksysb won't back it up. In fact, I
believe it will only do the root volume group.

I know aix boxes with that kind of capacity aren't cheap - but if you have
that much data you really need to make sure of your disaster recovery
methods.

hth
Colin Alfke
Calgary Canada


"chrismac2" <chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au> wrote in
message news:chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au...

"chrismac2" <chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au> wrote

A hundred mb of data shouldn't take that long to backup, even to tape. I seem to recall some questions arising regarding older AIX systems in which the FILE-SAVE attempted to span more than one reel and stalled and eventually timed out or something. If you experienced that, it possibly could have been a reflection of a problem inherent to your version of AIX, Your IBM hardware configuration, or your file sizing.

I wouldn't bother asking IBM, unless you are under a service agreement, and even then, sadly, I am forced to admit that even that might not help you. As Jim suggested, you would do well to investigate backing up from one disk to another, and there quite a few options.

Right off the bat, I'd suspect that unless your file system is seriously badly keyed or the file sizing is atrocious, it would be advisable for you to investigate directing your FILE-SAVE into a virtual disk file over a TCP/IP connection to a moderate PC running linux or NT. It would allow you the ability to maintain redundant copies of your backups and increase the speed of running and verifying the integrity of the backup.

I would first check your file sizing, if you need help I can try to help you. You can reply in the group,

Regards.

Thank you all for your responses....

From my observations, it doesn't seem to be the writing to tape that is causing the slowdowns. We have a fast scsi LTO drive and the majority of the time during the FILE-SAVe the tape is idle waiting for it's buffer to fill up and write the data to tape. The bottleneck really seems to be on the pick side reading the data.

Just to give you an idea of how big our system and files are....again we have over 100GB of data. We have 6 Master Dictionaries, each of those have 3 main files that are huge. We're talking over 3.8 million items in a file that has a modulo of 921,779 for each one. In addition to thousands of other smaller files for a total of 33,824 files. At peak usage during the day we have up to 350 telnet connections to this box

Just doing a dummy file-save where nothing gets written to tape, only shaves off about an hour off the total time.


I'm just wondering if we've gotten too big for what PICK was really designed to support.

Would migrating to Universe on a Windows box be advantageous? With Universe, do you need to do an Universe-based file-save? or is it done just at the OS level like Veritas backup on Windows?

I believe there's a simple answer here, which will also explain why you lost your other database too !
D3 AIX writes its' database to RAW Logical Volumes and NOT to a filesystem (there is no AIX/D3 filesystem) ~ mksysb writes its' output based on input files from an AIX filesystem in backup format (use restore to restore) for AIX 4.3/5.1/5.2/5.3 and tar format for pre 4.3 systems.
Even doing savevg, cpio or tar won't help you because these all need to access regular *nix files in a filesystem.

You might want to try using Virtual Tapes (I think that's the terminology) in D3 to write data to regular AIX files so you can mksysb, or savevg, or cpio or tar the output files (and compress too), which will be restorable back into D3 (famous last words there ~ don't sue me when it fails though).

Help that gives some insight, Brian.

chrismac2 wrote:

"Concerned_Netizen" <Concerned_netizen (AT) spammenot (DOT) com> wrote


"chrismac2" <chrismac2.253d8z (AT) no-mx (DOT) forums.yourdomain.com.au> wrote

<Snip>

OH! my apologies, A hundred gigabytes is a lot of data. Still, the time involved in writing to even the fastest tape drives won't compare with a disk to disk transfer. I'm not sure what kind of AIX hardware you are using, or even if you can add on another disk, even one that is external and removable, but, are you sure you want to even consider migrating to another platform at this point?

Remember, it's the modulo of the file that makes access fast from a d3 standpoint. And if you have only a small number of MD's, I'd suspect you might want to check out not only your file sizing, but your MD sizing.

Just my two cents. Good luck!

Here's a potential novel way you could maybe consider to do this by using AIX software mirroring and using 3 mirror copies on all your D3/AIX logical volumes:
See below for an excerpt from an IBM Redbook using the splitlvcopy command on how it splits off a mirrored copy complete with contents and then renames the Logical Volume for you.
D3 can have more than 1 invocation of databases co-residing on the same box, so once you have successfully split off your database copy, you could startup a single user D3 version to co-exist and "save" your data.
Before anyone starts with all the "noise" about the database over multiple LV's (all with 3 copies) and inconsistency of the multi LV contents, I am almost sure that the primary D3 application will probably have to be quiesed (shutdown even) in order to ensure memory and disk buffer flushing, even tho D3 disk I/O goes directly physical because it's RAW I/O.

This was just an idea (way out there) that came to me, and I thought I'd just bounce it off the wall.
If you did contemplate something of this nature, I am sure you would have to be extremely rigid in how the process was constructed and driven.
This should work for AIX 5.1/5.2/5.3, but I think not for 4.3 (that's even if the concept was even workable in the first place)

I guess I'm using my imagination far too much again, Brian.
<quote>
Excerpt from IBM Redbook sg246187 ~ IBM Certification Study Guide - pSeries HACMP for AIX

Split-mirror backups
No file system can be safely backed up while update activity is occurring. If you
are going to have any assurance as to which updates are on the backup and
which updates are not, you need to be able to mark exactly where the backup
was made. Therefore, it may be difficult to do a good backup on systems that
have applications or data that must be online continuously or offline for only a
very short time. In some installations, the time required to do a full backup to an
archival device, or even to another, might be longer than the availability
requirements of the application will allow it to be offline. The mirroring capability
of the AIX Logical Volume Manager (LVM) can be used to address this issue.

How to do a split-mirror backup
This same procedure can be used with just one mirrored copy of a logical
volume. If you remove a mirrored copy of a logical volume (and file system), and
then create a new logical volume (and file system) using the allocation map from
that mirrored copy, your new logical volume and file system will contain the same
data as was in the original logical volume.

Now, you can mount this new file system (read-only is recommended), back it
up, and you are really backing up a mirrored copy of the data in the original file
system, as it was when we removed the mirror copy. Since this file system,
created from the mirror copy, is mounted read-only, no inconsistency in the file
system from the point at which you removed the mirror originally is created
during the backup. After that, you can delete the new file system to release the
physical partitions back to the free pool. Finally, you can add and synchronize a
mirror back onto the original file system, and you are back to a mirrored mode of
operation, with fully updated data.
The splitlvcopy command of AIX does much of the work required to implement
this solution.
We can summarize the steps to do a split-mirror backup of a file system as
follows:
1. Use the lsvg -l VGNAME command to take note of the logical volume name
that contains the file system you want to back up.
2. Stop any application using the file system and unmount the file system.
3. Use the splitlvcopy command to break off one mirror of the logical volume,
and create a new logical volume with its contents. For example, if the existing
logical volume is named fslv, the command would be splitlvcopy -y newlv
fslv.
4. It is important to note that there is now one less mirror copy available to the
user in fslv.
5. Remount the file system and restart the application that was using it.
6. You can see that the application and data are offline for only a very short time.
7. Create a file system on your new logical volume and mount it read-only. This
is to ensure that no update activity will occur in the new logical volume, and
the consistency of the file system is guaranteed.
8. Perform the backup on the new file system by any means desired, such as
backup, tar, cpio, and pax.
9. After the backup is complete and verified, unmount and delete the new file
system and the logical volume you used for it.
10.Use the mklvcopy command to add back the logical volume copy you
previously split off to the fslv logical volume.
11.Resynchronize the logical volume.
Once the mirror copy has been recreated on the logical volume, the syncvg
command will resynchronize all physical partitions in the new copy, including any
updates that have occurred on the original copy during the backup process.
<unquote>

Brian Coles wrote: