About 3 years ago, when we setup our D3 database on Linux (we moved
from SCO), the system ran extremely slow. The slowness was alleviated
when we turned update protection off. We've run that way since. Today
we crashed and luckily nothing was damaged. The Raining Data tech said
the reason update protection caused the system to run slow was that
there must have been some poorly sized files.

So in order to keep a sound database, I need to turn update protection
on, but then resizing becomes critical. If I do the F-RESIZE backup
and restore, I'll need exclusive access. If I resize online, it will
put a heavy load on the system and I'll need to do it every day.

What would you do?

For those that don't know, here's what the manual shows:
update-prot
toggles or displays the status of the file update protection scheme.
When enabled, this feature globally protects all item updates so that a
power-off condition will not cause a GFE.
By default the update protection is on.

If the user turns update-protection off, and later turns it back on,
the user MUST use a "FLUSH" command afterwards to be sure that all
previous updates are written to disk. If update protection is desired
for selected files only, the user may disable the global update
protection flag (with "UPDATE-PROT-OFF"), and enable the protection on
the selected files by changing attribute 1 of the D-pointers of those
files to "DU". When all desired files have been changed to a "DU" type,
the user MUST use a "FLUSH" command to write all previous updates to
the disk.

If update protection is desired for most of the system, but the user
wishes to disable it on certain files, the user can change attribute 1
of the desired D-pointers to "DN". These files will not be protected
irrespective of the global update-protection status.

Syntax
update-prot {(option}

Options
f Toggles update protection off.
n Toggles update protection on.


Thanks,
Danny

The File Update Protecton scheme is faily simple, and it does ues a lot of
system resources.

Here's what happens. You update a record. Typically, the system will
remove the old record from the group, pull-up the group over the removed
item, then adds the new item to the end. Usually, this required no
additioal overflow, unless the item got considerably larger. While this is
happening, something bad could happen, like a power failure. The group is
half updated and everything in memory is lost.

With update protection, all data beyond the base frame is put into a totally
new overflow string. When the data is "right", the system changes the
forward link of that base frame to point to the new group. Theoretically,
up to that last instant, all was well. The actual update boils down to that
one link change. The old overflow string is returned to overflow table.

If the file is sized correctly (no overflow), update protection buys you
nothing. If there are lots of frames of overflow, you are putting a real
strain on the overflow management subsystem with each update.

The best approach is to keep files that change regularly sized properly.
Static files that seldom change can be badly sized without undue impact.

Mark Brown


"ddspell-m3" <ddspell (AT) yahoo (DOT) com> wrote

I think update-prot was an issue circa 2002 D3 7.2.1, related to a
problem with Red Hat, since fixed. I think the fix involved using a raw
partition for the D3 disk space (you did set up D3 in a raw partition,
didn't you?).

update-prot does incur some overhead, but nothing like the
near-crippling effect that many encountered in 7.2.1.

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

Scott,


I believe it is in a raw partition. How can I know for sure?

What I know is that Linux does not recognize the partition, so I think
it is purely a D3 partition.


Regards,
Danny

Scott Ballinger wrote:

Danny,

It looks like you likely do have the update-prot problem. RH 7.1 does
not support "raw" partitions, I think you need RH9 or greater. Although
your D3 partition(s) do not have a Linux filesystem, they are not "raw"
partitions. "raw" partitions are initialized (and re-initialized after
booting) via the linux (or Red Hat specific?) "raw" command. My limited
undertanding is that a "raw" partition can be accessed by the O/S or
kernel more efficiently than a regular partition.

So, I think you should probably stick to update-prot-off, or upgrade
your system to 7.4.0 (RH 9) or 7.4.2 (RHAS 3 or 4).

/Scott

ddspell-m3 wrote:

The pains of update-protection are not limited to that D37.2/RH7.1
platform, though Scott is right that this platform was especially
problematic for reasons not related to this.

Note that Mark said if your files are properly sized then you don't
need update protection for those files. Files that are poorly sized
by definition have a lot of links into overflow, and therefore are in
the most need of the sort of protection that this provides.

No community should be without police department coverage, but some
areas need less coverage than others. Similarly, I don't think
update-protection is something we should globally turn off, because it
provides a valuable service when it's required.

I think there is another factor that determines whether a file is good
for protection even it it's highly fragmented. If a history file is
only updated with new items and you don't change or delete items, then
the frame update process only needs to change the last frame in the
chain. In this case protection buys you nothing and in fact it will
cause a lot of frame copying and shuffling of the overflow table that
is completely unnecessary - translate that as a major performance
issue for no reason at all. I believe the same protection is provided
for changing the forward link of the last frame of a group whether or
not update protection is on. I've never seen that code and probably
wouldn't understand it if I did. Mark?

My advice would be this. Be vigilant about your primary files. Don't
protect history-only files. Do protect transaction files. You can
turn off protection for the system and DU the important ones that are
fragmented. Or, you can turn on protection for the system and DN the
files you don't care about. Whichever method you use, if you're
tweaking a lot of files I suggest you do it programmatically. Just
make informed decisions. If you can't resize files to avoid
fragmentation (this does happen occasionally) then you need to face
the decision of taking a performance hit on that file vs exposing that
file to GFE's. Approaching this on a file-by-file basis is much
better than turning off protection for all files on the system.

(The following is another one of those brain tickler comments where I
don't intend to be complete, just to make people aware that there's
more to be considered than the topic at hand.)
This sort of performance monitoring and tuning is a borderline
artform. There are few clear right/wrong answers. One thing to take
into consideration, for example, is that data is pulled into memory
and cached. You aren't always working from disk. As CPUs get faster,
we add more L2 Cache, and RAM gets larger and faster, these frame
shuffling operations get very fast and update protection gets less
painful - though certainly not painless. The real problem is not the
frame shuffling anymore but ensuring that the data is written to disk.
I don't care how effective your state management is, if the hardware
dies during a disk update you're in trouble. Another aspect of
performance monitoring is ensuring that memory is flushed fast enough,
but not too fast. The more data you hold in memory (even paged/swap)
the more you stand to lose if the system dies before a flush.
Combining this, D3 update protection is intended to fix the problem
where a single successful item update is dependent upon the writing of
a long chain of (perhaps) unrelated data. The fix is to make it so
that only one frame needs to get written at that critical moment in
order for everything else to be in sync. My caution here is that the
admin should look at other factors which affect those updates - the
term "update protection" can be unintentionally misleading.

(Hope that made sense)

Final note, Raining Data has D3 Performance sessions at Spectrum and
other shows occasionally. There is a lot of good info provided in
these sessions. Contact RD Support to find out when they might do
another one.

HTH
T

"Scott Ballinger" wrote:

One of my customers (an electrical supply company) set up a generator with a
complex set of switching gear. If a power failure lasted more than 5
minutes, the generator kicked on. After allowing another 5 minutes for the
generator to warm up or for the power to return, the entire facility
switched over to the generator. This was all done automatically. The cost?
Somewhere in the $15k range.

So, one day there really was a power failure and sure enough 10 minutes
later the facility was running on the generator. Unfortunately, some one
forgot to refill the propane tank after they had done substantial amounts of
testing and trial runs. 15 minutes later it was dark and all the systems
were down.

The moral of the story... (there are many to choose from or you can make up
your own)


"Glen B" <no$pamwebmaster@no$pamforallspec.com> wrote

Tony Gravagno wrote:
<snip>

I didn't know that update protection can be controlled on a
file-by-file basis. How is this done?

I thought it was only a global setting.


Thanks,
Danny

"ddspell-m3" rote:


To only protect certain files:
Turn off global protection with update-prot-off.
Then change the d-pointer of desired files to DU.

To turn off protection only for certain files:
Turn on global protection with update-prot-on.
Then change the d-pointer of desired files to DN.

This is in the reference manuals.
Your VAR gets paid to answer questions like this. If they decline the
invitation to earn their support fees, please pay me instead of them
and I'll be happy to provide more answers.