Hi All

I'm new to D3 and am after some system maintenance and peformance
monitoring tips or guidelines... areas to check/poke around in etc

I've picked up a customer running D3/Linux, not sure of the versions at
this point, who's concerned about performance. They haven't done much
in the way of system maintenance themselves so we're probably looking
at file sizing as an obvious starting point, but beyond that what is
there on D3???

Are there logs to check, particular processes to monitor, config
settings to verify etc etc

Any relevant advice would be much appreciated.

Rgds
Juss

Juss wrote:

Yes. A very important point for performance.


LIST-ERRORS can give you some insights into problems,though some also
like to use this file to log general info, such as file-save start and
stop times. It will also show forced logoffs.

I clear this file this file every so often if there no entries that
require attention.

Another setting sometimes worth investigating is

set-runaway-limit

This limit prevents a process from using up all available overflow.

I believe the default is 7000 frames, but you can hit this when
processing large files. We set this higher on some systems.

I'm sure there are others that people from the list will add.

Good luck,


--
Kevin Powick

If there hasn't been much maintenance done, I would think that a file
restore would be a good thing if there hasn't been one done in some
time.

Kevin mentioned LIST-ERRORS. You might want to take a look at
LIST-RUNTIME-ERRORS as well. This shows all runtime errors generated
by programs.

If you want to look at file details you can use the ISTAT filename (S
command.

"Tracy Raines" wrote:

WARNING: Before attempting a restore with a new client be absolutely
sure they have good backups! Last think you want to do is inherit a
new client with bad hardware, destroy their hard drive, then find out
the backup tapes are no good. Dummy restores are not entirely valid.
Restore their system to another system or HD and if you're positive
the tape drive and storage media are in good shape then attempt a
local restore. You can even save to a pseudo-floppy and restore that
to some other system. But DO NOT destroy end-user data without being
absolutely positive that you can recover it.

Also be sure that you know exactly what files have been set to resize
before you attempt a restore. If they don't have enough disk the
restore will die - you'll need to re-restore using the original
settings or add another partition to D3 (if you have the space) to
accommodate the new data.

Another thing - after you restore, depending on the release and what
you do in there, you may need to re-activate D3 and/or other
third-party software. Be sure the end-user is covered by a support
agreement or you will take them down and won't be able to get them
back up without a loss of time and perhaps some fees.

Just note from the above that you should never make assumptions about
what is going to happen with these operations.


....only if the RUNTIME-ERRORS file has been created in DM. If not
then don't worry about it. Many people will create the file and then
forget about it. If there are chronic errors this file can accumulate
lots of data quickly and create a perfomance issue of its own.


Ensure dictionaries for common files are sized properly. Many people
think the dict size doesn't matter (LOL, sorry) but it's critical for
many applications.

If they're running FlashCONNECT you should check the logfiles in the
WWW account. If they're running Apache you might want to check
access/error logs as well.

Look into FlashBASIC optimized compilation. You can get a lot of
extra performance if you flash-compile their code. Be sure the pick0
configuration file has memory allocated for BASIC. As above, don't
make any global changes here unless you're prepared to immediately
back-out your changes. Some sites have had problems with FlashBASIC
and every release has a fix for reported issues. On an older system
you're likely to create new problems in a sincere attempt to improve
the situation with a simple Flash-compile. I tend to take the "no
pain no gain" approach but the final decision should be left to the
end-user management.

Swap space in Linux should be D3 memory x 2. Check the Install Guide
for more info.

Keep an eye on the BASIC lock table. If end-users are contending for
records it will be perceived as a performance problem. There are
various ways to deal with this but more as an application issue.

Related to hardware:
- Ensure that neither D3 nor Linux are getting choked for memory.
- Check CPU usage and watch for D3 or Linux processes that are
consuming more than their fair share.
- If the site is running a lot of network traffic, you might be able
to improve throughput just by installing a new NIC in the server.
- If the server is older it might be running slow hard drives. An
upgrade from IDE-33 to SATA for example will help dramatically.

The Advanced Pick Reference Manual and D3 Ref have other important
tips. See this page as an example:
<http://www.rainingdata.com/support/techpubs2/1806head.html>

D3 Linux v7.3 is significantly faster than 7.2 due to some issues
between D3 7.2 and RH 7.2 (?). Upgrading the the current release
should help if the site is that far back.

For more performance-related tips, contact RD Support. They might
have an updated tech announcement and they periodically conduct D3
Performance Tuning sessions.


I hope that helps.
Tony
TG@ removethisNebula-RnD.com

Juss,

WHAT you need to check will vary somewhat on the version of Linux & D3
client is running.

Combining the warnings of Tony & suggestions of others, a fiull
backup/restore cycle can be useful --> but go & set file resizing
parameters first, otherwise the exercise may be of limited use !

Before doing this, check for any stray dx/dy type accounts & files -->
these can always be nasty, as they don't save (or restore) too well !!

As Tony suggests, check new modulo's BEFORE embarking on this,
especially as there may be work files involved ... you don't want to
reduce ANY modulo as a general rule, and be VERY CAREFUL of
FILE-OF-FILES --> if this is poorly sized you take a massive hit for
virtually everything

Depending on the app, another file that may "need attention" is
POINTER-FILE - tends to be a "collection point" for garbage on many
systems.

I don't believe the runaway-limit suggestions are going to be useful,
UNLESS you have processes that are currently stopping with "potential
runaway process" type warning messages.

Likewise, RUNTIME-ERRORS probably will not yield any clues in terms of
performance !

Commands like BUFFERS, and "top" from Linux, can give you some
indication of things going off the rails. I've also seen "general
housekeeping" prove an issue, with 10 years worth of transactions kept
as "current", because no-one had moved transactions to history files in
the last decade (now starting to get very app specific)

Then, there are always "physical" limitations of hardware. Last week I
had a client loose a hard disk (just what you need around end of year -
with no RAID, and a windows backup process that was "flawed" ...
anyway, they were running on a 200Mhz machine with old IDE drives.

If no-one has done "anything" with the system for a while, chances are
it may be due for replacement, in which case a 3Ghz CPU matched with
3Gb RAM (and D3 allocated 2.55Gb RAM) and mated with 15K SCSI raid can
work wonders :-)

Depending on WHERE you are, may also be able to suggest people locally
that can help, and TData provide EXCELLENT support to the Australian D3
VAR community

Ross Ferris
Stamina Software
Visage - Better by Design!

Juss wrote:

"Ross Ferris" wrote:
I agree with all other comments in this post but this one. I've had
recent experience with apps that have so many errors during the course
of the business day that the RUNTIME-ERRORS file itself gets very
fragmented, and every update results in delays as the system chases
frames into overflow. Identify your errors, fix them, then delete the
file. If you suspect more errors, recreate the file with a mod of
something like 101 and wait for more errors to get logged.

T

Thanks to all for the advice...

It turned out to be D3 on W2K so I completed resizing last night using
NT_RESIZE-MENU.

All went well... If there are still performance issues i'm sure all
else mentioned will be useful.

Juss

Depending on the version of D3/NT, and also if accounts/files are in
the FSI or the VME, you still may end up with "problems" if you ever
have to restore from a backup.

Once upon a time, files resized in the FSI didn't restore correctly
..... easy enough to test with a small test acount



Juss wrote:

Agreed, and the older versions of the NT resizer didn't calculate new
modulos well for certain data structures - proper resizing is
dependent on the key patterns as well as the data. I have no idea how
good the resizer is in D3 7.4, I simply haven't used it in a couple
years.

In D3NT you have the "luxury" of variable sized frames, from 1k to
128k.

And... you can convert some files so that all items are pointer items.
This allows you to compact many large items into a much smaller modulo
because all data other than the key is in overflow. Combining these,
you can use larger frames with pointer items. This would be good for
large numbers of infrequently used data where segmented keys contain
ample data for your normal sort/sselect processes.

In D3 Linux, I believe the frame size was increased from 2k to 4k.
Most people don't know this. If you restore from an older Linux box
or D3NT and don't tell the system that you came from a different
modulo, all files will be created with the same modulo and all items
will hash into the same groups. This results in some files consuming
twice as many frames as they need. As I think about this, this may
also explain the sudden performance benefit that some sites see when
they port to Linux, and they give Linux the credit rather than
recognizing that they now have twice as much disk allocated to their
data as they used to, which results in less frame faulting, and that's
perceived as improved performance. Anyway, the benefit of larger
frames, IIRC, comes from host OS disk access methods and the way
memory is paged. It's usually beneficial if the DBMS requests chunks
of the same block size that the disk/OS will provide. This way, for
example, we don't compel the disk to read a 4k chunk and then we only
use 2k of it and discard the rest. We've seen this progression of
frame sizes increasing as disk technology improves: 512 bytes, 1k, 2k,
now 4k ... and as mentioned above for NT it's up to 128k, though I'm
not sure how disk access is handled for FSI files.

Head spinning yet?

T

"Ross Ferris" <rossf (AT) stamina (DOT) com.au> wrote: