Many moons ago, from a source I cannot remember, I heard that sizing
(MOD) of the dictionary for Basic Program files can possibly effect
efficiency of the system. Since most files do not require large
dictionaries, I am in the habit of creating a file with dict mod =1
(e.g. CREATE-FILE MYFILE 1 29). However, because object code resides
in the dict of the program file, I am also in the habit of making my
dictionary equivalent to the size of the data section, when creating
"BP" files (CREATE-FILE MY.BP 101 101).

I have asked REALITY tech support for their opinion, and they confirm
that (at least for their product) that theory is correct, giving the
dict more room to "efficiently" store the object code, with fewer
chances of becoming fragmented, thus possibly causing corruption of
code.

Is this theory consistent across other MV products?

Jim Cronin
Kittery Trading Post

"JJCSR" wrote
item's frames based on the item id. If the item is bigger than a frame,
{or is that bigger than a group of frames?} then just the first part is
in the space you preallocated with your CREATE-FILE, and then
the rest of the item is in frames pulled from your POVF big ol' heap.

Pointer-files are a special case, istr, in that the id hashes to a stub
header that then links to the bulk of the item, said bulk being again
from POVF and is, if possible, in contiguous frames.
At least that's how it was way back when, if you had a DC-type file.
Today I see dict dm,pointer-file pointer-file, has just a "D" on a1,
and has "PF" on a12 ...

I'd suspect compiled programs are essentially treated the same,
since they would benefit from being in contiguous frames.

So I'd think you'd care more about the __number__ of items
expected than their overall size when planning file sizes, but
maybe give bigger items up to four {?} 'votes' because they
suck up the whole group of frames they're in, instead of just
one frame of that group.

JJCSR wrote:
In D3 the object code is pointed to by a small item in the dict that's
about 50 characters. So in a single 2k frame you can get a few
hundred object pointers. When someone requests a program there will
be a bit of scanning through that frame but it's probably already in
memory, so that should be painless. Because of these points, I'll
generally have a dict of 1 frame for small program files and 3 for
large ones.

With the D3 "K" option on Compile, you can also core-lock programs,
eliminating the search from MD to Dict to Object.

In D3 you can also change frame sizes. I wouldn't for this purpose,
but with 4K, 16K, and 32K frames, you can cause a single disk read to
pull in thousands of the object pointers. (IIRC FSI frame size is 4k
now anyway.)

HTH
T

On 3/08/2011 4:16 p.m., JJCSR wrote:
As has already been answered, this would appear to be true across the
PICK style MV products. But, in general, it isn't the case for the
Information style products such as UniVerse, UniData, and OpenQM.

These products tend to use directory files to store their basic source,
and their object code goes either into an object directory (BP.O on
UniVerse, BP.OUT on OpenQM) or into the main source folder with a
slightly different name (UniData: Program TEST, Object _TEST).

So, for Information style products, there are no file sizing issues, and
typically no dictionary for your source files.

Regards,

Brian Speirs

On Aug 3, 8:16*am, JJCSR <JCro... (AT) ktp (DOT) com> wrote:
Can you confirm with a DUMP that the object code *actually* lives in
the dictionary on your Reality ?

Perhaps I'm high on drugs but R83 creations all stored just an object
pointer in the Dict, that pointed at the *contiguous* frames where
that particular piece of object code lived.

No fragmentation is possible. The frames are not groups, they are
contiguous blobs not affected by any sort of sizing at all. Just
pulled off the overflow table based on the needed size. No extra
blank frames at the end either. No need for link pointers since they
are contiguous, just a compiled size at the front, which is why they
were limited to 16K or 32K because the "size" field was that small.
If code is recompiled and it's *bigger* than the existing space, it
just grabs a new *contiguous* blob and throws the old blob into
garbage collection.

Personally if your tech guy is right (and I'm a suspicious mole) I'd
be surprised that the Reality coders didn't wake up to this better
method 20 years ago.

W.J.

On Aug 3, 11:16*am, JJCSR <JCro... (AT) ktp (DOT) com> wrote:
According to Reality folks, and I quote: "The Dictionary (just another
data section of the file) can become heavily fragmented and
inefficient if it is not sized correctly". Further, I noticed that,
following a backup/restore, all dicts of program files are resized to
"fit the need". Thus, a 1-mod dict on a "BP" file coming from D3 was
resized to over 100 on Reality, due to the number of programs in that
file.

Thanks for all of the responses.

Jim Cronin
Kittery Trading Post

JJCSR wrote:

Reality stores platform-independent object code in the data file, but
only platform-specific object goes in the dict. If you're not using
the R option on compiles then your dict should be empty. If you are
using the R option, and I believe you should if you're an end-user,
sure you'll have dict items, but Reality has small indirect items just
like D3. I don't know if object code is automatically pushed out to
indirect items, but it sounds like that's not he case, which would
explain the need for a large modulo.

OK, cracking the manual but not firing up my DBMS system yet:
If you use environment settings or SET-OPTION and set BASIC options
EXEC.OBJ and DEL.OBJ then you will get optimized object in the dict,
something similar to D3 flashed modules. Are all object modules in
Reality indirect? Perhaps someone can tell us here.

Is there a problem with a mod of 100 for a dict? Not at all. Disk is
cheap and a larger mod spreads out items to ensure reads hit what they
need and there is very little scanning for object modules. In a dict
of mod 100 I'm guessing you have a couple hundred object modules. If
so, then sure, your mod 1 was grossly understated for a production
system. If not then personally I'd like to see a smaller number
depending on whether those items are inline or not.

It's nice to understand what's required vs what's optional, and how
efficient various systems are in this area. Thanks for the
discussion, Jim.

T

On 8/15/11 12:23 PM, Tony Gravagno wrote:
Same problem as modulo 100 for a data file:
[Conventional wisdom says] modulo should be a
prime number.

Might be interesting to see how the record IDs
hash into that modulo 100 file.

--
frosty

On Aug 15, 5:12*pm, frosty <fros... (AT) bogus (DOT) invalid> wrote:
Perhaps there is some misunderstanding about the "mod 100" that I
referenced, earlier in this thread. If you re-read my post, I
actually stated, "...resized to OVER 100"; I did not say the mod WAS
100. As for the number of items in the dict of this particular BP
file, there are 1325 programs' objects stored there. Using a
suggestion from the Reality techs, I did two backups/restores, over a
period of a couple of weeks. And, as Reality folks had told me, the
system behaved much more efficiently. With each restore, the files
were automatically resized, including dictionaries. The Reality
restore procedure selected the new mods - I had no input. And, my
backup-time dropped from over 1 hour to 12-15 minutes. It just
seemed to make sense that the space allocated to the dictionaries of
program files would help determine, to some extent, how efficiently
the system would perform.

Now, after having witnessed how sizing of a dict for a program file
can help in system-efficiency, I wonder to myself if perhaps the "mod
1" from the D3 program files had anything to do with the "corrupt
flashed memory" problems that persisted (leading to my exit from the
list of D3 clients), un-repaired, in my waning months as a RD/TL
client. Could it be that I was experiencing difficulties, having not
only compiled object, but flashed-compiled object, stored in a mod of
1, for large numbers of programs? Is it possible that the objects
were, in fact, too fragmented, as a result of the small space
allocated for the dict? The file I referenced, above (1325 items in
a BP file) was only one of many, many such-sized dictionaries of
program files; files that were extremely active on a day-to-day
basis. I guess I'll just have to consider that a "rhetorical"
question - one that I will never know the answer to.

Jim Cronin
Kittery Trading Post

Jim, I don't think the dict mod was a factor in the Flash issues.
There is initial access time to the object, and then there is the
runtime workspace where I believe your system experienced the
corruption. It's the difference between not being able to execute
code and executing code that does something bad. Like I said, you can
get a Lot of object pointers into a single D3 frame, and once such a
frame is pulled into memory it's not likely to get paged out again,
unless you're really short on memory _and_ that frame is not used
again for an extended period. I suspect Reality memory management is
similar.

T


JJCSR wrote: