Hello Group,
Is there a disadvantage to OPEN all files when a user logs in? Would
there be a relationship or formula based on the system memory?

Thanks,
-Peter G.

Although I am sure there are many reasons not to do this...

At least in Pick, an OPEN is nothing more than reading the MD item for
modulo and separation. Not like other systems which actually open an
I/O channel and start using up limited resources.

I have seen some packages that have a large COMMON that opens quite a
few files.



On Jan 29, 5:58*am, mdsi2000 <mdsi2... (AT) yahoo (DOT) com> wrote:

Just like the READV thread, this will depend on what system you are
using.

D3 runs everything within its own file system so you don't really open
anything at all, just pointers into the composite file system. One of
the downsides of this is that you lose the ability to use o/s
permissions to control access rights.

In our QM product, we would strongly recommend opening all main files
just once up front, putting the file variables in a common block.
Opening a file at the o/s level is relatively costly so it needs to be
avoided if possible. We recognise that applications often include the
potentially nasty situation of having a subroutine that is called in a
loop, opening files each time, so we maintain a cache of recently
closed files, not actually closing them at all. This gives a useful
performance improvement in such applications. The cache is
automatically flushed when necessary (e.g. DELETE.FILE).

Of course, you may have so many files that you hit the o/s limit on
the number of files that you can open at one time. QM will manage this
totally automatically, giving the developer the illusion that there is
no limit when we are actually quietly closing and reopening files
internaly on a least recently accessed basis.

UniVerse also works best with file variables in a common block. It too
has a "rotating file pool" mechanism though it has to be configured
manually (MFILES). As far as I am aware, it does not have a file
cache.

I believe that Unidata is much like UniVerse in this respect so the
same advantages should apply.


Martin Phillips, Ladybridge Systems.

I think this is application-specific and recommend against
over-generalization.

I'd think the rule(s) of thumb is/are:
1) Open what you need when you need it,
2) but open what you know you are going to need up-front.

So there are no absolutes and no formulas, it's very subjective.

An application is in an unexpected state when it starts processing but
somewhere mid-way tries to open a file and fails. That shouldn't be
allowed to happen, which is why a lot of developers open everything
they need at the top of a program rather than on the spot. Similarly,
it may help to know that the CustomerMaster file can't be opened at
user login rather than after they've already started their day and
they happen to use a program that requires that file. Something
important is wrong with the app and they probably shouldn't get
started if something that bad has happened. Therefore - open this
important file at login.

I believe it's a good idea to localize all opens, rather than opening
as you go. Many applications these days are being enhanced with
non-persistent clients, web service front-ends, etc. One user/client
may need different files than another. You don't want to allow a
remote user, perhaps a "headless" process to start executing code only
to find a file inaccessible later. So here again it's best to open
all necessary files up front, but not more than what is necessary.

Another point: When you say "all files", note that most apps have
hundreds of files, many of which aren't used every day by all users.
That's way too much baggage to carry around in memory if you don't
need it.

To follow-on to Doug about Open being "nothing more" than reading an
MD item: Most people don't know this but in D3 you don't actually
need file pointers or Q-pointers in your MD in order to open a file.
D3 allows for "global cataloging" of files as well as programs. So
for example you can tell your system to always try to open a file from
account Master if it doesn't find a file reference in the current
account. Most of us know the hassles of multi-account development
where we need to litter a system with q-pointers to access data we
commonly need, and utilities like AccuTerm or mv.NET where you create
q-pointers in every application account. Well, D3 can operate like
jBase in this regard, where the DBMS reads up the environment Path to
find files. I don't recall the verb off-hand - and I don't think it's
documented.

T

Douglas Tatelman wrote:

On Jan 29, 9:49*am, Douglas Tatelman <doug... (AT) pickteam (DOT) com> wrote:
When I was doing consulting to McGraw-Hill on a Sequoia which has
morphed into mvEnterprise, they would open all of their files when the
user initially logged on and put the pointers into named common so
that they were available throughout the session. The reason for this
was performance - they only had to open the file once.

Eugene

On Jan 29, 8:56*am, Tony Gravagno
The verb is SET PATH= with each account separated by, I believe,
commas

Dale

"Dale" <dbenedict99 (AT) gmail (DOT) com> wrote

space after Path - with virtually no parsing, testing, or validation.
semicolons, actually, but that was c lose.

BobJ

On Jan 30, 5:18*am, "RJ" <nob... (AT) nowhere (DOT) com> wrote:
Ok Guys,
My original question was asked get opinions. I was thinking that if I
redesign a MV package to be all SUBROUTINEs modular, I would be
opening the same files left and right. This may not be noticed on a
system with 100 users using terminals. But if I use those same
SUBROUTINEs in a background process or an overnight process with
millions of records, then it will have a big impact.

After reading these opinions, I'm leaning more that OPENing all files
is the best solution. Hardware speed is becoming less of an issue
these days. Thanks for the opinions and I'll report any pit falls that
may arise.

-Peter G.

Bob, Dale was right on! The D3 verb is indeed SET where the shell
variable to set is PATH. I believe I learned about this when I was
writing my blog "Lonely D3 Shell Variables", lamenting that most
people never know about them or use them:
nospamNebula-RnD.com/blog/tech/mv/2008/03/d3shell1.html

The D3 Reference Manual (v7.5.2) doesn't include the SET verb in the
searchable index, nor are standard environment variables found in the
index. One needs to look under "Terminal Control Language" > "TCL
Commands" > "set Command".

Thanks Dale!
T

"RJ" wrote:

Bob replied

Hi Peter,

I think it is a myth that file opens are expensive. Try this:

:ct bp open.test

open.test
001 t = time()
002 for n = 1 to 10000
003 open "sb.temp" to sb.temp else stop 201,"sb.temp"
004 next n
005 print time()-t

On D3 or UV, you can open a file 10,000 times in less than a second.

I would argue that:

A) If your program is slow it's not because you are (re-) opening the
same files too many times, and

B) whatever the minuscule speed advantage you may gain from either
opening all files at once, or caching the opens in common, you will
loose more in increased complexity and opportunity for confusion, and

C) if you implement your "open all files once" scheme via a standard
subroutine call, the CALL itself is probably more expensive than the
OPEN.

That being the case, Tony's rule-of-thumb:
.... is the best approach.

Just my opinion.

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006




On Jan 29, 5:58*am, mdsi2000 <mdsi2... (AT) yahoo (DOT) com> wrote: