I have an issue with BTrieve (Pervasive 2000i) that I have narrowed
down to either the BTrieve engine or the Win32 BTrieve client.

A bit of history:
Win2K system (Schedule system)
Novell Server v6
Object Rexx programming language

Schedule system runs a single program we'll call ProgramA. This
program wakes up every 30 seconds to check the records in a BTrieve
file to see if it has anything scheduled for the current day/time. If
it does, it calls ProgramB. ProgramB will do extensive reading on
multiple BTrieve files.

What we are witnessing is that over a period of time, all ProgramB's
called by ProgramA get slower and slower. This morning I used a
utility to check the memory in use by the thread ProgramA was running
in. It was in excess of 132MB. A simple close of the window ProgramA
was running in and reopening the window resulted in a memory usage of
3.2 MB.

Trust me, I have done extensive testing and everything points to
BTrieve. I set up a test system and have been running the same
ProgramB on it for several weeks. After assuring myself that I could
recreate the problem, I began changing ProgramB trying to eliminate
any possible causes of the memory leak. BTW: I copied my databases
used in ProgramB to a seperate server folder so I would be testing on
the same data on every run.
Here's what I tried:
1) Removed all ActiveX components from ProgramB (this was primarily a
component to print data to a PDF file). Still witnessed memory leak.

2) Removed all ::Requires statements from the code. A Requires
statement in Object Rexx simply allows the program to access classes
and methods in an external routine. Still witnessed memory leak

3) Exported all data from the BTrieve files into delimeted text files
- modified program to read text files vs. BTrieve files - removed all
BTrieve calls. Memory leak went away!!

So, I said all that to say this. "Something" with BTrieve is causing
the memory leak. It could be any one of the following:
1) The BTrieve API calls developed for Object Rexx, but in extensive
conversation with the author of these API calls, he assures me that
practically all memory usage is static with very little dynamic
allocation.
2) The Win32 Btrieve client
3) The NetWare BTrieve Engine, I have my doubts about it being the
engine since we also have several OS/2 systems accessing the same data
and never have a slow down or memory issue with them.

My next experiment will be to load an old work station version of
BTrieve that I still have laying around and run everything local.

If anyone has any ideas or suggestions, I'm all ears. (I don't have
much hair left).

Lee Peedin

Lee Peedin wrote:
We have had problems with P2000 and P2000i systems running out of memory
on Windows based systems.

It was with two apps that were contantly opening and closing tables and
databases. After a period of time, they would crash with out of memory
errors.

Our issue was apparently a timing issue, in that getting new handles,
etc. too quickly caused issues with the engine and with the method we
used to access the db (PDAC).

We got fixes for both the engine and our access API (PDAC), and it help,
but it didn't go away. We restructured one program a little to change
its file handling (so it wasn't doing as much opening and reopening),
and that helped more.

Our other apps running on the same db wouldn't have any issues.

Paul

Since the client is on one machine and the engine is on another you
should be able to discount the engine (3) as a source of problems.

While I have never heard of any kind of memory leak with the Btrieve
API client (which version by the way?) it is possible. But again I
have not seen this behavior even in very high volume environments.

Now the fact that you are looking for a memory leak and the developer
of the Rexx interface has said "very little dynamic allocation" leads
me to believe that they are more likely related. Because even a tiny
bit of dynamic memory allocation will build up over time if not
released. You may want to check with them to see when (what
triggering conditioins) they dynamically release the memory they
dynamically allocate. If he can't give you an answer that would
probably be it. If he does give you an answer and you are not
performing the triggering event, for example closing the session which
is different than closing all files, then it might just be a simple
change.

The problem is you are not actually making Btrieve API calls. You are
making Rexx driver calls which is hopefully being properly translated
to the Btrieve API. There may be a disconect in there somewhere.
Pehaps check the Pervasive monitor utility an see if file handles or
sessions are piling up on the server. If you are closing them and
they are not going away that can be a problem. Likewise opening and
closing files are the most "expensive" database operations you can do.
Normally just open them at the beginning of the application and close
them at the end and oh by the way close the session too with a
Stop(25) or Reset(28) call. If you really want to know what is
happening at the database level you can turn on database engine
tracing which traces all of the operations in / request and out /
response of the database engine. Be sure to turn tracing off when
done though.

You mention bypassing the Btrieve API, maybe bypass the Rexx supplied
"driver" if possible. The Btrieve API is just a DLL function call
that is well documented. Sof if Rexx allows you to call DLLs directly
that might be an option also.

Leonard

On Thu, 10 Feb 2005 01:21:03 GMT, Lee Peedin
<lpeedinREMOVE (AT) UPPERCASEnc (DOT) rr.com> wrote:

On Thu, 10 Feb 2005 14:42:03 GMT, Leonard <lharvey (AT) austin (DOT) rr.com>
wrote:

Leonard,
Thank you very much for your response.
Our Windows client is Pervasive.SQL 2000i Client v7.94

Our server engine is also at that leve. As far as we can tell this
was the latest available for v7.

We realize that there are newer versions available, but we're walking
on "thin ice" even with v7. We still have several OS/2 systems and I
believe the last BTrieve client for OS/2 was way back in v4. We've
been lucky that it will work with v7 and are not quite ready to take
the leap of faith and see if it will work with v8 and up.

I removed the client from my test system this morning and loaded an
old workstation version. The first test I ran was using this local
engine against local databases. The second test was using the local
engine against server side databases. BTW: exact same database, just
copied local. I did not see any increase in time with either of these
tests, but I "might" be seeing a tiny increase in memory utilization
on the process. I will continue to monitor this for appox. 24 hours -
this should help me narrow it down.

I used the Monitor utility and at the time I had 15 files and an equal
number of handles. This sounds very reasonable since there are no
less than 18 systems actively accessing BTrieve files on the server at
any given time.

An associate in Austria recently developed what he calls GCI for
Object Rexx (Generic Call Interface) which now allows Rexx to utilize
any function in a dll (even if the dll was not compiled with the Rexx
headers. I'll look into using this to directly access the BTrieve
dll, but with much caution. I want full assurance that his GCI is
thread safe. :-)

One more question, do you have any experience with any of the ActiveX
calls for BTrieve. All I've looked at always had examples of opening
a .ddf file. We have .ddf files but our databases are .dta files.
There may be numerous different .dta files that use the same .ddf

Lee



On Thu, 10 Feb 2005 14:42:03 GMT, Leonard <lharvey (AT) austin (DOT) rr.com>
wrote:

OK -- this may sound like a silly question, but WHY were you trying to
debug ProgramB when the memory for ProgramA seemed to be increasing?
Could it not also be that the parent process has a problem in
allocating memory and not freeing it up?

Just a thought...
BB

Lee Peedin wrote:

Bill,
Thanks for the reply. After giving this entire scenerio a bit more
thought and a whole lot of testings, I have a bit more information.

The method in which we were executing ProgramB made it simply an
"extension" to ProgramA. We were/are "calling" ProgramB. What is
happening is that the memory heap for ProgramA will continue to grow
up to the max. required by any ProgramB that might get called since it
is being executed in the same thread. So what I was seeing was not a
memory leak, just an increase in the heap. Subsequent ProgramB's
called by ProgramA will infact reuse the memory on the heap, but the
heap will never get any smaller than the largest "called" program's
requirements.

The answer to this has been that on a daily basis we acutually allow
ProgramA to start another instance of itself (in a new thread) and
then exit. This means that ProgramA starts with a clean memory heap.
The slowdown we were experiencing must have been a result of the
command processor having to "wade" through an extremely large heap to
determine what memory was still in use and what could be reused.

To the best of my testing ability, I could find NO memory leak in
Pervasive BTrieve.

Thanks again,
Lee

On 14 Feb 2005 09:27:00 EST, "Bill Bach"
<billbach (AT) goldstarsoftware (DOT) com> wrote: