dbTalk Databases Forums  

segmented files btrieve API

comp.databases.btrieve comp.databases.btrieve


Discuss segmented files btrieve API in the comp.databases.btrieve forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
nmm
 
Posts: n/a

Default segmented files btrieve API - 09-28-2009 , 08:55 AM






Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea

Reply With Quote
  #2  
Old   
BtrieveBill
 
Posts: n/a

Default Re: segmented files btrieve API - 09-28-2009 , 10:39 AM






I have never seen anything even approaching this symptom in all of my
travels. My first inkling is that there is some other process which is
interfering with the file access, such as an Antivirus application.
Many people remember to exclude their prmary database extension from the
AV scanning, but they forget to also block the ^01, ^02, and other
related extensions.

My second thought is that if this is ONLY a problem on PSQLv9 and
PSQLv10, both of which fully support large, single-file tables, that
yuou should consider rebuilding the files into a single extent anyway.
This typically improves performance, makes it easier to administer the
system. You should ONLY do this if you are running v9.5 or above --
PSQLv9.1 and older had some problems with this setting.

Third, I cannot imagine that anything would touch the client side of the
equation -- this doesn't seem to make much sense at all...
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Quote:
Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea


Reply With Quote
  #3  
Old   
nmm
 
Posts: n/a

Default Re: segmented files btrieve API - 09-29-2009 , 07:08 AM



Hi Bill, thank you for your endeavour in this matter!

Encouraged by your remarks, yesterday I accomplish further debugging,
because I said to me, that if you have never seen something like that,
then it *must* be a bug in our code.

Actually I found it! Here it is:

we use the function BTRCALL (not BTRV): and in BTRCALL one can pass
the keylength. The STAT-Cammand (Code 15) returns something in the
keybuffer, if the file is segmented (actually the filename of the
first segment), if not(segmented), it returns only one byte: #0. This
was new in PSQL9. We allways pass a keylength of 4 and as keybuffer a
longint.

In PSQL7/8 the passed keylength was respectet by the API(or in
keybuffer allways was returned #0?), in PSQL 9/10 seemingly not: the
BTRCALL-function returns more bytes in the keybuffer, than specified
in keylength: because we only had a longint-Variable, this collateral
bytes destroyed parts of the object-tables from the Delphi-VCL, so we
get this obscure beahvior.

Now we pass 256 byte keybuffer (as specified in the
btreive-API description V9.5), and all is fine again.

Thanks agin.
Mircea


BtrieveBill schrieb:
Quote:
I have never seen anything even approaching this symptom in all of my
travels. My first inkling is that there is some other process which is
interfering with the file access, such as an Antivirus application. Many
people remember to exclude their prmary database extension from the AV
scanning, but they forget to also block the ^01, ^02, and other related
extensions.

My second thought is that if this is ONLY a problem on PSQLv9 and
PSQLv10, both of which fully support large, single-file tables, that
yuou should consider rebuilding the files into a single extent anyway.
This typically improves performance, makes it easier to administer the
system. You should ONLY do this if you are running v9.5 or above --
PSQLv9.1 and older had some problems with this setting.

Third, I cannot imagine that anything would touch the client side of the
equation -- this doesn't seem to make much sense at all...
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea


Reply With Quote
  #4  
Old   
BtrieveBill
 
Posts: n/a

Default Re: segmented files btrieve API - 09-29-2009 , 11:00 AM



Interesting. The documentation for the STAT call does indicate:

Quote:
Specify a Key Buffer that is at least 255 characters long.
What I actually expected to see, if you properly specified a KeyLength
of 4, is a Status 21 get returned. Please check the setting in your
client for "Verify Key Length" and make surew that this is set to ON
(Checked). If disabled, additional memory corruption can arise.

If you find that the key length is being verified, and you indeed have a
4 for the key length on this call, then this sounds like a bug in the
client that needs to be fixed.
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***



nmm wrote:
Quote:
Hi Bill, thank you for your endeavour in this matter!

Encouraged by your remarks, yesterday I accomplish further debugging,
because I said to me, that if you have never seen something like that,
then it *must* be a bug in our code.

Actually I found it! Here it is:

we use the function BTRCALL (not BTRV): and in BTRCALL one can pass
the keylength. The STAT-Cammand (Code 15) returns something in the
keybuffer, if the file is segmented (actually the filename of the
first segment), if not(segmented), it returns only one byte: #0. This
was new in PSQL9. We allways pass a keylength of 4 and as keybuffer a
longint.

In PSQL7/8 the passed keylength was respectet by the API(or in
keybuffer allways was returned #0?), in PSQL 9/10 seemingly not: the
BTRCALL-function returns more bytes in the keybuffer, than specified
in keylength: because we only had a longint-Variable, this collateral
bytes destroyed parts of the object-tables from the Delphi-VCL, so we
get this obscure beahvior.

Now we pass 256 byte keybuffer (as specified in the
btreive-API description V9.5), and all is fine again.

Thanks agin.
Mircea


BtrieveBill schrieb:
I have never seen anything even approaching this symptom in all of my
travels. My first inkling is that there is some other process which is
interfering with the file access, such as an Antivirus application. Many
people remember to exclude their prmary database extension from the AV
scanning, but they forget to also block the ^01, ^02, and other related
extensions.

My second thought is that if this is ONLY a problem on PSQLv9 and
PSQLv10, both of which fully support large, single-file tables, that
yuou should consider rebuilding the files into a single extent anyway.
This typically improves performance, makes it easier to administer the
system. You should ONLY do this if you are running v9.5 or above --
PSQLv9.1 and older had some problems with this setting.

Third, I cannot imagine that anything would touch the client side of the
equation -- this doesn't seem to make much sense at all...
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea



Reply With Quote
  #5  
Old   
nmm
 
Posts: n/a

Default Re: segmented files btrieve API - 09-29-2009 , 12:36 PM



"Verify Key Length" was already set to ON(the whole time). Actually
keybuffer and keylength are not used in connection with keys/indices
in the STAT command, so may be pervasive does not (longer?) check
keylength in STAT?
(the 4 for the keylength is "hard codeed" in our call of STAT, so its
really a 4).

Regards
Mircea

BtrieveBill schrieb:
Quote:
Interesting. The documentation for the STAT call does indicate:

Specify a Key Buffer that is at least 255 characters long.

What I actually expected to see, if you properly specified a KeyLength
of 4, is a Status 21 get returned. Please check the setting in your
client for "Verify Key Length" and make surew that this is set to ON
(Checked). If disabled, additional memory corruption can arise.

If you find that the key length is being verified, and you indeed have a
4 for the key length on this call, then this sounds like a bug in the
client that needs to be fixed.
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***



nmm wrote:
Hi Bill, thank you for your endeavour in this matter!

Encouraged by your remarks, yesterday I accomplish further debugging,
because I said to me, that if you have never seen something like that,
then it *must* be a bug in our code.

Actually I found it! Here it is:

we use the function BTRCALL (not BTRV): and in BTRCALL one can pass
the keylength. The STAT-Cammand (Code 15) returns something in the
keybuffer, if the file is segmented (actually the filename of the
first segment), if not(segmented), it returns only one byte: #0. This
was new in PSQL9. We allways pass a keylength of 4 and as keybuffer a
longint.

In PSQL7/8 the passed keylength was respectet by the API(or in
keybuffer allways was returned #0?), in PSQL 9/10 seemingly not: the
BTRCALL-function returns more bytes in the keybuffer, than specified
in keylength: because we only had a longint-Variable, this collateral
bytes destroyed parts of the object-tables from the Delphi-VCL, so we
get this obscure beahvior.

Now we pass 256 byte keybuffer (as specified in the
btreive-API description V9.5), and all is fine again.

Thanks agin.
Mircea


BtrieveBill schrieb:
I have never seen anything even approaching this symptom in all of my
travels. My first inkling is that there is some other process which is
interfering with the file access, such as an Antivirus application. Many
people remember to exclude their prmary database extension from the AV
scanning, but they forget to also block the ^01, ^02, and other related
extensions.

My second thought is that if this is ONLY a problem on PSQLv9 and
PSQLv10, both of which fully support large, single-file tables, that
yuou should consider rebuilding the files into a single extent anyway.
This typically improves performance, makes it easier to administer the
system. You should ONLY do this if you are running v9.5 or above --
PSQLv9.1 and older had some problems with this setting.

Third, I cannot imagine that anything would touch the client side of the
equation -- this doesn't seem to make much sense at all...
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea



Reply With Quote
  #6  
Old   
BtrieveBill
 
Posts: n/a

Default Re: segmented files btrieve API - 10-02-2009 , 01:16 PM



I never use the BTRCALL function, favoring instead BTRV, which does not
use the extra parameter. I'll need to build a test application to
confirm if this is a bug or not.
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Quote:
"Verify Key Length" was already set to ON(the whole time). Actually
keybuffer and keylength are not used in connection with keys/indices
in the STAT command, so may be pervasive does not (longer?) check
keylength in STAT?
(the 4 for the keylength is "hard codeed" in our call of STAT, so its
really a 4).

Regards
Mircea

BtrieveBill schrieb:
Interesting. The documentation for the STAT call does indicate:

Specify a Key Buffer that is at least 255 characters long.
What I actually expected to see, if you properly specified a KeyLength
of 4, is a Status 21 get returned. Please check the setting in your
client for "Verify Key Length" and make surew that this is set to ON
(Checked). If disabled, additional memory corruption can arise.

If you find that the key length is being verified, and you indeed have a
4 for the key length on this call, then this sounds like a bug in the
client that needs to be fixed.
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***



nmm wrote:
Hi Bill, thank you for your endeavour in this matter!

Encouraged by your remarks, yesterday I accomplish further debugging,
because I said to me, that if you have never seen something like that,
then it *must* be a bug in our code.

Actually I found it! Here it is:

we use the function BTRCALL (not BTRV): and in BTRCALL one can pass
the keylength. The STAT-Cammand (Code 15) returns something in the
keybuffer, if the file is segmented (actually the filename of the
first segment), if not(segmented), it returns only one byte: #0. This
was new in PSQL9. We allways pass a keylength of 4 and as keybuffer a
longint.

In PSQL7/8 the passed keylength was respectet by the API(or in
keybuffer allways was returned #0?), in PSQL 9/10 seemingly not: the
BTRCALL-function returns more bytes in the keybuffer, than specified
in keylength: because we only had a longint-Variable, this collateral
bytes destroyed parts of the object-tables from the Delphi-VCL, so we
get this obscure beahvior.

Now we pass 256 byte keybuffer (as specified in the
btreive-API description V9.5), and all is fine again.

Thanks agin.
Mircea


BtrieveBill schrieb:
I have never seen anything even approaching this symptom in all of my
travels. My first inkling is that there is some other process which is
interfering with the file access, such as an Antivirus application. Many
people remember to exclude their prmary database extension from the AV
scanning, but they forget to also block the ^01, ^02, and other related
extensions.

My second thought is that if this is ONLY a problem on PSQLv9 and
PSQLv10, both of which fully support large, single-file tables, that
yuou should consider rebuilding the files into a single extent anyway.
This typically improves performance, makes it easier to administer the
system. You should ONLY do this if you are running v9.5 or above --
PSQLv9.1 and older had some problems with this setting.

Third, I cannot imagine that anything would touch the client side of the
equation -- this doesn't seem to make much sense at all...
Goldstar Software Inc.
Pervasive-based Products, Training & Services
Bill Bach
BillBach (AT) goldstarsoftware (DOT) com
http://www.goldstarsoftware.com
*** Pervasive Training - October 2009 in Chicago ***


nmm wrote:
Hi
again we made a strange observation (PSQL 9.x and 10.x).

We have a application written with delphi6 unsing btrieveAPI. If we
open a btrieve-file, we send opcode 0, 15 and 13.
This works fine for us, but only if the file we open is not segmented
from PSQL. If it has segments, than the application crashes from time
to time with varying exceptions (not allways the same type of
exception: invalid pointer op, stack overflow, extection while writung
to adress #nnnnnn and so on), but not immediate after the
btrieve-calls (they return status 0).
The open-procedure runs through exactly the same code.
If the file is not segmented, all is fine, if it is segmented, than
the errors occur. This reproducible, but we can not determin a point
in the code, from where the errors rise, because these are at each
occurence different - and we can not find any system in this.
(we have the source-code of that applicatiuon an can debug it unsing
Delphi).

This strange things happen in PSQL 9.x as well as in PSQL 10.x (server
and workstation engines), but seemingly not in PSQL 8.x!

We switched off segmentantion and run a rebuild on that segmented
file: after this, the error does not occur.

BTW: the files in question are by no means corrupted: we made a test
on different machines with different created files: it is simply as
described: if it is segmented, than the eroors occur, if not, than not.

We still tried to use a bigger buffer for pos_block (255 bytes, not
128 as demanded), because the only thing we can imagine, is that the
btrcall-function wirtes more data the the pos_bloc, than it should:
but this does not solve the problm.



My question is:
-is there anything, that has to be considered on the client, if the
file in question is segmented?
-does anyboday make a similar obseravtion and has any idea, what the
reason is?


I would guess, that segmentation should be completely transparent to
the clinet, but seemingly there is something different.

Any hint will by highly appreciated!


Regards
Mircea


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.