dbTalk Databases Forums  

invalid seconday index delete storing C++ objects.

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss invalid seconday index delete storing C++ objects. in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
danwald
 
Posts: n/a

Default invalid seconday index delete storing C++ objects. - 03-27-2006 , 11:43 AM






I am storing a C++ object in my database with an auto_increment integer
key value.
<code>

class obj {
public:
int a, b, c, d, e, user_data_len;
char *user_data;
void *get_buffer(); //returns a user allocated buffer that marshalls
all the data;
int get_buffer_size(){ return sizeof(int)*5+user_data_len; }
obj(obj &o);
obj(void *data);
print(FILE *fp);
}

Dbt value;
value.set_data( obj.get_buffer());
value.set_size(obj.get_buffer_size());

//I dont set value.set_flags( DB_USR_MALLOC )
</code>

There are secondary databases with the associated key extractors are
working and I can walk throught the databases and secondary indices
printing out the contents to a file. This verifies that inserts work.
<code>
//Here is some code for the keyextractor
memset(skey, 0, sizeof(Dbt));
int b = ((Obj*) pdata.get_value())->b;
skey.set_data(b);
skey.get_data(sizeof(int));
fprintf(stderr, "\nb callback . value %d", b);
</code>

However, I have a seperate thread that deletes (via a cursor on a
secondary index which is keyed on a value in the contiguous buffer)
records from a secondary database.
This used to work fine if it was a struct. But it does work with it as
an object. I get
"Failed to delete a b_key 29950: DB_SECONDARY_BAD: Secondary index
inconsistent with primary"

The code is below. Please could you tell me what I am doing wrong and
also, and also whether or not I should set memory handling flags for
the Dbt values.

Much thanks,
danny

<code>
//delete thread code. I do a partial get on the value for the cursor as
I just want the key. This used to work fine when it was a struct. But
not now.
//I also tried getting whole values .
Dbt key,value;
memset(key, 0, sizeof(Dbt));
key.set_data( j )
key.set_size(sizeof(int));

memset(value, 0, sizeof(Dbt));
value.set_flags(DB_DBT_PARTIAL);
value.set_size(0);
value.set_dlen(0);
value.set_doff(0);

if((ret=cursor->get(&key, &value, DB_SET_RANGE)))
{
index->err(ret, "\nNo ttl match");
goto end;
}

while(ret == 0)
{
int k = *((int*)key.get_data());
int dup = 0;
if(cursor->count((db_recno_t*)&dup, 0 ))
fprintf(stderr, "\nFailed to get dup count");
else
--dup;

if((ret = cursor->del(0)) )
{
index->err(ret, "\nFailed to delete an index with value %d",
k);

}
else
++rm;
ret=cursor->get(&key, &value, DB_NEXT);
}
</code>


Reply With Quote
  #2  
Old   
Michael Cahill
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-27-2006 , 08:17 PM






Hi Dan,

I think your problem is here:

Quote:
code
//Here is some code for the keyextractor
memset(skey, 0, sizeof(Dbt));
int b = ((Obj*) pdata.get_value())->b;
skey.set_data(b);
skey.get_data(sizeof(int));
fprintf(stderr, "\nb callback . value %d", b);
/code
The code above will set the data *pointer* in the secondary key to the
integer *value* you want. I'm surprised that your C++ compiler doesn't
at least issue a warning about this (if not an error).

Instead, I think you need something like:

skey.set_data(&((Obj*) pdata.get_value())->b);
skey.get_data(sizeof(int));

Also note that there is no need to zero out objects with memset when
using the C++ API: the "new" operator takes care of initializing
objects.

Regards,
Michael.



Reply With Quote
  #3  
Old   
danwald
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-28-2006 , 09:22 AM



Hey Mike,

that was a typo. The actual key extractor code is below. data is
essentially one contigious array of the data that I am concerned with.
They key is set to the address of the third int in the array. The value
that printed with the debug is valid. Also when I walk through all the
databases (pri/secondary) and print the contents, it is all valid,
keyed on the right values and sorted how I need them to be.
Do you know what I am doing wrong?

I do have a concern however. When I insert the Dbt data into the
primary database, I do it w/o specfing flags.

value.set_data(md.get_buffer());
value.set_size(md.get_size());
where md is my obj and the get_buffer() call returns a contiguos void*
array of my values.

int ret;
if( (ret= db->put( NULL, &key, &value, 0 )) )
{
db->err(ret, "\nFailed to insert record[%d]:", id);
return 0;
}


thanks,
danny

----------- Key Extractor -----------------

char *d = (char*) pdata->get_data();
skey->set_data((int*)(d+(sizeof(int)*2)));
skey->set_size(sizeof(int));

#ifdef DEBUG_VERBOSE
fprintf(stderr, "\n callback: TTL = %d",
*(int*)skey->get_data());
#endif
return 0;


Reply With Quote
  #4  
Old   
Michael Cahill
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-28-2006 , 05:34 PM



Hi Dan,

I can't see anything obviously wrong with the code you've posted. The
best way to resolve this would be if you can send a small but complete
test case to support (AT) sleepycat (DOT) com so that we can reproduce the problem
locally.

Regards,
Michael.


Reply With Quote
  #5  
Old   
danwald
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-28-2006 , 06:41 PM



Mike,
I think I have narrowed the problem down. It has something to do with
my set_bt_cmp and set_dup_cmp functions.
for all my db including indices, the bt_cmp is the same function as the
keys are all integers.

for dup_cmp for the indices, they are all the same as well and are set
to the third int in the data buffer returned by dbt pointers. So when I
tried to compare those values, the first value seem valid, but the
second one is garbage.
Why is this? I thought that the function would return a pointer to the
data Dbt that was inserted into the primary?

So for now I have not set_dup_cmp for the indices.
thanks,
dan


Reply With Quote
  #6  
Old   
Michael Cahill
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-28-2006 , 08:01 PM



Hi Dan,

The key comparison and duplicate comparison functions apply only to the
database on which they are set, regardless of whether the database is a
secondary index. So when you set a duplicate comparison routine on the
secondary, it will be passed the data items *from the secondary
itself*.

In your case, if I understand correctly, those data items will be
single integers (the primary keys), so any duplicate comparison routine
should expect items of that size and format.

Regards,
Michael.


Reply With Quote
  #7  
Old   
danwald
 
Posts: n/a

Default Re: invalid seconday index delete storing C++ objects. - 03-28-2006 , 09:23 PM



Mike,

that makes a whole lot more sense.

Mike I had another clarification that I hope you could help with.

I was going through the example_cxx code and I noticed that when the
example generates the contiguous variable char* array (Inventory) for
insertion into the database, there are no flags associated with the Dbt
value and it is simply inserted.

However, when the user wanted to put a Vendor object into its database,
it sets the flag DB_DBT_USERMEM for the data dbt. Why does it do this
and not just insert it like the previous inventory data item?

To clarify the doubt, in what circumstances to you specify which flag
and if it is a flag like DB_DBT_MALLOC, do you clear it whenever you
retreive the data?

Lastly, when the pointers are specifed for the data dbt's
(data.set_data(&vendor)); does the db make a copy of it? The reason I
ask is that the value is reset and the address would be invalid?

thanks for you help so far,
dan


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.