dbTalk Databases Forums  

how to create a in memory db ?

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss how to create a in memory db ? in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
William
 
Posts: n/a

Default how to create a in memory db ? - 10-11-2005 , 03:18 AM







Dear all:
i want a database which read all data in memory when opened. can
berkeley-db do that ?
thanks in advance!


-----------------
Regards.


Reply With Quote
  #2  
Old   
Ron
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-11-2005 , 05:40 PM






Your title in your question is asking how to create an in-memory
database in Berkeley DB. Documentation for how create a RAM only
database can be found here:

http://www.sleepycat.com/docs/ref/program/ram.html

In your message you ask about how to pre-load a file backed database in
cache. At least that is what I think you are asking. In order to do
that you should create a cache large enough to hold the database and
create a cursor that will walk the database. You can use the
DB_MULTIPLE flag to make this go faster. For more information:

http://www.sleepycat.com/docs/api_cxx/dbc_get.html

Ron


Reply With Quote
  #3  
Old   
William
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-11-2005 , 09:42 PM





--
Dear :
thanks for your reply. acturally i am using OpenLdap which takes
berkeley-db as its backend db. and for performance reason i want to read all
entries in memory when startup .but i dont know whether i can do it , could
you give me some advise ?

-----------------
Regards.


Quote:
Your title in your question is asking how to create an in-memory
database in Berkeley DB. Documentation for how create a RAM only
database can be found here:

http://www.sleepycat.com/docs/ref/program/ram.html

In your message you ask about how to pre-load a file backed database in
cache. At least that is what I think you are asking. In order to do
that you should create a cache large enough to hold the database and
create a cursor that will walk the database. You can use the
DB_MULTIPLE flag to make this go faster. For more information:

http://www.sleepycat.com/docs/api_cxx/dbc_get.html

Ron



Reply With Quote
  #4  
Old   
Patrick Schaaf
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-11-2005 , 11:59 PM



"William" <W (AT) abc (DOT) com> writes:

Quote:
thanks for your reply. acturally i am using OpenLdap which takes
berkeley-db as its backend db. and for performance reason i want to read all
entries in memory when startup .but i dont know whether i can do it , could
you give me some advise ?
I think that OpenLdap developers will be vastly more competent to
help you with this idea. Or talk you out of it, if appropriate.

Berkeley DB, as far as I know, does not contain an application-independant
"magically pull everything into RAM as fast as possible" knob.

best regards
Patrick


Reply With Quote
  #5  
Old   
Howard Chu
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-12-2005 , 11:42 AM



Patrick Schaaf wrote:
Quote:
"William" <W (AT) abc (DOT) com> writes:
thanks for your reply. acturally i am using OpenLdap which takes
berkeley-db as its backend db. and for performance reason i want to read all
entries in memory when startup .but i dont know whether i can do it , could
you give me some advise ?

I think that OpenLdap developers will be vastly more competent to
help you with this idea. Or talk you out of it, if appropriate.

Berkeley DB, as far as I know, does not contain an application-independant
"magically pull everything into RAM as fast as possible" knob.
But something that may be almost as good is to define a BDB cache that
resides in shared memory. If the BDB cache is large enough to contain
all of the data, then once the database is primed everything will stay
in memory; on the next application startup it will still be there.

--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/


Reply With Quote
  #6  
Old   
Patrick Schaaf
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-12-2005 , 03:11 PM



Howard Chu <XYZ.hyc (AT) highlandsun (DOT) com> writes:

Quote:
But something that may be almost as good is to define a BDB cache that
resides in shared memory. If the BDB cache is large enough to contain
all of the data, then once the database is primed everything will stay
in memory; on the next application startup it will still be there.
If the OS is worth its pagecache algorithms, the data in the backing
file will be there just as well. A file backed shared memory segment,
as is usually used for the cache, is practically identical in handling
to the backing file itself.

None of this is relevant for cold start first time reading of
the data, and possible latency stabilization one hopes to
gain by doing it in one go. As seemed to be the topic.

best regards
Patrick


Reply With Quote
  #7  
Old   
Howard Chu
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-12-2005 , 10:03 PM



Patrick Schaaf wrote:
Quote:
Howard Chu <XYZ.hyc (AT) highlandsun (DOT) com> writes:
But something that may be almost as good is to define a BDB cache that
resides in shared memory. If the BDB cache is large enough to contain
all of the data, then once the database is primed everything will stay
in memory; on the next application startup it will still be there.

If the OS is worth its pagecache algorithms, the data in the backing
file will be there just as well.
That is a Very Big IF.

Quote:
A file backed shared memory segment,
as is usually used for the cache, is practically identical in handling
to the backing file itself.
There is a marked performance boost on Solaris using SysV shared memory
regions instead of shared mmap'd files for the BDB cache. The difference
on Linux is unmeasurable, but not everybody runs Linux. There are
probably many other platforms where the difference is also significant;
you don't have enough information to make such definitive statements.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc
OpenLDAP Core Team http://www.openldap.org/project/


Reply With Quote
  #8  
Old   
Patrick Schaaf
 
Posts: n/a

Default Re: how to create a in memory db ? - 10-13-2005 , 01:11 AM



Howard Chu <XYZ.hyc (AT) highlandsun (DOT) com> writes:

Quote:
A file backed shared memory segment,
as is usually used for the cache, is practically identical in handling
to the backing file itself.

There is a marked performance boost on Solaris using SysV shared memory
regions instead of shared mmap'd files for the BDB cache. The difference
on Linux is unmeasurable, but not everybody runs Linux. There are
probably many other platforms where the difference is also significant;
you don't have enough information to make such definitive statements.
Note that I explicitly wrote 'file backed shared memory segment'.
SysV shared memory is not file backed.

But, anyway, point taken. I was indeed speaking from my Linux experience,
only, as I haven't had any other meaningful experience in the last >10 years.

best regards
Patrick


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.