dbTalk Databases Forums  

Database in RAM

comp.databases.berkeley-db comp.databases.berkeley-db


Discuss Database in RAM in the comp.databases.berkeley-db forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Alea
 
Posts: n/a

Default Database in RAM - 10-08-2003 , 10:31 AM






Hi,

I am new to Berkeley DB so please forgive my relative Ignorance. The
application I am developing should work as follows:


1. On startup - load as much as possible from a database file into RAM
for fast access.

2. During normal operation the data is hold in RAM (only if data size
exceeds available RAM, disk IO is performed).

3. On shutdown (or any other desired point in time) - flush the memory
content back to the disk file.


I realize that the relevant manual sections are probably "The Memory
Pool Subsystem" and "The Berkeley DB Environment". However I don't
understand how to code a solution. The memory pool solution seems to
be overly complex to me (as I understand it I'd have to manually
retrieve pages (What are pages?) from a memory file (I only have a
database file)). Could someone in this group please outline a general
approach for the design of an application like the one I imagined
above?

Kind regards,
Alea Miras

Reply With Quote
  #2  
Old   
Dan Creswell
 
Posts: n/a

Default Re: Database in RAM - 10-08-2003 , 11:28 AM






Alea wrote:
Quote:
Hi,

I am new to Berkeley DB so please forgive my relative Ignorance. The
application I am developing should work as follows:


1. On startup - load as much as possible from a database file into RAM
for fast access.

2. During normal operation the data is hold in RAM (only if data size
exceeds available RAM, disk IO is performed).

3. On shutdown (or any other desired point in time) - flush the memory
content back to the disk file.


I realize that the relevant manual sections are probably "The Memory
Pool Subsystem" and "The Berkeley DB Environment". However I don't
understand how to code a solution. The memory pool solution seems to
be overly complex to me (as I understand it I'd have to manually
retrieve pages (What are pages?) from a memory file (I only have a
database file)). Could someone in this group please outline a general
approach for the design of an application like the one I imagined
above?

Kind regards,
Alea Miras
I'd suggest, to start with, that you just use the standard database
access methods. You can specify a cache size for such a database (or
for the environment depending on the flags specified).

This solution will behave roughly as follows:

(1) The cache will be initially empty but access to the database will
gradually fill the cache - I guess you could artificially pre-load the
cache by methods such as iterating with a cursor.

(2) The cache may not be big enough to store all data in the database
but Db will silently handle the necessary paging for you.

(3) Depending on your recovery requirements, you may get more I/O than
you expect simply because Db must do some logging to ensure data is
recoverable.

This will give you a simple approximation of what you're asking for. If
you want greater control, then you'll have to confront the more complex
API's and learn about paging etc.

Hope that helps,

Dan.



Reply With Quote
  #3  
Old   
Hans-Bernhard Broeker
 
Posts: n/a

Default Re: Database in RAM - 10-08-2003 , 11:31 AM



Alea <alea101 (AT) postmaster (DOT) co.uk> wrote:

Quote:
1. On startup - load as much as possible from a database file into RAM
for fast access.

2. During normal operation the data is hold in RAM (only if data size
exceeds available RAM, disk IO is performed).

3. On shutdown (or any other desired point in time) - flush the memory
content back to the disk file.
Use a DB Environment with a rather large cache, then.
--
Hans-Bernhard Broeker (broeker (AT) physik (DOT) rwth-aachen.de)
Even if all the snow were burnt, ashes would remain.


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.