dbTalk Databases Forums  

webmail schema design?

comp.databases comp.databases


Discuss webmail schema design? in the comp.databases forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
meranayaghar@yahoo.com
 
Posts: n/a

Default webmail schema design? - 05-01-2009 , 11:03 AM






Hi Folks,
Does anyone have any insight into what user mailbox stores look like
for yahoo, gmail, hotmail, etc? I am curious if they have, e.g. a
mailbox table for each user that contains all their email or whether
they have a single large table with email for all users or whether
they have a smaller subset of tables with the users distributed
between them? Each user can have a large number of messages, so a
single table would end up having billions of records (for one of these
webmail systems with millions of users) and likewise you wouldn't want
separate tables for each user because then you'd have millions of
tables. Any thoughts? Thanks.
Sean

Reply With Quote
  #2  
Old   
Gary R. Schmidt
 
Posts: n/a

Default Re: webmail schema design? - 05-02-2009 , 12:35 AM






meranayaghar (AT) yahoo (DOT) com wrote:
Quote:
Hi Folks,
Does anyone have any insight into what user mailbox stores look like
for yahoo, gmail, hotmail, etc? I am curious if they have, e.g. a
mailbox table for each user that contains all their email or whether
they have a single large table with email for all users or whether
they have a smaller subset of tables with the users distributed
between them? Each user can have a large number of messages, so a
single table would end up having billions of records (for one of these
webmail systems with millions of users) and likewise you wouldn't want
separate tables for each user because then you'd have millions of
tables. Any thoughts? Thanks.
Sean
It's is very unlikely that they use a single database (in the sense of
tables and SQL and DDL) to store messages for such large user bases, the
overheads would be too great. Having said that, of course if when a
user authenticates you direct it to a particular (set of) server(s), you
would be able to do it.

Early on-line mail services used bog-standard email systems underneath,
usually an IMAP variant - see <http://www.imap/org> to begin learning
far more than you want to about IMAP - of which some implementations use
an underlying database, but most are file-store based.

Gmail probably uses the Google file system.

Squirrelmail, which may be the commonest webmail interface in the world,
sits on top of IMAP, see above for what can be below that.

Once upon a time, mentioning storing mail in a database would stir up a
veritable hornets nest in the various comp.mail.* newsfroups, nowadays
you might cop as much as, "Well, if you want to, I'm not going to waste
my breath telling you why it's a bad idea."[1]

Cheers,
Gary B-)

1 - Why do some people consider using a database to store email a bad idea?

Here is an example, view it in the context of a very large (corporate or
academic) user base, who expect to be able to retrieve messages they
have mistakenly deleted, and assume that messages they received 20+
years ago are still available to them.

Consider the problem of restoring *one* user's messages from backup.

It is trivial in relation to a file-store "pull this directory tree from
the tape and put it over there, so the user can examine it at heeshes
leisure".

With a database, "restore table X from the backup media" is/was either
not available, or an expensive add-on. And as for "restore table X but
keep anything currently there" - I can't say I've seen that one
anywhere, but I could be wrong.

Same for "restore this related group of tables."

And "select all messages from mirror-database where user = 'xyz'" just
won't work!

Cheers,
Gary B-)

--
__________________________________________________ ____________________________
Armful of chairs: Something some people would not know
whether you were up them with or not
- Barry Humphries


Reply With Quote
  #3  
Old   
Robert Klemme
 
Posts: n/a

Default Re: webmail schema design? - 05-02-2009 , 08:00 AM



On 02.05.2009 07:35, Gary R. Schmidt wrote:

Quote:
1 - Why do some people consider using a database to store email a bad idea?

Here is an example, view it in the context of a very large (corporate or
academic) user base, who expect to be able to retrieve messages they
have mistakenly deleted, and assume that messages they received 20+
years ago are still available to them.

Consider the problem of restoring *one* user's messages from backup.

It is trivial in relation to a file-store "pull this directory tree from
the tape and put it over there, so the user can examine it at heeshes
leisure".

With a database, "restore table X from the backup media" is/was either
not available, or an expensive add-on. And as for "restore table X but
keep anything currently there" - I can't say I've seen that one
anywhere, but I could be wrong.
Oracle Flashback comes to mind.

Kind regards

robert


Reply With Quote
  #4  
Old   
Gary R. Schmidt
 
Posts: n/a

Default Re: webmail schema design? - 05-02-2009 , 10:21 AM



Robert Klemme wrote:
Quote:
On 02.05.2009 07:35, Gary R. Schmidt wrote:

1 - Why do some people consider using a database to store email a bad
idea?

Here is an example, view it in the context of a very large (corporate
or academic) user base, who expect to be able to retrieve messages
they have mistakenly deleted, and assume that messages they received
20+ years ago are still available to them.

Consider the problem of restoring *one* user's messages from backup.

It is trivial in relation to a file-store "pull this directory tree
from the tape and put it over there, so the user can examine it at
heeshes leisure".

With a database, "restore table X from the backup media" is/was either
not available, or an expensive add-on. And as for "restore table X
but keep anything currently there" - I can't say I've seen that one
anywhere, but I could be wrong.

Oracle Flashback comes to mind.

Emmachisit?

Cheers,
Gary B-)



Reply With Quote
  #5  
Old   
meranayaghar@yahoo.com
 
Posts: n/a

Default Re: webmail schema design? - 05-02-2009 , 11:26 AM



On May 1, 10:35*pm, "Gary R. Schmidt" <grschm... (AT) acm (DOT) org> wrote:
Quote:
meranayag... (AT) yahoo (DOT) com wrote:
Hi Folks,
*Does anyone have any insight into what user mailbox stores look like
for yahoo, gmail, hotmail, etc? I am curious if they have, e.g. a
mailbox table for each user that contains all their email or whether
they have a single large table with email for all users or whether
they have a smaller subset of tables with the users distributed
between them? Each user can have a large number of messages, so a
single table would end up having billions of records (for one of these
webmail systems with millions of users) and likewise you wouldn't want
separate tables for each user because then you'd have millions of
tables. Any thoughts? Thanks.
Sean

It's is very unlikely that they use a single database (in the sense of
tables and SQL and DDL) to store messages for such large user bases, the
overheads would be too great. *Having said that, of course if when a
user authenticates you direct it to a particular (set of) server(s), you
would be able to do it.

Early on-line mail services used bog-standard email systems underneath,
usually an IMAP variant - see <http://www.imap/org> to begin learning
far more than you want to about IMAP - of which some implementations use
an underlying database, but most are file-store based.

Gmail probably uses the Google file system.

Squirrelmail, which may be the commonest webmail interface in the world,
sits on top of IMAP, see above for what can be below that.

Once upon a time, mentioning storing mail in a database would stir up a
veritable hornets nest in the various comp.mail.* newsfroups, nowadays
you might cop as much as, "Well, if you want to, I'm not going to waste
my breath telling you why it's a bad idea."[1]

* * * * Cheers,
* * * * * * * * Gary * *B-)

1 - Why do some people consider using a database to store email a bad idea?

Here is an example, view it in the context of a very large (corporate or
academic) user base, who expect to be able to retrieve messages they
have mistakenly deleted, and assume that messages they received 20+
years ago are still available to them.

Consider the problem of restoring *one* user's messages from backup.

It is trivial in relation to a file-store "pull this directory tree from
the tape and put it over there, so the user can examine it at heeshes
leisure".

With a database, "restore table X from the backup media" is/was either
not available, or an expensive add-on. *And as for "restore table X but
keep anything currently there" - I can't say I've seen that one
anywhere, but I could be wrong.

Same for "restore this related group of tables."

And "select all messages from mirror-database where user = 'xyz'" *just
won't work!

* * * * Cheers,
* * * * * * * * Gary * *B-)

Thanks Gary. What you say above makes a lot of sense, the idea of
storing all user email in databases does seem a might untenable.
Sean

Quote:
--
__________________________________________________ _________________________*___
Armful of chairs: Something some people would not know
* * * * * * * * * *whether you were up them with or not
* * * * * * * * * * * * * * * * * * * - Barry Humphries


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.