dbTalk Databases Forums  

Table Size- 50-100Million Rows Will It Work!

comp.databases.sybase comp.databases.sybase


Discuss Table Size- 50-100Million Rows Will It Work! in the comp.databases.sybase forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Jim Douglas
 
Posts: n/a

Default Table Size- 50-100Million Rows Will It Work! - 04-14-2004 , 06:23 PM






I am looking at some new system requirements where we will have one table
with 50M rows and another with 100M.
We debated all afternoon about various methods of how we should handle these
tables.

DataEntry will be done with the primarykey and reading is the same with the
same primarykey. Reporting is something else and is a requirement. I am not
a DBA type and was wondering how others are handling these large tables,
some folks are saying break them up into smaller subset tables, use Sybase
distributed tables ( is there such a thing), and others are indicating that
Sybase might not be the best choice for database with tables this size.

Any input is appreciated. What would you do? How would you do it?


Thanks!



--

Jim Douglas
http:\\www.genesis-software.com

Latitude 32.5818
Longitude -96.5412
Elevation 497






Reply With Quote
  #2  
Old   
Vlad
 
Posts: n/a

Default Re: Table Size- 50-100Million Rows Will It Work! - 04-14-2004 , 10:56 PM






Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
Quote:
I am looking at some new system requirements where we will have one table
with 50M rows and another with 100M.
We debated all afternoon about various methods of how we should handle these
tables.

DataEntry will be done with the primarykey and reading is the same with the
same primarykey. Reporting is something else and is a requirement. I am not
a DBA type and was wondering how others are handling these large tables,
some folks are saying break them up into smaller subset tables, use Sybase
distributed tables ( is there such a thing), and others are indicating that
Sybase might not be the best choice for database with tables this size.

Any input is appreciated. What would you do? How would you do it?
Sybase is quite capable to hold reliably a 100M of rows in a table.
We developed a multi tera-byte, multi-server set of related databases,
and one of the tables consists of almost 400M of rows. The database has
the public web access, and you can see it for yourself how fast you can
locate a data subset. The data maintenance also is not a problem, I mean
regular various dbcc runs, database dumps, etc. The only limitation
that we faced was the maximum combined size of the dataset per server,
which is 8TB (256 devices * 32GB), so we had to distribute the data among
multiple servers when the time came. It is on-line most of the time, and
goes off-line for a couple of hours 6 times a year. Here is the link to
that archive <www.ncbi.nlm.nih.gov/Traces>. Go to 'Statistics' pages to get
some counters. So, not to worry, Sybase would not be a problem here.
Hope it helps. Vlad.

Quote:
Thanks!


Quote:
--

Jim Douglas
http:\\www.genesis-software.com

Latitude 32.5818
Longitude -96.5412
Elevation 497






Reply With Quote
  #3  
Old   
Jim Douglas
 
Posts: n/a

Default Re: Table Size- 50-100Million Rows Will It Work! - 04-15-2004 , 04:56 PM



Thanks for the response. I went to your web site and ran some queries, it
was great. Last question is what type of hardware is this DB on, and memory
it has!

Thanks Again!

"Vlad" <fake (AT) address (DOT) com> wrote

Quote:
Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
I am looking at some new system requirements where we will have one
table
with 50M rows and another with 100M.
We debated all afternoon about various methods of how we should handle
these
tables.

DataEntry will be done with the primarykey and reading is the same with
the
same primarykey. Reporting is something else and is a requirement. I am
not
a DBA type and was wondering how others are handling these large tables,
some folks are saying break them up into smaller subset tables, use
Sybase
distributed tables ( is there such a thing), and others are indicating
that
Sybase might not be the best choice for database with tables this size.

Any input is appreciated. What would you do? How would you do it?

Sybase is quite capable to hold reliably a 100M of rows in a table.
We developed a multi tera-byte, multi-server set of related databases,
and one of the tables consists of almost 400M of rows. The database has
the public web access, and you can see it for yourself how fast you can
locate a data subset. The data maintenance also is not a problem, I mean
regular various dbcc runs, database dumps, etc. The only limitation
that we faced was the maximum combined size of the dataset per server,
which is 8TB (256 devices * 32GB), so we had to distribute the data among
multiple servers when the time came. It is on-line most of the time, and
goes off-line for a couple of hours 6 times a year. Here is the link to
that archive <www.ncbi.nlm.nih.gov/Traces>. Go to 'Statistics' pages to
get
some counters. So, not to worry, Sybase would not be a problem here.
Hope it helps. Vlad.

Thanks!



--

Jim Douglas
http:\\www.genesis-software.com

Latitude 32.5818
Longitude -96.5412
Elevation 497








Reply With Quote
  #4  
Old   
Vlad
 
Posts: n/a

Default Re: Table Size- 50-100Million Rows Will It Work! - 04-16-2004 , 09:33 AM



Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
Quote:
Thanks for the response. I went to your web site and ran some queries, it
was great. Last question is what type of hardware is this DB on, and memory
it has!
I seem answered already, but the message is not shown on my server. Anyway,
Here it goes again.

Sun 420 with 4 450 MHz sparcs/slugs and 4 GB of memory.

Quote:
Thanks Again!

"Vlad" <fake (AT) address (DOT) com> wrote in message
news:407e07fb$0$2758$61fed72c (AT) news (DOT) rcn.com...
Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
I am looking at some new system requirements where we will have one
table
with 50M rows and another with 100M.
We debated all afternoon about various methods of how we should handle
these
tables.

DataEntry will be done with the primarykey and reading is the same with
the
same primarykey. Reporting is something else and is a requirement. I am
not
a DBA type and was wondering how others are handling these large tables,
some folks are saying break them up into smaller subset tables, use
Sybase
distributed tables ( is there such a thing), and others are indicating
that
Sybase might not be the best choice for database with tables this size.

Any input is appreciated. What would you do? How would you do it?

Sybase is quite capable to hold reliably a 100M of rows in a table.
We developed a multi tera-byte, multi-server set of related databases,
and one of the tables consists of almost 400M of rows. The database has
the public web access, and you can see it for yourself how fast you can
locate a data subset. The data maintenance also is not a problem, I mean
regular various dbcc runs, database dumps, etc. The only limitation
that we faced was the maximum combined size of the dataset per server,
which is 8TB (256 devices * 32GB), so we had to distribute the data among
multiple servers when the time came. It is on-line most of the time, and
goes off-line for a couple of hours 6 times a year. Here is the link to
that archive <www.ncbi.nlm.nih.gov/Traces>. Go to 'Statistics' pages to
get
some counters. So, not to worry, Sybase would not be a problem here.
Hope it helps. Vlad.

Thanks!



--

Jim Douglas
http:\\www.genesis-software.com

Latitude 32.5818
Longitude -96.5412
Elevation 497








Reply With Quote
  #5  
Old   
Jim Douglas
 
Posts: n/a

Default Re: Table Size- 50-100Million Rows Will It Work! - 04-16-2004 , 04:56 PM



Can you email me offline, we have a DBA type that would like to discuss
large database with you, if you don't mind. Thanks.

james.douglas (AT) genesis-software (DOT) com


"Vlad" <fake (AT) address (DOT) com> wrote

Quote:
Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
Thanks for the response. I went to your web site and ran some queries,
it
was great. Last question is what type of hardware is this DB on, and
memory
it has!

I seem answered already, but the message is not shown on my server.
Anyway,
Here it goes again.

Sun 420 with 4 450 MHz sparcs/slugs and 4 GB of memory.

Thanks Again!

"Vlad" <fake (AT) address (DOT) com> wrote in message
news:407e07fb$0$2758$61fed72c (AT) news (DOT) rcn.com...
Jim Douglas <james.douglas (AT) genesis-software (DOT) com> wrote:
I am looking at some new system requirements where we will have one
table
with 50M rows and another with 100M.
We debated all afternoon about various methods of how we should
handle
these
tables.

DataEntry will be done with the primarykey and reading is the same
with
the
same primarykey. Reporting is something else and is a requirement. I
am
not
a DBA type and was wondering how others are handling these large
tables,
some folks are saying break them up into smaller subset tables, use
Sybase
distributed tables ( is there such a thing), and others are
indicating
that
Sybase might not be the best choice for database with tables this
size.

Any input is appreciated. What would you do? How would you do it?

Sybase is quite capable to hold reliably a 100M of rows in a table.
We developed a multi tera-byte, multi-server set of related databases,
and one of the tables consists of almost 400M of rows. The database
has
the public web access, and you can see it for yourself how fast you
can
locate a data subset. The data maintenance also is not a problem, I
mean
regular various dbcc runs, database dumps, etc. The only limitation
that we faced was the maximum combined size of the dataset per server,
which is 8TB (256 devices * 32GB), so we had to distribute the data
among
multiple servers when the time came. It is on-line most of the time,
and
goes off-line for a couple of hours 6 times a year. Here is the link
to
that archive <www.ncbi.nlm.nih.gov/Traces>. Go to 'Statistics' pages
to
get
some counters. So, not to worry, Sybase would not be a problem here.
Hope it helps. Vlad.

Thanks!



--

Jim Douglas
http:\\www.genesis-software.com

Latitude 32.5818
Longitude -96.5412
Elevation 497










Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.