SQLServer Table Partitioning

#1

Hi!

I have a question:

I already have a DB that uses partitions to divide data in US
Counties, partitioned by state.

Can I use TWO levels of partitioning?

I mean... 3077 filegroups and 50 partition functions that address
them, but can I use another function to group the 50 states?

Thanks!

Piero

#2

Piero 'Giops' Giorgi (giorgi.piero (AT) gmail (DOT) com) writes:

Quote:

I already have a DB that uses partitions to divide data in US
Counties, partitioned by state.

Can I use TWO levels of partitioning?

I mean... 3077 filegroups and 50 partition functions that address
them, but can I use another function to group the 50 states?

Do I understand it correctly that you already have 50 partitions, and
now you want even more? About what size do you expect per partition?

I'm not sure that partitioning by state is the best strategy. The partition
for Californina will be a lot bigger than the ones for Alaska and Rhode
Island.

--
Erland Sommarskog, SQL Server MVP, esquel (AT) sommarskog (DOT) se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx

#3

Quote:

Do I understand it correctly that you already have 50 partitions, and
now you want even more? About what size do you expect per partition?

I'm not sure that partitioning by state is the best strategy. The partition
for California will be a lot bigger than the ones for Alaska and Rhode
Island.

I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.

The best way to do that would be having a table partitioned over 3077
filegroups, so storing the data will go by COUNTY in this way:

CA_ALAMEDA
CA_ALPINE
CA_AMADOR
CA_BUTTE
CA_CALAVERAS
CA_COLUSA
CA_CONTRA_COSTA
CA_DEL_NORTE
CA_EL_DORADO
CA_FRESNO
CA_GLENN
CA_HUMBOLDT
CA_IMPERIAL
CA_INYO

With the COUNTY as the partition Parameter.

But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?

Thanks!

Piero

#4

Piero 'Giops' Giorgi (giorgi.piero (AT) gmail (DOT) com) writes:

Quote:

I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.

Deleting 8000 rows is a breeze, but deleting 3 million rows takes
some resources, particularly if the rows are wide. But it still only
a matter of minutes.

Quote:

But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?

No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.

--
Erland Sommarskog, SQL Server MVP, esquel (AT) sommarskog (DOT) se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx

#5

On Jun 5, 3:24 pm, Erland Sommarskog <esq... (AT) sommarskog (DOT) se> wrote:

Quote:

But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?

No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.

Dang it... I kew there was a catch.

So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?

I mean Filegroups Counties (3077) - grouped by state (50) - all
together in ONE partitioned table.

Any Ideas?

Thank you!

Piero

#6

Erland Sommarskog wrote:

Quote:

Piero 'Giops' Giorgi (giorgi.piero (AT) gmail (DOT) com) writes:
I know that, but partitioning by county makes the DB a lot easier to
maintain.
I have to work that way because I'm dealing with criminal records, and
they are separated by county with a ton of different files, so for
many of them I have to clear the table and reload the whole county
every time I get an update. Easier on partitions... :-)

Table size can be anywhere from 8000 to 3 million records, depending
on the county.

Deleting 8000 rows is a breeze, but deleting 3 million rows takes
some resources, particularly if the rows are wide. But it still only
a matter of minutes.

I do assume that (state, county) is an index. If not, then get
that fixed yesterday.

#7

Quote:

So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?

Why do you need separate filegoups? It seems to me that the main purpose of
partitioning here is for manageability and all those files/filegroups only
add to administration complexity and wasted space.

You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by county.
This approach would leverage partitioning to quickly reload individual
counties yet provide a seamless view of the entire country.

--
Hope this helps.

Dan Guzman
SQL Server MVP

"Piero 'Giops' Giorgi" <giorgi.piero (AT) gmail (DOT) com> wrote

Quote:

On Jun 5, 3:24 pm, Erland Sommarskog <esq... (AT) sommarskog (DOT) se> wrote:

But, before trying, can I have 3077 files in ONE partition, and drop
all the states stuff?

No, in the topic for CREATE PARTITION FUNCTION, I found that you
cannot have more than 999 boundary values.

Dang it... I kew there was a catch.

So, I'll be forced to have 3077 filegroups, grouped with 50
partitions.
Is there a way to have a partition function/scheme that sees other
schemes, instead of filegroups?

I mean Filegroups Counties (3077) - grouped by state (50) - all
together in ONE partitioned table.

Any Ideas?

Thank you!

Piero

#8

Quote:

You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by county.
This approach would leverage partitioning to quickly reload individual
counties yet provide a seamless view of the entire country.

THANK YOU!
That is exactly what I want to do, but unfortunately I'm not (YET)
able to do it.

How can I have a partitioned view of partitioned tables?
I have the 50 state tables partitioned by county, but I can't get to
the next step.

Can someone post a small example of the thing?

Thanks

Piero

#9

On Jun 5, 10:02 pm, Ed Murphy <emurph... (AT) socal (DOT) rr.com> wrote:

Quote:

I do assume that (state, county) is an index. If not, then get
that fixed yesterday.

Of course!
Actually it was fixes the day BEFORE yesterday... :-)

Piero

#10

Piero 'Giops' Giorgi (giorgi.piero (AT) gmail (DOT) com) writes:

Quote:

You might consider a hybrid solution with 50 individual state tables
included in a partitioned view, with each state table partitioned by
county. This approach would leverage partitioning to quickly reload
individual counties yet provide a seamless view of the entire country.

That is exactly what I want to do, but unfortunately I'm not (YET)
able to do it.

How can I have a partitioned view of partitioned tables?
I have the 50 state tables partitioned by county, but I can't get to
the next step.

Can someone post a small example of the thing?

To me that sounds like a managability nightmare. While you can query
the beast in one query, when you need to flush the rows for Orange
County, you would have to explicitly to go to the CA table to
switch partitions, which would mean a lot of dynamic SQL.

I don't know if there is any catch with partition views over partitioned
tables (I really need to find some time to play with partitioned tables
to learn them!), but in a normal partitioned view you would have:

CREATE TABLE CA (state char(2) DEFAULT 'CA' CHECK (state = 'CA'),
-- other columns
PRIMARY KEY (state, county, whatever))

CREATE TABLE RI (state char(2) DEFAULT 'RI' CHECK (state = 'RI'),
...

CREATE VIEW thewholebunch AS
SELECT state, county, .....
FROM CA
UNION ALL
SELECT state, county, .....
FROM RI
....

But personally I would look into make the merging of new files more
effective than just dropping all existing rows.

--
Erland Sommarskog, SQL Server MVP, esquel (AT) sommarskog (DOT) se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx