"Communications of the ACM" had an interesting article on "big data" this
month.

The author created a fake database of 6.75 billion rows (100 GB)
representing the human population on earth and issued this query:

SELECT country,age,sex,count(*)
FROM people
GROUP BY country,age,sex;

On an eight-core Mac Pro workstation with 20GB RAM and two terabytes of
RAID 0 disk it took 24 hours, not least because Postgres decided to sort
the table on the GROUP BY columns first.*

How does your server do?

--jkl

It would be interesting to run this through the vectorwise engine/solution
to see how it performs ..

--rdh

-----Original Message-----
From: info-ingres-bounces (AT) kettleriver...ting (DOT) com
[mailto:info-ingres-bounces (AT) kettleriverconsulting (DOT) com] On Behalf Of James K.
Lowden
Sent: Wednesday, 5 August 2009 4:16 p.m.
To: Info-Ingres
Subject: [Info-Ingres] big data

"Communications of the ACM" had an interesting article on "big data" this
month.

The author created a fake database of 6.75 billion rows (100 GB)
representing the human population on earth and issued this query:

SELECT country,age,sex,count(*)
FROM people
GROUP BY country,age,sex;

On an eight-core Mac Pro workstation with 20GB RAM and two terabytes of
RAID 0 disk it took 24 hours, not least because Postgres decided to sort
the table on the GROUP BY columns first.*

How does your server do?

--jkl


_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://www.kettleriverconsulting.com...fo/info-ingres

On Aug 5, 2009, at 12:15 AM, James K. Lowden wrote:


I'm not going to try it, but I'd expect a hash aggregation. With
optimizer stats on the group by columns, so that Ingres can make a
reasonable guess at the output cardinality, it ought to be able to
aggregate the whole thing in memory without spillage.

Karl

I thought I'd try this with Ingres. According to the article they
only had 1 billion 16 byte rows. On a cheap 2.80GHz pentium
with 2Gb ram and Ingres Linux Version II 9.3.0 the query
took 105 minutes.

Mike


On Aug 5, 6:15*am, "James K. Lowden" <jklow... (AT) schemamania (DOT) org> wrote: