dbTalk Databases Forums  

[BUGS] Problem with large number of index conditions

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss [BUGS] Problem with large number of index conditions in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Brian Herlihy
 
Posts: n/a

Default [BUGS] Problem with large number of index conditions - 08-05-2005 , 12:38 AM






Hi,

I posted this to pgsql-performance a month ago, but have yet to receive a
response, so I am trying this list instead.

I am using PostgreSQL 7.4.7

I have a query looking something like this:

SELECT * FROM tbl WHERE (n = 0::smallint AND (string = 'Hello' OR string =
'Pineapple' OR string = 'Kitten' OR ..... string = 'Potato));

There is an index on (n, string) as well as an index on (string). The casting
of n allows use of the (n, string) index.

The problem is that once the number of 'OR' conditions goes past around 500,
postgresql will revert to doing a sequential scan with a filter. This kills
performance, as the table has 3 million rows. Setting enable_seqscan to OFF
does not affect the query plan.

I have solved the problem by breaking the query into sections, as follows:

SELECT * FROM tbl WHERE (n = 0 AND (string = 'Hello' OR string = 'Pineapple' OR
string = 'Kitten' OR ..... string = 'Piano'))
UNION
SELECT * FROM tbl WHERE (n = 0 AND (string = 'Panic' OR ... OR string =
'Potato));

where each subquery has no more then 200 conditions. Each subquery uses the
index, and then the results are unioned. This takes a few seconds to run,
whereas the sequential scan can take an hour (on a loaded system).

Is there any better solution?

Thanks,
Brian Herlihy

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.