dbTalk Databases Forums  

[BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20"

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
aschetinin@gmail.com
 
Posts: n/a

Default [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" - 02-01-2012 , 10:14 AM






The following bug has been logged on the website:

Bug reference: 6426
Logged by: Andrew Schetinin
Email address: aschetinin (AT) gmail (DOT) com
PostgreSQL version: 9.1.2
Operating system: Debian Linux
Description:

I have a complex query that, when unlimited, runs in 2.5 seconds and returns
400+ records.

The same query, with added "LIMIT 20 OFFSET 0" at the end, runs 25 seconds!
(and returns correct 20 records)

I think there is a bug in the query optimizer. Other variations of this
query (with slightly different constraints and joins) work fast with and
without LIMIT.

The same query works fast in PostgreSQL 8.4 (tested on another machine with
Ubuntu and not exactly the same but very-very similar database).

I can provide the query execution plans of the two cases (as long as they
are not published on the Web); but not the database because there is
customer's data in it.

Please let me know if anybody is interested in researching the problem.

Regards,

Andrew Schetinin


--
Sent via pgsql-bugs mailing list (pgsql-bugs (AT) postgresql (DOT) org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Reply With Quote
  #2  
Old   
Andrew Schetinin
 
Posts: n/a

Default Re: [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" - 02-01-2012 , 02:05 PM






Hi Alex,

Thank you. I played with subqueries, but in this case I did not see a good
enough improvement.

In my specific case, what I've seen from the query execution plans, is that
without LIMIT the query uses Hash Joins, but once I add LIMIT, it starts
using Nested Loop Joins almost everywhere.

Regards,

Andrew

On Wed, Feb 1, 2012 at 9:42 PM, Alex Lai <alai (AT) sesda2 (DOT) com> wrote:

Quote:
Hi Andrew,
I posted for another post, its may give you a workaround.
I still not fully understand how PG choose execute plan that slow down so
much.

I had the same situation in one of my query.
Use the subquery can speed up almost by 100 times faster.

..........


Quote:
Hope this help.


Best Regards,
Alex Lai

--
Best regards,


Alex Lai
OMI SIPS DBA ADNET Systems , Inc. 7515 Mission Drive, Suite A100 Lanham,
MD 20706 301-352-4657 (phone) 301-352-0437 (fax) alai (AT) sesda2 (DOT) com



--
--
Andrew Schetinin

Reply With Quote
  #3  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" - 02-01-2012 , 03:48 PM



Andrew Schetinin <aschetinin (AT) gmail (DOT) com> writes:
Quote:
In my specific case, what I've seen from the query execution plans, is that
without LIMIT the query uses Hash Joins, but once I add LIMIT, it starts
using Nested Loop Joins almost everywhere.
Usually, that's an appropriate change for a small LIMIT. It's certainly
not a priori evidence of a planner bug.

If you want useful comments about this, please review
http://wiki.postgresql.org/wiki/Slow_Query_Questions
about how to provide an adequate description of your problem.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs (AT) postgresql (DOT) org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Reply With Quote
  #4  
Old   
Andrew Schetinin
 
Posts: n/a

Default Re: [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" - 02-02-2012 , 03:17 AM



Hi Tom,

Thanks for pointing the FAQ out, I did not see it.
I especially liked the link to http://explain.depesz.com - it's a useful
tool.

I succeeded to fix my problem by changing the order of JOINs (the query
remained exactly the same otherwise). According to EXPLAIN ANALIZE, it
eliminated those problematic nested loop joins. BTW, changing the order of
JOINs did not affect the times of the query without LIMIT - it works the
same way, while for the query with LIMIT it fixed the problem.

Previously I always thought that the order of JOINs or conditions in WHERE
is irrelevant, and query optimizer rearranges the order according to its
logic. Now it appears that sometimes it may be important.

Regards,

Andrew

On Wed, Feb 1, 2012 at 11:48 PM, Tom Lane <tgl (AT) sss (DOT) pgh.pa.us> wrote:

Quote:
Andrew Schetinin <aschetinin (AT) gmail (DOT) com> writes:
In my specific case, what I've seen from the query execution plans, is
that
without LIMIT the query uses Hash Joins, but once I add LIMIT, it starts
using Nested Loop Joins almost everywhere.

Usually, that's an appropriate change for a small LIMIT. It's certainly
not a priori evidence of a planner bug.

If you want useful comments about this, please review
http://wiki.postgresql.org/wiki/Slow_Query_Questions
about how to provide an adequate description of your problem.

regards, tom lane



--
--
Andrew Schetinin

Reply With Quote
  #5  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] BUG #6426: Complex query runs 10 times longer with "LIMIT 20" - 02-02-2012 , 10:40 AM



Andrew Schetinin <aschetinin (AT) gmail (DOT) com> writes:
Quote:
Previously I always thought that the order of JOINs or conditions in WHERE
is irrelevant, and query optimizer rearranges the order according to its
logic. Now it appears that sometimes it may be important.
If you have more than join_collapse_limit JOINs in the query, the order
can matter.

regards, tom lane

--
Sent via pgsql-bugs mailing list (pgsql-bugs (AT) postgresql (DOT) org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.