dbTalk Databases Forums  

Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Oliver Jowett
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 06:17 PM






Ernst Bachmann wrote:
Quote:
The following bug has been logged online:

Bug reference: 1753
Logged by: Ernst Bachmann
Email address: e.bachmann (AT) xebec (DOT) de
PostgreSQL version: 8.0.3
Operating system: Linux
Description: Query Optimizer does not work well with libpg /
PQexecParams
Details:

It looks like the query optimizer isn't taking the value of parameters sent
with PQexecParams into account, thus generating (in my case, very) unoptimal
plans
If PQexecParams uses the unnamed statement (it appears to), this
shouldn't happen -- planning of the unnamed statement is delayed until
the first set of parameter values is bound. This behaviour started in 8.0.

What's the query?

Are you definitely using PQexecParams() and not PQexecPrepared()?

-O

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo (AT) postgresql (DOT) org)


Reply With Quote
  #2  
Old   
Andrew - Supernews
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 07:04 PM






On 2005-07-05, Oliver Jowett <oliver (AT) opencloud (DOT) com> wrote:
Quote:
Ernst Bachmann wrote:
The following bug has been logged online:

Bug reference: 1753
Logged by: Ernst Bachmann
Email address: e.bachmann (AT) xebec (DOT) de
PostgreSQL version: 8.0.3
Operating system: Linux
Description: Query Optimizer does not work well with libpg /
PQexecParams
Details:

It looks like the query optimizer isn't taking the value of parameters sent
with PQexecParams into account, thus generating (in my case, very) unoptimal
plans

If PQexecParams uses the unnamed statement (it appears to), this
shouldn't happen -- planning of the unnamed statement is delayed until
the first set of parameter values is bound. This behaviour started in 8.0.
The problem is that even with the unnamed statement and deferred planning,
the planner still has to treat the parameters as variables, not constants,
since nothing in the protocol stops you from running multiple portals from
the unnamed statement. This can make a significant difference, especially
where function calls are involved and major optimizations can be made on
constant values as a result of inlining.

--
Andrew, Supernews
http://www.supernews.com - individual and corporate NNTP services

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo (AT) postgresql (DOT) org so that your
message can get through to the mailing list cleanly


Reply With Quote
  #3  
Old   
Oliver Jowett
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 07:27 PM



Andrew - Supernews wrote:

Quote:
The problem is that even with the unnamed statement and deferred planning,
the planner still has to treat the parameters as variables, not constants,
since nothing in the protocol stops you from running multiple portals from
the unnamed statement. This can make a significant difference, especially
where function calls are involved and major optimizations can be made on
constant values as a result of inlining.
Sure, expression optimization is less aggressive, but is that on its own
really going to produce a 100-fold difference in query execution?

The main problem pre-8.0 (or with named statements) is that index
selectivity estimates go out the window with a parameterized query, so a
much more general (and slower) plan gets chosen. The 8.0
unnamed-statement behaviour glues the actual parameter values into the
selectivity estimates so in theory you should get the same plan for the
unparameterized and parameterized-unnamed-statement cases.

This is why I'd like to see the actual query..

-O

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo (AT) postgresql (DOT) org


Reply With Quote
  #4  
Old   
Andrew - Supernews
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 07:53 PM



On 2005-07-06, Oliver Jowett <oliver (AT) opencloud (DOT) com> wrote:
Quote:
Andrew - Supernews wrote:
The problem is that even with the unnamed statement and deferred planning,
the planner still has to treat the parameters as variables, not constants,
since nothing in the protocol stops you from running multiple portals from
the unnamed statement. This can make a significant difference, especially
where function calls are involved and major optimizations can be made on
constant values as a result of inlining.

Sure, expression optimization is less aggressive, but is that on its own
really going to produce a 100-fold difference in query execution?
Sure. Only for specific types of queries, of course.

Quote:
The main problem pre-8.0 (or with named statements) is that index
selectivity estimates go out the window with a parameterized query, so a
much more general (and slower) plan gets chosen. The 8.0
unnamed-statement behaviour glues the actual parameter values into the
selectivity estimates
correct so far...

Quote:
so in theory you should get the same plan for the unparameterized and
parameterized-unnamed-statement cases.
But that doesn't follow, since selectivity estimation isn't the only
factor.

Quote:
This is why I'd like to see the actual query..
Yes, it would certainly help.

--
Andrew, Supernews
http://www.supernews.com - individual and corporate NNTP services

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match


Reply With Quote
  #5  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 09:50 PM



Oliver Jowett <oliver (AT) opencloud (DOT) com> writes:
Quote:
Sure, expression optimization is less aggressive, but is that on its own
really going to produce a 100-fold difference in query execution?
It's certainly possible, depending on query details. Personally, I
ignored the original report in this thread as being utterly content
free...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


Reply With Quote
  #6  
Old   
Oliver Jowett
 
Posts: n/a

Default Re: [BUGS] BUG #1753: Query Optimizer does not work well with libpg - 07-05-2005 , 10:11 PM



Tom Lane wrote:
Quote:
Oliver Jowett <oliver (AT) opencloud (DOT) com> writes:

Sure, expression optimization is less aggressive, but is that on its own
really going to produce a 100-fold difference in query execution?


It's certainly possible, depending on query details.
Andrew pointed out in some offlist discussion that it's actually more
than just expression optimization via eval_const_expressions; there are
some other cases where the tree is transformed differently if you have a
Const vs. Param in ways that estimate_expression_value() doesn't deal with:

- predicate_implied_by_simple_clause can remove parts of an expression
based on the actual (not estimated) values involved;
- match_special_index_operator manipulates LIKE/regexp/etc expressions
based on the actual patterns provided;
- LIMIT/OFFSET modifications to query cost only kick in with Consts.

I can't see any simple way to fix the first two since they change the
meaning of the expression, but the LIMIT/OFFSET query cost code could in
theory use estimate_expression_value().

-O

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.