dbTalk Databases Forums  

Execution plans for cursor FOR loops

comp.databases.ibm-db2 comp.databases.ibm-db2


Discuss Execution plans for cursor FOR loops in the comp.databases.ibm-db2 forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
jefftyzzer
 
Posts: n/a

Default Execution plans for cursor FOR loops - 06-30-2009 , 04:53 PM






Colleagues,

I'm trying to troubleshoot some performance issues pertaining to an
SP. Despite setting/calling

CALL SET_ROUTINE_OPTS(GET_ROUTINE_OPTS()||'EXPLAIN ALL');
SET CURRENT EXPLAIN MODE EXPLAIN;

before executing the SP, none of the SQL associated with any of the
cursor FOR loops within it is showing up in the EXPLAIN tables and
therefore in db2exfmt. Note that all of the other (dynamic) SQL in the
SP is explained.

Here's a snippet of one of my cursor FOR loops, in case it helps:

FOR L_DUP_IDS AS
SELECT
ID
FROM
SESSION.A_TABLE
GROUP BY
ID
HAVING
COUNT(REP) > 1

DO

SET V_MAX_REP = (SELECT MAX(E.REP) FROM SESSION.A_TABLE E
WHERE E._ID = L_DUP_IDS.ID);--

....

Anyone know how I can solve this?

Thanks,

--Jeff

Reply With Quote
  #2  
Old   
Serge Rielau
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 06-30-2009 , 09:33 PM






jefftyzzer wrote:
Quote:
Colleagues,

I'm trying to troubleshoot some performance issues pertaining to an
SP. Despite setting/calling

CALL SET_ROUTINE_OPTS(GET_ROUTINE_OPTS()||'EXPLAIN ALL');
SET CURRENT EXPLAIN MODE EXPLAIN;

before executing the SP, none of the SQL associated with any of the
cursor FOR loops within it is showing up in the EXPLAIN tables and
therefore in db2exfmt. Note that all of the other (dynamic) SQL in the
SP is explained.

Here's a snippet of one of my cursor FOR loops, in case it helps:

FOR L_DUP_IDS AS
SELECT
ID
FROM
SESSION.A_TABLE
GROUP BY
ID
HAVING
COUNT(REP) > 1

DO

SET V_MAX_REP = (SELECT MAX(E.REP) FROM SESSION.A_TABLE E
WHERE E._ID = L_DUP_IDS.ID);--

...

Anyone know how I can solve this?
Since you are accessing a SESSION table you need to actually execute
(drive) the procedure before you will see the plan.
SQL referring to SESSION temps doesn't get compiled until execution time.

Cheers
Serge

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Reply With Quote
  #3  
Old   
jefftyzzer
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-01-2009 , 02:42 AM



On Jun 30, 6:33*pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
Quote:
jefftyzzer wrote:
Colleagues,

I'm trying to troubleshoot some performance issues pertaining to an
SP. Despite setting/calling

CALL SET_ROUTINE_OPTS(GET_ROUTINE_OPTS()||'EXPLAIN ALL');
SET CURRENT EXPLAIN MODE EXPLAIN;

before executing the SP, none of the SQL associated with any of the
cursor FOR loops within it is showing up in the EXPLAIN tables and
therefore in db2exfmt. Note that all of the other (dynamic) SQL in the
SP is explained.

Here's a snippet of one of my cursor FOR loops, in case it helps:

FOR L_DUP_IDS AS
* * SELECT
* * * * ID
* * FROM
* * * * SESSION.A_TABLE
* * GROUP BY
* * * * ID
* * HAVING
* * * * COUNT(REP) > 1

* * DO

* * * * SET V_MAX_REP = (SELECT MAX(E.REP) FROM SESSION.A_TABLE E
WHERE E._ID = L_DUP_IDS.ID);--

...

Anyone know how *I can solve this?
Since you are accessing a SESSION table you need to actually execute

(drive) the procedure before you will see the plan.
SQL referring to SESSION temps doesn't get compiled until execution time.

Cheers
Serge

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab
Thank you for your reply, Serge, but forgive me: what do you mean by
"(drive) the procedure"?

--Jeff

Reply With Quote
  #4  
Old   
Serge Rielau
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-01-2009 , 08:06 PM



jefftyzzer wrote:
Quote:
On Jun 30, 6:33 pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
jefftyzzer wrote:
Colleagues,
I'm trying to troubleshoot some performance issues pertaining to an
SP. Despite setting/calling
CALL SET_ROUTINE_OPTS(GET_ROUTINE_OPTS()||'EXPLAIN ALL');
SET CURRENT EXPLAIN MODE EXPLAIN;
before executing the SP, none of the SQL associated with any of the
cursor FOR loops within it is showing up in the EXPLAIN tables and
therefore in db2exfmt. Note that all of the other (dynamic) SQL in the
SP is explained.
Here's a snippet of one of my cursor FOR loops, in case it helps:
FOR L_DUP_IDS AS
SELECT
ID
FROM
SESSION.A_TABLE
GROUP BY
ID
HAVING
COUNT(REP) > 1
DO
SET V_MAX_REP = (SELECT MAX(E.REP) FROM SESSION.A_TABLE E
WHERE E._ID = L_DUP_IDS.ID);--
...
Anyone know how I can solve this?
Since you are accessing a SESSION table you need to actually execute
(drive) the procedure before you will see the plan.
SQL referring to SESSION temps doesn't get compiled until execution time.

Cheers
Serge

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Thank you for your reply, Serge, but forgive me: what do you mean by
"(drive) the procedure"?

--Jeff
CALL proc();


--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Reply With Quote
  #5  
Old   
jefftyzzer
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-02-2009 , 06:41 PM



On Jul 1, 5:06*pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
Quote:
jefftyzzer wrote:
On Jun 30, 6:33 pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
jefftyzzer wrote:
Colleagues,
I'm trying to troubleshoot some performance issues pertaining to an
SP. Despite setting/calling
CALL SET_ROUTINE_OPTS(GET_ROUTINE_OPTS()||'EXPLAIN ALL');
SET CURRENT EXPLAIN MODE EXPLAIN;
before executing the SP, none of the SQL associated with any of the
cursor FOR loops within it is showing up in the EXPLAIN tables and
therefore in db2exfmt. Note that all of the other (dynamic) SQL in the
SP is explained.
Here's a snippet of one of my cursor FOR loops, in case it helps:
FOR L_DUP_IDS AS
* * SELECT
* * * * ID
* * FROM
* * * * SESSION.A_TABLE
* * GROUP BY
* * * * ID
* * HAVING
* * * * COUNT(REP) > 1
* * DO
* * * * SET V_MAX_REP = (SELECT MAX(E.REP) FROM SESSION.A_TABLE E
WHERE E._ID = L_DUP_IDS.ID);--
...
Anyone know how *I can solve this?
Since you are accessing a SESSION table you need to actually execute
(drive) the procedure before you will see the plan.
SQL referring to SESSION temps doesn't get compiled until execution time.

Cheers
Serge

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Thank you for your reply, Serge, but forgive me: what do you mean by
"(drive) the procedure"?

--Jeff

CALL proc();

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab
Thanks. Another question related to cursor FOR LOOPs.

Consider this completely contrived example:

FOR L_REPS AS
SELECT
X,
Y
FROM
Z

DO
UPDATE A SET C = N WHERE A.C = L_REPS.X;--

UPDATE B SET C = N WHERE B.C = L_REPS.X;--

UPDATE C SET C = N WHERE C.C = L_REPS.X;--

UPDATE Z SET C = L_REPS.Y WHERE Z.C = L_REPS.X;--

END FOR;--

Again, this is not the real SQL but (for very good reasons) it follows
the same pattern: cursor FOR LOOPs (sometimes one nested inside
another) doing thousands of index-based updates against large-ish (say
500K-1M rows) DGTTs using host variables (e.g., the L_REPS.X and
L_REPS.Y).

We are seeing a HUGE (from minutes to hours) negative performance
impact on the SP when the package this SQL is part of is bound with
REOPT ALWAYS vs. being left at the default. Is it reasonable to you
that recompiling the access plan with each iteration (right?) of the
loop could have such a dramatic effect? Can anyone ballpark the
general overhead of the recompilation of small updates like the ones
above?

Environment: "DB2 v8.1.1.96", "s050811", "U803920", and FixPak "10";
AIX 5.3)

Thanks,

--Jeff

Reply With Quote
  #6  
Old   
Serge Rielau
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-02-2009 , 08:02 PM



For such a simple single row UPDATE? Absolutely reasonable.
The compiler can easily dominate the cost for such a simple statement.
I am somewhat surprised that you have REOPT ALWAYS as default.

Also I think there is a lot of room to improve this code.
One way would be to use a dynamic compound instead (that would require
you do use dynamic SQL). Essentially te FOR LOOP and UPDATE stay the
same, but you wrap it into a BEGIN ATOMIC block and EXECUTE [IMMEDIATE].
As a result the entire loop gets poured into a single plan and should
execute much faster.

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Reply With Quote
  #7  
Old   
jefftyzzer
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-02-2009 , 08:20 PM



On Jul 2, 5:02*pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
Quote:
For such a simple single row UPDATE? Absolutely reasonable.
The compiler can easily dominate the cost for such a simple statement.
I am somewhat surprised that you have REOPT ALWAYS as default.

Also I think there is a lot of room to improve this code.
One way would be to use a dynamic compound instead (that would require
you do use dynamic SQL). Essentially te FOR LOOP and UPDATE stay the
same, but you wrap it into a BEGIN ATOMIC block and EXECUTE [IMMEDIATE].
As a result the entire loop gets poured into a single plan and should
execute much faster.

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab
Sorry--REOPT ONCE is the default, but when the package is instead
rebound with ALWAYS, calamity ensues. As to your suggested
improvement, I'm all ears. As I mentioned, the SQL I included here is
just meant to facilitate discussion. If it's OK with you, I'd like to
email you an example of the real SQL/PL, as I don't completely
understand your recommendation.

--Jeff

Reply With Quote
  #8  
Old   
jefftyzzer
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-02-2009 , 08:59 PM



On Jul 2, 5:20*pm, jefftyzzer <jefftyz... (AT) sbcglobal (DOT) net> wrote:
Quote:
On Jul 2, 5:02*pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:

For such a simple single row UPDATE? Absolutely reasonable.
The compiler can easily dominate the cost for such a simple statement.
I am somewhat surprised that you have REOPT ALWAYS as default.

Also I think there is a lot of room to improve this code.
One way would be to use a dynamic compound instead (that would require
you do use dynamic SQL). Essentially te FOR LOOP and UPDATE stay the
same, but you wrap it into a BEGIN ATOMIC block and EXECUTE [IMMEDIATE]..
As a result the entire loop gets poured into a single plan and should
execute much faster.

--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Sorry--REOPT ONCE is the default, but when the package is instead
rebound with ALWAYS, calamity ensues. As to your suggested
improvement, I'm all ears. As I mentioned, the SQL I included here is
just meant to facilitate discussion. If it's OK with you, I'd like to
email you an example of the real SQL/PL, as I don't completely
understand your recommendation.

--Jeff
Before I spam your Inbox--and for the benefit of other posters--let me
see if this is what you mean, using a modified example from the v9
docs:

DECLARE V_SQL VARCHAR(40);--

BEGIN ATOMIC
FOR row AS
SELECT pk, c1 FROM source
DO
SET V_SQL = 'UPDATE SESSION.T SET C = '||ROW.C1||' WHERE X= '||
ROW.PK;--
EXECUTE IMMEDIATE V_SQL;--
END FOR;--
END;

Again, this code is just an example for pedagogical purposes only--I
grant that there are better (set-oriented) ways to update table A
based on values from B.

Thanks again,

--Jeff

Reply With Quote
  #9  
Old   
Serge Rielau
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-03-2009 , 02:44 AM



DECLARE txt VARCHAR(32000);
SET txt = 'BEING ATOMIC '
Quote:
| 'FOR L_REPS AS '
| ' SELECT X, Y FROM Z '
| ' DO '
| ' UPDATE A SET C = N WHERE A.C = L_REPS.X; '
| ' UPDATE B SET C = N WHERE B.C = L_REPS.X; '
| ' UPDATE C SET C = N WHERE C.C = L_REPS.X; '
| ' UPDATE Z SET C = L_REPS.Y WHERE Z.C = L_REPS.X; '
| ' END FOR; END'
EXECUTE IMMEDIATE txt;
If you EXPLAIN PLAN on this beats you should see a join between the FOR
LOOP query and either a FILTER or a UNION operator where each arm is an
UPDATE.

Cheers
Serge
--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab

Reply With Quote
  #10  
Old   
jefftyzzer
 
Posts: n/a

Default Re: Execution plans for cursor FOR loops - 07-03-2009 , 02:07 PM



On Jul 2, 11:44*pm, Serge Rielau <srie... (AT) ca (DOT) ibm.com> wrote:
Quote:
DECLARE txt VARCHAR(32000);
SET txt = 'BEING ATOMIC '
|| 'FOR L_REPS AS '
|| ' SELECT X, Y FROM Z '
|| ' DO '
|| ' * *UPDATE A SET C = N WHERE A.C = L_REPS.X; '
|| ' * *UPDATE B SET C = N WHERE B.C = L_REPS.X; '
|| ' * *UPDATE C SET C = N WHERE C.C = L_REPS.X; '
|| ' * *UPDATE Z SET C = L_REPS.Y WHERE Z.C = L_REPS.X; '
|| ' END FOR; END'

EXECUTE IMMEDIATE txt;
If you EXPLAIN PLAN on this beats you should see a join between the FOR
LOOP query and either a FILTER or a UNION operator where each arm is an
UPDATE.

Cheers
Serge
--
Serge Rielau
SQL Architect DB2 for LUW
IBM Toronto Lab
Incredible. I had no idea that you could put (procedural) control
structures in dynamic SQL like that. I'll let you know how it pans
out.

Thanks very much,

--Jeff

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.