dbTalk Databases Forums  

Common Table Expression slow

comp.databases.ms-sqlserver comp.databases.ms-sqlserver


Discuss Common Table Expression slow in the comp.databases.ms-sqlserver forum.



Reply
 
Thread Tools Display Modes
  #11  
Old   
Hugo Kornelis
 
Posts: n/a

Default Re: Common Table Expression slow - 10-20-2010 , 10:00 AM






On Tue, 19 Oct 2010 17:54:22 -0700, tshad wrote:

Quote:
I had looked at CTE as just another way to write a derived table (when not
dealing with recursion).

I wanted to see if when I unwound the procedure if it would run the same. I
tried to do the following but I get an error on MyCTE.

SELECT RetailerName 'Retailer'
,sQuestionText 'Question'
,sChoiceText 'Response Choice'
-- ,COUNT(*)
FROM (
SELECT ...
) MyCTE
WHERE (@Status = 'ALL') OR ((@Status <> 'ALL') AND (dStart = (Select
MAX(dStart)
from MyCTE
WHERE MyCTE.RetailerName = RetailerName and
MyCTE.StoreID = StoreID and
MyCTE.nQuestionID = nQuestionID)))

The error I get is Invalid object name: MyCTE.

I thought you could refer to the derived table in my where clause but
apparently not.
You can use the alias of the derived table as a qualifier for column
names, but not as a table in the FROM clause of a subquery. Just as you
can't write
FROM SomeTable AS a
WHERE xyz = (SELECT ... FROM a WHERE ...)
but have to repeat the table name there.

This is one of the reasons why CTEs were invented, to enable you to
refer to the same subquery multiple times.

--
Hugo Kornelis, SQL Server MVP
My SQL Server blog: http://sqlblog.com/blogs/hugo_kornelis

Reply With Quote
  #12  
Old   
Andrew Morton
 
Posts: n/a

Default Re: Common Table Expression slow - 10-21-2010 , 03:18 AM






Hugo Kornelis wrote:
Quote:
On Tue, 19 Oct 2010 16:25:25 -0700, tshad wrote:


"Andrew Morton" wrote
tshad wrote:
WHERE (@Status = 'ALL') OR ((@Status <> 'ALL') AND (dStart =
(Select ...

Unless my brain deceives me at this early hour,
A OR (NOT A AND B) is the same as A OR B.
Umm... would I be wrong if @Status was NULL?

--
Andrew

Reply With Quote
  #13  
Old   
Hugo Kornelis
 
Posts: n/a

Default Re: Common Table Expression slow - 10-21-2010 , 10:26 AM



On Thu, 21 Oct 2010 09:18:02 +0100, Andrew Morton wrote:

Quote:
Hugo Kornelis wrote:
On Tue, 19 Oct 2010 16:25:25 -0700, tshad wrote:


"Andrew Morton" wrote
tshad wrote:
WHERE (@Status = 'ALL') OR ((@Status <> 'ALL') AND (dStart =
(Select ...

Unless my brain deceives me at this early hour,
A OR (NOT A AND B) is the same as A OR B.

Umm... would I be wrong if @Status was NULL?
Yes, you would. If @Status is NULL, both @Status = 'ALL' and @Status <>
'ALL' evaluate to Unknown.
Unknown AND (dStart = ....) evaluates to either False or Unknown
(depends on the result of the second condition
Unknown OR (Unknown or False) evaluates to Unknown.

And since Unknown means the row is rejected, nothing will ever be
returned from this query when @Status is NULL.

I can't believe I overlooked that.

--
Hugo Kornelis, SQL Server MVP
My SQL Server blog: http://sqlblog.com/blogs/hugo_kornelis

Reply With Quote
  #14  
Old   
Erland Sommarskog
 
Posts: n/a

Default Re: Common Table Expression slow - 10-21-2010 , 05:10 PM



tshad (tfs (AT) dslextreme (DOT) com) writes:
Quote:
WHERE (@sRecordStatus = 'ALL') OR ((@sRecordStatus <> 'ALL') AND (dStart =
(Select MAX(dStart)
from @tfs
WHERE @tfs.Retailer = Retailer and
@tfs.StoreID = StoreID and
@tfs.QuestionID = QuestionID)))
Hugo explained why this does not work, and also provided the best
solution. But for completeness sake, I like to add that you can
use the table variable directly as a prefix, but you need to quote it:

from @tfs
WHERE [@tfs].Retailer = Retailer and
[@tfs].StoreID = StoreID and
[@tfs].QuestionID = QuestionID)))



--
Erland Sommarskog, SQL Server MVP, esquel (AT) sommarskog (DOT) se

Links for SQL Server Books Online:
SQL 2008: http://msdn.microsoft.com/en-us/sqlserver/cc514207.aspx
SQL 2005: http://msdn.microsoft.com/en-us/sqlserver/bb895970.aspx

Reply With Quote
  #15  
Old   
tshad
 
Posts: n/a

Default Re: Common Table Expression slow - 12-05-2010 , 01:31 PM



"--CELKO--" <jcelko212 (AT) earthlink (DOT) net> wrote

Quote:
I thought the CTE actually creates temp table internally.

Not in SQL Server. It is an in-line macro and the recursive CTEs are
cursors and procedural code in disguise. DB2 and Oracle optimizers can
make a decision to materialize or not materialize a CTE.
That may be the case, but in a recursion case I find that the CTE works
better (faster) than cursors or recursive rountines.

Reply With Quote
  #16  
Old   
tshad
 
Posts: n/a

Default Re: Common Table Expression slow - 12-05-2010 , 01:53 PM



I have found lately, that CTE's can be a problem in many cases.

I have read the it is
"Erland Sommarskog" <esquel (AT) sommarskog (DOT) se> wrote

Quote:
tshad (tfs (AT) dslextreme (DOT) com) writes:
It's just curious as to what it is doing since all the time is being
done by the SELECT after the CTE definition.

Yes, that's the way it is. The CTE is just a definition. The CTE does
nothing on it own.

It is almost like it is doing the Select in the definition for each row
in
the SELECT statement.

Yes, that is what is going on.

I would have assumed that the Select with the Temp table would work the
same as the CTE but obviously that is not the case.

There are other products that are smart enough to do this, but SQL Server
is
not there yet.

I agree there.

I started using CTEs after reading an article by John Papa where is states:

"Many projects that developers work on involve writing complex SQL
statements that deviate from basic SELECT/FROM/WHERE types of statements.
One such scenario involves writing Transact-SQL (T-SQL) queries that use
derived tables (also known as inline views) inside a FROM clause. This
common practice allows a developer to grab a rowset and immediately join
that rowset to other tables, views, and user-defined functions in a SELECT
statement. Another option is to use a view instead of a derived table. Both
of these options have their advantages and disadvantages.
When working with SQL ServerT 2005, I prefer a third option of using Common
Table Expressions (CTEs). CTEs can help improve the readability (and thus
the maintainability) of the code without compromising performance."

I am finding that this is not the case in many (most cases where I and
others on my team have been using them).

I agree with him that it is useful with recursion. I have taken changed
recursive code that took 5 minutes down to 10 seconds just by changing the
code to a CTE.

But in other cases, I am finding that changing the CTE to a temp table will
fix or speed up the query.

I also am finding that where the CTE will work fine (for months), it
sometimes quits working. After much investigation, I end up changing the
CTE to temp tables to solve the issue.

For example, we had a CTE that worked fine taking about 2 or 3 minutes to
run that was really 2 CTEs joined together. This was working for about 2
months. Then last week it started to take 30 minutes plus ending up with an
error (something lile illegal syntax near '-'). Then we would run it again
and it would take 8 minutes to run but no error. Later, it wouldn't return
at all and after about an hour and half we would just kill it. We did see
that the tempDB had increased to 120GB (not sure if this query caused it).

It looked something like:

WITH myCTE
(
SELECT ...
), myCTE2
(
SELECT ...
)
SELECT .. (joining the 2 CTEs)

We thought maybe there was something with the server but after I changed it
to: 2 SELECT/INTO statments with 2 temp tables, it worked fine and took
about a minute to run.

We also converted a couple regular CTEs that were taking about 50 seconds to
using a temp table (SELECT/INTO):

WITH myCTE
(
SELECT...
)
SELECT ...

to

SELECT/INTO

and it took about 20 seconds.

Not a lot of difference but not quite what John was saying.

My concern is taking the time to write the CTE when it may just be better to
write it the old ways.

Thanks,

Tom

Quote:
--
Erland Sommarskog, SQL Server MVP, esquel (AT) sommarskog (DOT) se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/pro...ads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinf...ons/books.mspx

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.