dbTalk Databases Forums  

How to force full table scan

comp.databases.ibm-db2 comp.databases.ibm-db2


Discuss How to force full table scan in the comp.databases.ibm-db2 forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Desmodromic
 
Posts: n/a

Default How to force full table scan - 08-19-2011 , 07:10 AM






How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Reply With Quote
  #2  
Old   
Rhino
 
Posts: n/a

Default Re: How to force full table scan - 08-20-2011 , 07:01 PM






"Desmodromic" <davies_ms (AT) yahoo (DOT) com.au> wrote

Quote:
How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?
Are you concerned that the actual number of rows in the table differs from
the number of entries in the (presumably unique) index? Otherwise, I can't
imagine why you'd want to do a table scan rather than an index scan. After
all, most people want their answer faster, not slower and the index scan is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index; then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino

Reply With Quote
  #3  
Old   
Frederik Engelen
 
Posts: n/a

Default Re: How to force full table scan - 08-21-2011 , 02:18 PM



On 21 aug, 02:01, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com>
wrote:
Quote:
"Desmodromic" <davies... (AT) yahoo (DOT) com.au> wrote in message

news:9ddaea7c-f327-4497-81b3-732f9237a6f0 (AT) t30g2000prm (DOT) googlegroups.com...





How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Are you concerned that the actual number of rows in the table differs from
the number of entries in the (presumably unique) index? Otherwise, I can't
imagine why you'd want to do a table scan rather than an index scan. After
all, most people want their answer faster, not slower and the index scan is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index; then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino
Count through an index is not always 100% accurate if updates are
occurring on the index keys (looks like a delete/insert for the
index). that might be a reason.

Are optimization profiles an option? That should help:

<OPTGUIDELINES>
<TBSCAN TABLE='t1'/>
</OPTGUIDELINES>

--
Frederik Engelen

Reply With Quote
  #4  
Old   
Rhino
 
Posts: n/a

Default Re: How to force full table scan - 08-21-2011 , 03:26 PM



"Frederik Engelen" <engelenfrederik (AT) gmail (DOT) com> wrote

Quote:
On 21 aug, 02:01, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:
"Desmodromic" <davies... (AT) yahoo (DOT) com.au> wrote in message

news:9ddaea7c-f327-4497-81b3-732f9237a6f0 (AT) t30g2000prm (DOT) googlegroups.com...





How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Are you concerned that the actual number of rows in the table differs
from
the number of entries in the (presumably unique) index? Otherwise, I
can't
imagine why you'd want to do a table scan rather than an index scan.
After
all, most people want their answer faster, not slower and the index scan
is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index;
then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino

Count through an index is not always 100% accurate if updates are
occurring on the index keys (looks like a delete/insert for the
index). that might be a reason.

Would a count(*) query be any more accurate though? It seems to me that if
an accurate count is essential, updates (meaning updates, deletes AND
inserts) be stopped for the duration of the counting. At that point, I would
like to think that using or not using the index would be irrelevant to
getting the desired result.

I wonder if Desmodromic is indeed worried about differences between the
count given by a unique index and the table itself? I dimly remember a case
I saw roughly 20 years ago where an index had four fewer (or more?) entries
than there were rows in the table. It wasn't my database so I don't know
much about what had happened or how it was resolved; I only remember that it
happened.

Quote:
Are optimization profiles an option? That should help:

OPTGUIDELINES
TBSCAN TABLE='t1'/
/OPTGUIDELINES

--
Frederik Engelen

Reply With Quote
  #5  
Old   
Desmodromic
 
Posts: n/a

Default Re: How to force full table scan - 08-21-2011 , 08:17 PM



On Aug 22, 4:26*am, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com>
wrote:
Quote:
"Frederik Engelen" <engelenfrede... (AT) gmail (DOT) com> wrote in message

news:8ea99a98-f1bc-4150-b6cc-ecb60cda24cb (AT) t29g2000vby (DOT) googlegroups.com...









On 21 aug, 02:01, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:
"Desmodromic" <davies... (AT) yahoo (DOT) com.au> wrote in message

news:9ddaea7c-f327-4497-81b3-732f9237a6f0 (AT) t30g2000prm (DOT) googlegroups.com....

How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Are you concerned that the actual number of rows in the table differs
from
the number of entries in the (presumably unique) index? Otherwise, I
can't
imagine why you'd want to do a table scan rather than an index scan.
After
all, most people want their answer faster, not slower and the index scan
is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index;
then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino

Count through an index is not always 100% accurate if updates are
occurring on the index keys (looks like a delete/insert for the
index). that might be a reason.

Would a count(*) query be any more accurate though? It seems to me that if
an accurate count is essential, updates (meaning updates, deletes AND
inserts) be stopped for the duration of the counting. At that point, I would
like to think that using or not using the index would be irrelevant to
getting the desired result.

I wonder if Desmodromic is indeed worried about differences between the
count given by a unique index and the table itself? I dimly remember a case
I saw roughly 20 years ago where an index had four fewer (or more?) entries
than there were rows in the table. It wasn't my database so I don't know
much about what had happened or how it was resolved; I only remember thatit
happened.







Are optimization profiles an option? That should help:

OPTGUIDELINES
* * <TBSCAN TABLE='t1'/
/OPTGUIDELINES

--
Frederik Engelen
Thanks for your responses. Firstly, I found a way to force a full
table scan by adding a filter on a non-indexed INTEGER NOT NULL column
such as WHERE COL1 >= 0. Since there are no NULL or negative values in
COL1 this will give me the row count of the table.

The reason why I want to force a full table scan and also get the row
count is for benchmarking and so I can calculate the number of rows
scanned per second. I have a number of tables with the same structure,
similar volumes of data but different compression methods. What I have
discovered is that for those tables compressed using REORG the average
rows scanned per second is 400,000. For those tables not compressed it
is 109,000 rows scanned per second. However, for those tables
compressed using TABLE GROWTH the scan rate is a pitiful 2,400 rows
per second.

Now I have another question - how can this terrible scan performance
for tables compressed by TABLE GROWTH be explained?

Reply With Quote
  #6  
Old   
Frederik Engelen
 
Posts: n/a

Default Re: How to force full table scan - 08-22-2011 , 05:44 AM



On Aug 22, 3:17*am, Desmodromic <davies... (AT) yahoo (DOT) com.au> wrote:
Quote:
On Aug 22, 4:26*am, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:





"Frederik Engelen" <engelenfrede... (AT) gmail (DOT) com> wrote in message

news:8ea99a98-f1bc-4150-b6cc-ecb60cda24cb (AT) t29g2000vby (DOT) googlegroups.com....

On 21 aug, 02:01, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:
"Desmodromic" <davies... (AT) yahoo (DOT) com.au> wrote in message

news:9ddaea7c-f327-4497-81b3-732f9237a6f0 (AT) t30g2000prm (DOT) googlegroups.com...

How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 =col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Are you concerned that the actual number of rows in the table differs
from
the number of entries in the (presumably unique) index? Otherwise, I
can't
imagine why you'd want to do a table scan rather than an index scan.
After
all, most people want their answer faster, not slower and the index scan
is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index;
then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino

Count through an index is not always 100% accurate if updates are
occurring on the index keys (looks like a delete/insert for the
index). that might be a reason.

Would a count(*) query be any more accurate though? It seems to me thatif
an accurate count is essential, updates (meaning updates, deletes AND
inserts) be stopped for the duration of the counting. At that point, I would
like to think that using or not using the index would be irrelevant to
getting the desired result.

I wonder if Desmodromic is indeed worried about differences between the
count given by a unique index and the table itself? I dimly remember a case
I saw roughly 20 years ago where an index had four fewer (or more?) entries
than there were rows in the table. It wasn't my database so I don't know
much about what had happened or how it was resolved; I only remember that it
happened.

Are optimization profiles an option? That should help:

OPTGUIDELINES
* * <TBSCAN TABLE='t1'/
/OPTGUIDELINES

--
Frederik Engelen

Thanks for your responses. Firstly, I found a way to force a full
table scan by adding a filter on a non-indexed INTEGER NOT NULL column
such as WHERE COL1 >= 0. Since there are no NULL or negative values in
COL1 this will give me the row count of the table.

The reason why I want to force a full table scan and also get the row
count is for benchmarking and so I can calculate the number of rows
scanned per second. I have a number of tables with the same structure,
similar volumes of data but different compression methods. What I have
discovered is that for those tables compressed using REORG the average
rows scanned per second is 400,000. For those tables not compressed it
is 109,000 rows scanned per second. However, for those tables
compressed using TABLE GROWTH the scan rate is a pitiful 2,400 rows
per second.

Now I have another question - how can this terrible scan performance
for tables compressed by TABLE GROWTH be explained?- Hide quoted text -

- Show quoted text -
What do you mean by TABLE GROWTH? Automatic Dictionary Creation?

--
Frederik Engelen

Reply With Quote
  #7  
Old   
Yonghang Wang
 
Posts: n/a

Default Re: How to force full table scan - 08-22-2011 , 08:58 AM



before goes into the complex of optimization profile, try,

select * from ur_table where some_predication_always_false_but_not_just_1=0

the predication includes no column in a index.

Reply With Quote
  #8  
Old   
Desmodromic
 
Posts: n/a

Default Re: How to force full table scan - 08-22-2011 , 09:03 PM



On Aug 22, 6:44*pm, Frederik Engelen <engelenfrede... (AT) gmail (DOT) com>
wrote:
Quote:
On Aug 22, 3:17*am, Desmodromic <davies... (AT) yahoo (DOT) com.au> wrote:









On Aug 22, 4:26*am, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:

"Frederik Engelen" <engelenfrede... (AT) gmail (DOT) com> wrote in message

news:8ea99a98-f1bc-4150-b6cc-ecb60cda24cb (AT) t29g2000vby (DOT) googlegroups.com....

On 21 aug, 02:01, "Rhino" <no_offline_contact_ple... (AT) example (DOT) com
wrote:
"Desmodromic" <davies... (AT) yahoo (DOT) com.au> wrote in message

news:9ddaea7c-f327-4497-81b3-732f9237a6f0 (AT) t30g2000prm (DOT) googlegroups.com...

How can I force an SQL query to perform a full table scan ratherthan
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?

Are you concerned that the actual number of rows in the table differs
from
the number of entries in the (presumably unique) index? Otherwise,I
can't
imagine why you'd want to do a table scan rather than an index scan.
After
all, most people want their answer faster, not slower and the index scan
is
going to be faster in all but a few trivial cases....

Still, if you really must have a table scan you could delete the index;
then
you can't possibly do an index scan since DB2 can't use an index that
doesn't exist.

--
Rhino

Count through an index is not always 100% accurate if updates are
occurring on the index keys (looks like a delete/insert for the
index). that might be a reason.

Would a count(*) query be any more accurate though? It seems to me that if
an accurate count is essential, updates (meaning updates, deletes AND
inserts) be stopped for the duration of the counting. At that point, I would
like to think that using or not using the index would be irrelevant to
getting the desired result.

I wonder if Desmodromic is indeed worried about differences between the
count given by a unique index and the table itself? I dimly remember a case
I saw roughly 20 years ago where an index had four fewer (or more?) entries
than there were rows in the table. It wasn't my database so I don't know
much about what had happened or how it was resolved; I only remember that it
happened.

Are optimization profiles an option? That should help:

OPTGUIDELINES
* * <TBSCAN TABLE='t1'/
/OPTGUIDELINES

--
Frederik Engelen

Thanks for your responses. Firstly, I found a way to force a full
table scan by adding a filter on a non-indexed INTEGER NOT NULL column
such as WHERE COL1 >= 0. Since there are no NULL or negative values in
COL1 this will give me the row count of the table.

The reason why I want to force a full table scan and also get the row
count is for benchmarking and so I can calculate the number of rows
scanned per second. I have a number of tables with the same structure,
similar volumes of data but different compression methods. What I have
discovered is that for those tables compressed using REORG the average
rows scanned per second is 400,000. For those tables not compressed it
is 109,000 rows scanned per second. However, for those tables
compressed using TABLE GROWTH the scan rate is a pitiful 2,400 rows
per second.

Now I have another question - how can this terrible scan performance
for tables compressed by TABLE GROWTH be explained?- Hide quoted text -

- Show quoted text -

What do you mean by TABLE GROWTH? Automatic Dictionary Creation?

--
Frederik Engelen
Yes, I mean Automatic Dictionary Creation. The table is created with
compression on and then data is inserted over time. I got "TABLE
GROWTH" from the DICT_BUILDER column of the
SYSIBMADM.ADMINTABCOMPRESSINFO view.

Reply With Quote
  #9  
Old   
Manoj Sutar
 
Posts: n/a

Default Re: How to force full table scan - 08-29-2011 , 02:14 PM



On Aug 19, 5:10*am, Desmodromic <davies... (AT) yahoo (DOT) com.au> wrote:
Quote:
How can I force an SQL query to perform a full table scan rather than
use an index? I also need to obtain the number of rows in the table,
so all I really want to do is:

select count(1)
from table;

but by scanning the table and not an index.

I have tried a few things so far such as including "where col1 = col1
+ 0" and "select count(1) from (select * from table)".

However, the DB2 9.5 FP7 optimizer seems to be too smart for me and
always uses an index.

Any idea how I can force a full table scan and get the row count?
If you are on Unix/Linux,

db2 "select * from tablename" | tail
It will read all data from table forcing a tablescan and show last 10
lines of result.

Cheers...
Manoj

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.