dbTalk Databases Forums  

To many indexes for search feature?

comp.databases.mysql comp.databases.mysql


Discuss To many indexes for search feature? in the comp.databases.mysql forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Simon
 
Posts: n/a

Default To many indexes for search feature? - 10-19-2011 , 04:30 AM






Hi,

If I have a simple 'user' table containing some basic information.

Something like "fname, lname, displayname, age, city, country, state"
and so forth.

And, I would like the visitors to be able to search by fname, lname,
displayname, age, city, country _or_ state

so my select would be "SELECT * FROM table_user WHERE fname='xyz'"

Must I create a separate index for each "fname, lname, displayname, age,
city, country, state, ..."?
Or would a single select be enough?

Would I need to do something different if my select was using a like,
("SELECT * FROM table_user WHERE fname LIKE '%xyz%'")?

Or do you think that the design itself should be different altogether?

Many thanks

Simon

Reply With Quote
  #2  
Old   
Jerry Stuckle
 
Posts: n/a

Default Re: To many indexes for search feature? - 10-19-2011 , 05:33 AM






On 10/19/2011 5:30 AM, Simon wrote:
Quote:
Hi,

If I have a simple 'user' table containing some basic information.

Something like "fname, lname, displayname, age, city, country, state"
and so forth.

And, I would like the visitors to be able to search by fname, lname,
displayname, age, city, country _or_ state

so my select would be "SELECT * FROM table_user WHERE fname='xyz'"

Must I create a separate index for each "fname, lname, displayname, age,
city, country, state, ..."?
Or would a single select be enough?

RDBMS's can only use an index for a single column search if it's the
first column in the index. For instance, if you had an index on "lname,
fname", the RDBMS can use it for a search on lname, but not for one on
fname.

Quote:
Would I need to do something different if my select was using a like,
("SELECT * FROM table_user WHERE fname LIKE '%xyz%'")?

This will almost assuredly require a table scan.

Quote:
Or do you think that the design itself should be different altogether?

Many thanks

Simon
Think of how indexes work. The index is basically a sorted list of data
for the columns specified. This allows the RDBMS to perform a binary
search of the index to find the requested row(s).

But the sort is in the order of the column specification. In the
example above, could have the following index:

anderson, mark
doe, john
smith, don

The index could be used for a search by lname, but not fname.

Likewise, a search on "lname like '%it%'" could not use the index
(although a search on "lname like 'sm%'" could).

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex (AT) attglobal (DOT) net
==================

Reply With Quote
  #3  
Old   
Simon
 
Posts: n/a

Default Re: To many indexes for search feature? - 10-19-2011 , 06:17 AM



On 2011/10/19 12:33 PM, Jerry Stuckle wrote:
Quote:
On 10/19/2011 5:30 AM, Simon wrote:

RDBMS's can only use an index for a single column search if it's the
first column in the index. For instance, if you had an index on "lname,
fname", the RDBMS can use it for a search on lname, but not for one on
fname.

Thanks for the reply and explanation.

It would then make sense to create an index for each and every possible
select, (and also to _not_ use LIKE '%...%').

In my case, that would mean 13 new indexes.

Do you think that the design is flawed in the first place?
Should that information be in a different table structure?

Thanks

Simon

Reply With Quote
  #4  
Old   
Peter H. Coffin
 
Posts: n/a

Default Re: To many indexes for search feature? - 10-19-2011 , 09:01 AM



On Wed, 19 Oct 2011 13:17:49 +0200, Simon wrote:
Quote:
On 2011/10/19 12:33 PM, Jerry Stuckle wrote:
On 10/19/2011 5:30 AM, Simon wrote:

RDBMS's can only use an index for a single column search if it's the
first column in the index. For instance, if you had an index on "lname,
fname", the RDBMS can use it for a search on lname, but not for one on
fname.


Thanks for the reply and explanation.

It would then make sense to create an index for each and every possible
select, (and also to _not_ use LIKE '%...%').

In my case, that would mean 13 new indexes.

Do you think that the design is flawed in the first place?
Should that information be in a different table structure?
It depends. Lots of index can be a drawback if you're updating or
inserting data, because each index may have to be updated as well. So,
inserting a lot of records may take 20 or 30 times as long to do as not
having those indexes.

However, lots of indexes may not be a big deal if you aren't inserting a
lot of data all the time. If you only insert 20 records a day, who gives
a crap if it takes 0.003 seconds or it takes 0.124 seconds per? On the
rare occasions when you DO have tens of thousands of records to put in,
you can unhook the indexes, load the data, then re-add the indexs and
they get built only once instead of getting adjusted for every one.

You might also consider what people are actually searching FOR. Are they
looking for states because they have legal or jurisdictional reasons to?
Are they looking for location data for mailing? Are they looking to find
out what's close by? If people mostly only care what's close, you may
want to rethink searching by state, city, and zipcode entirely and
switch to a spatial process instead: find a nice way to get coordinates
for addresses and store and index that instead of the legal names of
places.

--
32. I will not fly into a rage and kill a messenger who brings me bad
news just to illustrate how evil I really am. Good messengers are
hard to come by.
--Peter Anspach's list of things to do as an Evil Overlord

Reply With Quote
  #5  
Old   
Brian Cryer
 
Posts: n/a

Default Re: To many indexes for search feature? - 10-19-2011 , 09:47 AM



"Peter H. Coffin" <hellsop (AT) ninehells (DOT) com> wrote

Quote:
On Wed, 19 Oct 2011 13:17:49 +0200, Simon wrote:
On 2011/10/19 12:33 PM, Jerry Stuckle wrote:
On 10/19/2011 5:30 AM, Simon wrote:

RDBMS's can only use an index for a single column search if it's the
first column in the index. For instance, if you had an index on "lname,
fname", the RDBMS can use it for a search on lname, but not for one on
fname.

Thanks for the reply and explanation.

It would then make sense to create an index for each and every possible
select, (and also to _not_ use LIKE '%...%').

In my case, that would mean 13 new indexes.

Do you think that the design is flawed in the first place?
Should that information be in a different table structure?

It depends. Lots of index can be a drawback if you're updating or
inserting data, because each index may have to be updated as well. So,
inserting a lot of records may take 20 or 30 times as long to do as not
having those indexes.
As well as time to update those indexes (which as may or may not be a big
deal), be aware of the storage requirements for each index. With one index
per field the indexes will be taking up more space than the table data. This
is probably not a significant factor for a small database, but is worth
being aware of.

Quote:
However, lots of indexes may not be a big deal if you aren't inserting a
lot of data all the time. If you only insert 20 records a day, who gives
a crap if it takes 0.003 seconds or it takes 0.124 seconds per? On the
rare occasions when you DO have tens of thousands of records to put in,
you can unhook the indexes, load the data, then re-add the indexs and
they get built only once instead of getting adjusted for every one.

You might also consider what people are actually searching FOR. Are they
looking for states because they have legal or jurisdictional reasons to?
Are they looking for location data for mailing? Are they looking to find
out what's close by? If people mostly only care what's close, you may
want to rethink searching by state, city, and zipcode entirely and
switch to a spatial process instead: find a nice way to get coordinates
for addresses and store and index that instead of the legal names of
places.
--
Brian Cryer
http://www.cryer.co.uk/brian

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.