dbTalk Databases Forums  

OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements

comp.databases.pick comp.databases.pick


Discuss OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements in the comp.databases.pick forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Martin Phillips
 
Posts: n/a

Default OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 03:39 AM






Hi all,

Thursday once more...

The latest Tip of the Week article can be found at http://www.openqm.com/tow.htm.


Martin Phillips, Ladybridge Systems

Reply With Quote
  #2  
Old   
Tony Gravagno
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 09:59 AM






Martin, that's great stuff. Thanks for the tip.

For anyone not using QM, I documented similar functionality a few
years ago for D3:

nospamNebula-RnD.com/blog/tech/mv/2008/03/d3shell1.html

Reply With Quote
  #3  
Old   
Frank Winans
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 12:06 PM



"Tony Gravagno" wrote
Quote:
Martin, that's great stuff. Thanks for the tip.

For anyone not using QM, I documented similar functionality a few
years ago for D3:

nospamNebula-RnD.com/blog/tech/mv/2008/03/d3shell1.html
Martin's tip is as thread Subject indicates;
Tony G's blog is about shell vars.

About that blog; your code is good solid Structured Programming,
but I'd probably collapse this code

DONE = 0
LOOP
READNEXT ID ELSE DONE=1
UNTIL DONE DO
THE.LIST := \ "\:ID:\"\
REPEAT

to just

SELF: *
READNEXT ID THEN THE.LIST := \ "\:ID:\"\ ; GO SELF

Unless the client specifically forbade using GO or GOTO.

Reply With Quote
  #4  
Old   
Tony Gravagno
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 12:45 PM



Thanks for the note on the code. I'm sure if I had included the use
of GOTO in my blog that someone would gripe about that rather than
commenting on the actual content. That's the way things work in this
community, so I try to keep focus on the topic with common code that
most people wouldn't bother to argue about.

Quote:
Martin's tip is as thread Subject indicates;
Tony G's blog is about shell vars.
I understand what you're saying, but disagree - at least my intent was
other than what you have understood.

The tip is about how to use the IN operator in QM to process an
attribute against an item which has a list of values.

That's exactly what my blog describes, though using shell vars.

It's about functionality, not technique. It's about Pick guys using
available tools to get their job done, regardless of their platform.

And I'm not trying to pit D3 against QM - just providing info for a
segment of the audience because that's a good use for this forum.

Reply With Quote
  #5  
Old   
Ed Sheehan
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 12:49 PM



LOOP WHILE READNEXT ID DO THE.LIST := \ "\:ID:\"\ REPEAT

This works in Universe. Try. Buy.

Ed

"Frank Winans" <fwinans (AT) sbcglobal (DOT) net> wrote

Quote:
"Tony Gravagno" wrote
Martin, that's great stuff. Thanks for the tip.

For anyone not using QM, I documented similar functionality a few
years ago for D3:

nospamNebula-RnD.com/blog/tech/mv/2008/03/d3shell1.html
Martin's tip is as thread Subject indicates;
Tony G's blog is about shell vars.

About that blog; your code is good solid Structured Programming,
but I'd probably collapse this code

DONE = 0
LOOP
READNEXT ID ELSE DONE=1
UNTIL DONE DO
THE.LIST := \ "\:ID:\"\
REPEAT

to just

SELF: *
READNEXT ID THEN THE.LIST := \ "\:ID:\"\ ; GO SELF

Unless the client specifically forbade using GO or GOTO.


Reply With Quote
  #6  
Old   
Kevin Powick
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 01:31 PM



On 2011-10-20 10:59:12 -0400, Tony Gravagno
<tony_gravagno (AT) nospam (DOT) invalid> said:

Quote:
Martin, that's great stuff. Thanks for the tip.

For anyone not using QM, I documented similar functionality a few
years ago for D3:

That's all well and good T., but does D3 come in mauve?



For those that don't get the "mauve" reference:
http://dilbert.com/strips/comic/1995-11-17/

--
Kevin Powick

Reply With Quote
  #7  
Old   
Brian Speirs
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor SelectionElements - 10-20-2011 , 03:27 PM



Now, that is quite cool ... although I guess I'd have done something like:

QSELECT somefile item
LIST anotherfile WITH selection-criteria output

And the NOT.IN would be achieved with a combination of SELECT and NSELECT.

Cheers,

Brian

On 20/10/2011 9:39 a.m., Martin Phillips wrote:
Quote:
Hi all,

Thursday once more...

The latest Tip of the Week article can be found at http://www.openqm.com/tow.htm.


Martin Phillips, Ladybridge Systems

Reply With Quote
  #8  
Old   
Tony Gravagno
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor Selection Elements - 10-20-2011 , 07:28 PM



Brian, QSelect is limited to IDs, so IN/NOT.IN do have value over that
technique.

T

Brian Speirs wrote:

Quote:
Now, that is quite cool ... although I guess I'd have done something like:

QSELECT somefile item
LIST anotherfile WITH selection-criteria output

And the NOT.IN would be achieved with a combination of SELECT and NSELECT.

Reply With Quote
  #9  
Old   
Martin Phillips
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query ProcessorSelection Elements - 10-21-2011 , 05:52 AM



On Oct 20, 6:49*pm, "Ed Sheehan" <NOedsS... (AT) xmission (DOT) com> wrote:
Quote:
LOOP WHILE READNEXT ID DO THE.LIST := \ "\:ID:\"\ REPEAT

This works in Universe. Try. Buy.
And this same syntax was recently added to QM to ease a migration.

However.....

Think carefully about this loop. You start with an empty string and
then concatenate each id, one by one.

I am fairly certain that UV will copy the string for each
concatenation because it uses contiguous strings. QM optimises this by
use of "chunked strings" that gain in this sort of operation but can
lose in others. For this particular statement, we will reduce, but not
eliminate, the copying.

We can, however, do much better

READLIST THE.LIST THEN THE.LIST = \"\ : CHANGE(THE.LIST, @FM, \
"\:ID:\"\) : \"\

The rather messy conversion of the field mark delimited select list to
a space separated list of quoted ids can probably be improved but this
works fine.

Just for fun, I tried both methods on UV and QM with a select list
containing 162182 items that I had conveniently available. Like all
test results, these should be verified before trusting them
completely...

LOOP method
QM 37.299 seconds
UV 61.803 seconds

READLIST method
QM 0.281 secods
UV 40.770 seconds

I expected this test to be super fast on QM because of how READLIST
works. The READLIST figure for UV surprised me as the UV internals
course materials recommend this approach on the basis that is simply
changes a memory pointer. Out of the 40.770 seconds all but 0.312
seconds are used by the READLIST.

Sadly, there is no internals data available for Unidata. The same test
produces an astonishingly fast execution of the LOOP method that ought
to be slow but a result similar to that of UV for the READLIST
method. I do not quote figures because the Unidata test was on a
different system.


Martin Phillips, Ladybridge Systems.

Reply With Quote
  #10  
Old   
sdavmor
 
Posts: n/a

Default Re: OpenQM Tip of the Week: The IN and NOT.IN Query Processor SelectionElements - 10-21-2011 , 09:46 AM



On 10/21/2011 03:52 AM, Martin Phillips wrote:
Quote:
On Oct 20, 6:49 pm, "Ed Sheehan"<NOedsS... (AT) xmission (DOT) com> wrote:
LOOP WHILE READNEXT ID DO THE.LIST := \ "\:ID:\"\ REPEAT

This works in Universe. Try. Buy.

And this same syntax was recently added to QM to ease a migration.

However.....

Think carefully about this loop. You start with an empty string and
then concatenate each id, one by one.

I am fairly certain that UV will copy the string for each
concatenation because it uses contiguous strings. QM optimises this by
use of "chunked strings" that gain in this sort of operation but can
lose in others. For this particular statement, we will reduce, but not
eliminate, the copying.

We can, however, do much better

READLIST THE.LIST THEN THE.LIST = \"\ : CHANGE(THE.LIST, @FM, \
"\:ID:\"\) : \"\

The rather messy conversion of the field mark delimited select list to
a space separated list of quoted ids can probably be improved but this
works fine.

Just for fun, I tried both methods on UV and QM with a select list
containing 162182 items that I had conveniently available. Like all
test results, these should be verified before trusting them
completely...

LOOP method
QM 37.299 seconds
UV 61.803 seconds

READLIST method
QM 0.281 secods
UV 40.770 seconds

I expected this test to be super fast on QM because of how READLIST
works. The READLIST figure for UV surprised me as the UV internals
course materials recommend this approach on the basis that is simply
changes a memory pointer. Out of the 40.770 seconds all but 0.312
seconds are used by the READLIST.

Sadly, there is no internals data available for Unidata. The same test
produces an astonishingly fast execution of the LOOP method that ought
to be slow but a result similar to that of UV for the READLIST
method. I do not quote figures because the Unidata test was on a
different system.


Martin Phillips, Ladybridge Systems.
Cool stuff, Martin! Thanks very much for the examples.
--
Cheers, SDM -- a 21st Century Schizoid Man
Systems Theory project website: http://systemstheory.net
find us on MySpace, GarageBand, Reverb Nation, Last FM, CDBaby
free MP3s of Systems Theory, Mike Dickson & Greg Amov music at
http://mikedickson.org.uk

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.