Hi all,

Are there any limits active lists generated by a SELECT? I'm thinking
about things like number of items, overall size and so on. I know in
BASIC a dynamic array starts to bog things down when it goes over a few
thousand attributes. I'm wondering if the same thing holds true with
the way active lists get built in memory and of course where the wall
is on max limits.

Thanks in advance,

Tedd

Tedd Scofield wrote:

"Tedd Scofield" <tedd_scofield (AT) hotmail (DOT) com> wrote:

One of the reasons dynamic arrays start bogging down as they get
larger is that a reference to an attribute causes a scan from the
beginning of the item all the way down to correct "attribute mark
count". So if you go atb 4985 to 4986, you're not going down one atb,
you're going down 4986. IIRC an internal select list is not like
this, and D3 maintains an index to the current ID. Note however that
in generating the list, you're still going to traverse frames through
overflow as you build X thousands or millions of ID's. I'm not sure
how that plays with RAM usage but since lists are flushed to disk out
of overflow I'm assuming your entire list is going to hit memory too.

Now, that is only for Execute "Select..." and none of that applies
with a BASIC SELECT statement.
In that case you're setting up a pointer to item ID's directly on
disk, and not actually building a list in memory. So if you can work
with a hashed file, there's virtually no footprint. If you need to
work with sorted lists, then try working with Indexes and then use
root/key to traverse the list in sorted order - it's much less painful
than building the list every time.

HTH.
T

Thanks for the info in that reply Tony. I've used key/root before and
it was real quick but I'm actually thinking about something a little
different.

"Note however that in generating the list, you're still going to
traverse frames through overflow as you build X thousands or millions
of ID's. I'm not sure how that plays with RAM usage but since lists
are flushed to disk out of overflow I'm assuming your entire list is
going to hit memory too."

This is what I am curious about, how that list is built and handled.
I've only recently grokked the trick of using SELECT and dictionaries
to load the active list with correlated data =)

Stefano wrote:
on the *bog things down* factor although I have not noticed it. It also
depends on what you are doing with the list.

Patrick, <;=)

Goo'day,

On Fri, 22 Apr 2005 16:25:57 -0700, Tony Gravagno
<g6q3x9lu53001 (AT) sneakemail (DOT) com.invalid> wrote:

What's wrong with always playing with the "first cab off the rank" of
the list and deleting/removing the 1st item on the list as you've
finished with it?

Is there an overhead in squashing the remiander of the list upwards vs
traversing down the list for the next attribute?

If the list needs to be "reused", couldn't it be saved somewhere/how
and retrieved again when required.....


Regards,

Bruce Nichol
Talon Computer Services
ALBURY NSW Australia

http://www.taloncs.com.au

If it ain't broke, fix it until it is....

Tedd Scofield wrote:
You mean output specifiers? Useful little trick no? I've used it in the
past where I only wante a few attributes, and I wrote a
report-generating tool that way once as well. Nice because there is no
practical way of calling A- or F-code correlative from BASIC (and Tfile
translates are so inefficient from BASIC).

The biggest limiting factor will be disk space, especially on D3/NT
because the lists are built in the VME.

Luke

Bruce Nichol wrote:

There's one hell of an overhead. The machine spend all its time moving
the variable into temp string space and manipulating it. Better to
traverse the dynamic array than that.

Of course, it's also possible to SELECT a dynamic array to a list vbl if
that's what you want to do. Just us a string vbl where you would
normally put the file vbl. Another neat little trick.

Then of course, you have the REMOVE statement, which avoid the need for
scanning. REMOVE was initially a PR1ME Information construct, but most
implementations now seem to have it. I don't know about mvEnterprise of
mvBASE, but certainly U2, D3 and jBASE have it.

By all means.

Luke

Even though we say Pick Basic does not use data types (integer, string,
variant, etc), that's not technically true. There are several data "types"
including, but not limited to:

short string (< 8 bytes)
long string (> 8 bytes)
short number (binary, < 8 bytes)
long number (stored as a string in medium-sized string space)
file (saved as binary 4 bytes fid, 2 bytes modulo)
list

This last "list" variable is stored like a file pointer, where the address
of the next item in the list is saved. READNEXT returns the item, moves
that pointer and saves it so the next time we don't have to traverse the
whole list again.

Since lists are not reusable, there's no reason to save the beginning once
you've readnext'ed past, so frames are dropped off the front a they become
useless.

Some of the newer file handlers like PDP and mvDotNet and, I believe, some
of the Pick-a-likes allow you to save the whole list, use the items and
"re-select" or read-previous to the beginning or whatnot. All that is
required to accmplish that is another address pointing at the current
location as well as the one pointing at the beginning. Same features, but
the frames are not returned to overflow until the entire list is completed
and "released"

Mark Brown


"Tedd Scofield" <tedd_scofield (AT) hotmail (DOT) com> wrote

Luke Webber <luke (AT) webber (DOT) com.au> wrote:

For deleting atb1 of a dynarray, Bruce's real life analogy applies
well, when the first cab pulls out you'll usually see the line of cabs
all shift up to fill the gap. When you're looking at items shifting in
frames and overflow frames getting released off the back end as
they're no longer needed, this starts to look like real sloppy stuff
which should have been re-engineered decades ago.

FWIW, string handling in FlashBASIC is much better. I dunno why,
because we're still doing a strcpy of a memory segment to fill gaps -
maybe someone with a clue can explain how this one works.

I was googling the other day and saw this one again, I haven't had a
chance to use it yet but it's nice to have it in the back pocket!


Hmm, never used it, just looked it up... prolly never will use it.

T