Optimizing extraction of elements from large dyn arrays

#11

"Frank Winans" <fwinans (AT) sbcglobal (DOT) net> wrote in news:BZyGe.1238
$gQ5.341 (AT) newssvr33 (DOT) news.prodigy.com:

Quote:

"Joe"wrote
iJah wrote
large dynamic arrays - in Advanced Pick.
This is being done all over with the usual

N=DCOUNT(REC,CHAR(254)
FOR I=1 TO N
LINE = REC<I
NEXT I

obviously the deeper you get into the array the slower the crawl.
any clues?

If you've already got the array in memory, can't you simply SELECT
the
array, then do a LOOP WHILE READNEXT thing?

SELECT REC
LOOP
WHILE READNEXT LINE DO
whatever...
REPEAT

The syntax may be different for your platform, but you get the
idea...

Regards,
Joe
* Close, but AP READNEXT wants an ELSE clause.
SELECT REC
LINE='dummy'
LOOP WHILE LINE # "sentryvalue"
READNEXT LINE ELSE LINE = "sentryvalue"
DO
whatever...
REPEAT

As I said, the syntax may be different for the platform in question.

Quote:

**** And just because it is so easy to code, I'd try Frosty's &
**** Marvin Fisher's idea of a dimensioned array, for a 'plan B'
worksize = DCOUNT(rec, @AM)
DIM work(worksize)
work = MAT rec
FOR J=1 to worksize
line = work(J)
NEXT J
* or code it as MATPARSE work FROM rec using @AM
***** Dale Benedict advocated QSELECT, but I just cannot
***** see the charm of REC to a disk item just so QSELECT can
***** read it back in. Just SELECT REC avoids a read/write
pair.

As Luke already pointed out, there's no reason at all to introduce
dimensioned arrays when the dynamic array is already in memory ready
and waiting to go.

Regards,
Joe

#12

1. To use a dimensioned array you can (in D3 anyway, probably AP too):

max = dcount(dyn.array,@am)
dim dim.array(max)
dim.array = dyn.array
for n = 1 to to max
line = dim.array(n)
next n

2. That said, it's probably faster to:

select dyn.array to mylist
loop
readnext line from mylist else exit
repeat

(This only works with @am lists, but "convert @vm to @am in dyn.array"
is nearly instantaneous, even for vast numbers of elements.)

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

frosty wrote:

Quote:

Write the dynamic array to a workfile, then MATREAD it back in
(as a dimensioned array.) There might be a more direct way to
do this in BASIC, like MATPARSE or similar? (Been a long time
since I've used A/P.)

#13

On Fri, 29 Jul 2005 18:41:59 GMT, iJah <iJah (AT) sbcglobal (DOT) net> wrote:

Quote:

I'm looking for some way to improve performance of data extraction
from large dynamic arrays - specifically in Advanced Pick.

The mechanics of what I'm working with are already deeply ingrained in
the application software and I'm not going to re-invent the wheel, but
the sitch is that reports get written to a single record in one
big/fat attribute mark delimited array, then each line (attribute) has
to be extracted and parsed and handled for a variety of reasons.

This is being done all over with the usual

N=DCOUNT(REC,CHAR(254)
FOR I=1 TO N
LINE = REC<I
NEXT I

obviously the deeper you get into the array the slower the crawl.

in Universe i'd just use the 'REMOVE' statement and it would fly.

i've tried advanced picks flavor of remove - i think the syntax is
like 'remove element at position from array setting delimiter.code'
but i just don't seem to get much if any performance boost using that
method.

any clues for how to turbo charge this kind of extraction?

thanks everyone for the most excellent replies!

i think the

select myarray to myslist
loop
readnext line from mylist else exit
...
repeat

idea is really gonna do the trick.

i had no idea you could do an internal 'select' on a variable
containing an array.

i thought you could only do internal selects on file handles -
variables files had been opened to.

#14

iJah wrote:

Quote:

thanks everyone for the most excellent replies!

i think the

select myarray to myslist
loop
readnext line from mylist else exit
...
repeat

idea is really gonna do the trick.

It really is, too. <g>

Quote:

i had no idea you could do an internal 'select' on a variable
containing an array.

i thought you could only do internal selects on file handles -
variables files had been opened to.

It's a very common misperception, so don't sweat it. It's a good thing
you asked, because most people just muddle through, and I've seen an
enormous variety of half-arsed, baroque solutions to the same problem,
created in desparation by wild-eyed, muttering coders with bald patches. <g>

Luke

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Optimizing extraction of elements from large dyn arrays

comp.databases.pick comp.databases.pick

Re: Optimizing extraction of elements from large dyn arrays - 07-29-2005 , 10:46 PM

Re: Optimizing extraction of elements from large dyn arrays - 07-29-2005 , 11:37 PM

Re: Optimizing extraction of elements from large dyn arrays - 07-30-2005 , 08:12 AM

Re: Optimizing extraction of elements from large dyn arrays - 07-30-2005 , 06:26 PM