Re: Arrays of Arrays is useful, a hallmark of OO, and missing inMVBASIC

#1

SteveB wrote:

Quote:

Hi all,

Having used other languages heavily in conjunction with mvbasic for six
years now, I often feel serious limitations inherent in pick basic.

While I am running a process I want to build up a variable *unknown*
number of two dimensional STATIC arrays. I need the arrays to be STATIC
for performance because they are quite large (often about 1000x50) and
the process is furiously building them up from scratch.

mvbasic only allows you to acquire memory for storage at the point of
dimensioning static arrays or by building strings. Object orientated
programs are quite happy for you to create a kind of floating array
that can be stored within.

A lot of people think OO is a design process. Well its much more than
that. In this case its about the freedom for variables to contain
"things", in this case arrays.

While I agree with you about the usefulness of OO and of this type of
structure, I feel I have to point out that you can have those structures
without OO. You could do it in C, though it would take a lot of work.
Even in MV BASIC, you could simulate them throught the use of one big
array, allocated in chunks, with links at either end of those chunks,
rather like the hashed storage model. Ugly, but possible.

Luke

#2

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote

Quote:

Luke Webber wrote:

Even in MV BASIC, you could simulate them throught the use of one big
array, allocated in chunks, with links at either end of those chunks,
rather like the hashed storage model. Ugly, but possible.

The need to preallocate the "one big array" up front is the main
problem. I believe that in some MVBASICS you can enlarge an array
without losing the original data. This counts a kind of poor mans
dynamic memory allocation but is an abomination compared to the easy
and allocation and deallocation of memory resources under virtually any
other language.

Cheers,
Steve

#3

HI Steve
I frequently set up a very large array by Dim and then Mat with a big chunk
of space. What is the problem?
I definitely would not use redim to extend an array that is a disaster time
wise as is -1 on a dynamic where it is large.
RD staff reckon an array of 10,000 is very small.
Peter McMurray
"SteveB" <steve.bush (AT) neosys (DOT) com> wrote

Quote:

Luke Webber wrote:

Even in MV BASIC, you could simulate them throught the use of one big
array, allocated in chunks, with links at either end of those chunks,
rather like the hashed storage model. Ugly, but possible.

The need to preallocate the "one big array" up front is the main
problem. I believe that in some MVBASICS you can enlarge an array
without losing the original data. This counts a kind of poor mans
dynamic memory allocation but is an abomination compared to the easy
and allocation and deallocation of memory resources under virtually any
other language.

Cheers,
Steve

#4

Steve:

It's not often I run into a problem where I need floating sized Dimensioned
arrays. Kind of like a solution looking for a problem. :-)

Whenever I do I normally just use the dbms. However, I normally use dynamic
arrays for up to about 1-2Mb of data, which meets your (1000x50)
requirement, which is very small.

Bill

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote

Quote:

Luke Webber wrote:

Even in MV BASIC, you could simulate them throught the use of one big
array, allocated in chunks, with links at either end of those chunks,
rather like the hashed storage model. Ugly, but possible.

The need to preallocate the "one big array" up front is the main
problem. I believe that in some MVBASICS you can enlarge an array
without losing the original data. This counts a kind of poor mans
dynamic memory allocation but is an abomination compared to the easy
and allocation and deallocation of memory resources under virtually any
other language.

Cheers,
Steve

#5

Also I read in 60meg XML files into one big dimensioned and dynamic
array which i then analyse and spit out data into MySQL, csv and
another xml. All in unidata. and it is sub second response time. We do
it in unidata as it was originally done in .net on a windows box and
was taking several minutes. I think you will find memory allocation is
handled fairly well, atleast by unidata. (although i admit the xml
handleing is a bit flawed)

Bill H wrote:

Quote:

Steve:

It's not often I run into a problem where I need floating sized Dimensioned
arrays. Kind of like a solution looking for a problem. :-)

Whenever I do I normally just use the dbms. However, I normally use dynamic
arrays for up to about 1-2Mb of data, which meets your (1000x50)
requirement, which is very small.

Bill

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote in message
news:1151712075.448469.203240 (AT) b68g2000cwa (DOT) googlegroups.com...

Luke Webber wrote:

Even in MV BASIC, you could simulate them throught the use of one big
array, allocated in chunks, with links at either end of those chunks,
rather like the hashed storage model. Ugly, but possible.

The need to preallocate the "one big array" up front is the main
problem. I believe that in some MVBASICS you can enlarge an array
without losing the original data. This counts a kind of poor mans
dynamic memory allocation but is an abomination compared to the easy
and allocation and deallocation of memory resources under virtually any
other language.

Cheers,
Steve

#6

Steve,

Your code samples appear to be identical except for the word
Dynamic or Dimensioned at the top?

How is one thereby Dynamic and the other Dimensioned?

Did you mis-copy the code?

Thanks,

Larry Hazel

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote

Quote:

Bill H wrote:
Whenever I do I normally just use the dbms. However, I normally use
dynamic
arrays for up to about 1-2Mb of data, which meets your (1000x50)
requirement, which is very small.

Hmm I thought I better get back to basics and do a real test why I
cannot use dynamic arrays. According to the tests below, for the case
being discussed, it takes around 160 SECONDS to use a dynamic array
versus 70 MILLISECONDS to use a DIMENSIONED array in the following
reasonable test.

Dimensioned arrays in this simple case are more than 2000 times faster.

I think this is very strong evidence that dynamic arrays are totally
unsuitable as a memory allocation method for very small 1-2Mb arrays.

Since dynamic arrays are no use for high performance structures one
needs dimensioned arrays and this is my point that due to the lack of
OO data structures in the typical MVBASIC they are of limited use.

DIMENSIONED ARRAY TEST
------------------------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 70 MILLISECONDS

DYNAMIC ARRAY
----------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 160 SECONDS

(XP Professional SP2 Intel T2400 1.83Ghz
using OpenQM under Cygwin and QM for PC from Ladybridge)

#7

Homer L. Hazel wrote:

Quote:

Steve,

Your code samples appear to be identical except for the word
Dynamic or Dimensioned at the top?

How is one thereby Dynamic and the other Dimensioned?

Did you mis-copy the code?

Quite apart from that little peccadillo <g>, I have other concerns about
the code. God knows we all love to carve up anybody who posts code in
this group (or is that just me?), but what is it with this bit?

PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

AFAICT, the MD00 mask does nothing at all, except to screw up completely
on D3. On D3, you need to use an OCONV, but it still does bugger-all.

Oddly enough, I'm generally on Steve's side in the matter of modern
languages as opposed to the archaic MV BASICs, but given the sample code
he's posted here, I'm starting to suspect that he could "write FORTRAN
in any language" as the old saying goes. Loop and index variables named
"II" and "JJ"? Classic FORTRAN, in which implicit integers are named
with initial letters starting in the range I to N! Conversions that do
nothing?

Yes, I know that this is just a test program, but if I was in Steve's
place, I'd be wanting to justify my claim that I know MV BASIC well
enough to criticise it, and the test code has, for me, the opposite effect.

Sorry Steve, I don't wish to offend but, well...

This also makes me wonder whether Steve's present purposes might be as
readily served /without/ building an array of arrays in virtual memory.
How about it, Steve? Care to describe the problem in more detail so we
can suggest alternative approaches?

Luke

Quote:

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote in message
news:1151757003.132100.140560 (AT) v61g2000cwv (DOT) googlegroups.com...
Bill H wrote:
Whenever I do I normally just use the dbms. However, I normally use
dynamic
arrays for up to about 1-2Mb of data, which meets your (1000x50)
requirement, which is very small.
Hmm I thought I better get back to basics and do a real test why I
cannot use dynamic arrays. According to the tests below, for the case
being discussed, it takes around 160 SECONDS to use a dynamic array
versus 70 MILLISECONDS to use a DIMENSIONED array in the following
reasonable test.

Dimensioned arrays in this simple case are more than 2000 times faster.

I think this is very strong evidence that dynamic arrays are totally
unsuitable as a memory allocation method for very small 1-2Mb arrays.

Since dynamic arrays are no use for high performance structures one
needs dimensioned arrays and this is my point that due to the lack of
OO data structures in the typical MVBASIC they are of limited use.

DIMENSIONED ARRAY TEST
------------------------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 70 MILLISECONDS

DYNAMIC ARRAY
----------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 160 SECONDS

(XP Professional SP2 Intel T2400 1.83Ghz
using OpenQM under Cygwin and QM for PC from Ladybridge)

#8

SteveB wrote:

Quote:

Homer L. Hazel wrote:
Steve,

Your code samples appear to be identical except for the word
Dynamic or Dimensioned at the top?

Umm, yeah. Sorry. Here is the dynamic array version cut and pasted from
my test program.

YY='ABCDEFGHIJK'
NI=1000
NJ=50
ZZ=''
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ<RND(NI)+1,RND(NJ)+1>=STR('X',RND(10))
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Steve

Well gee. If you compare code that works with 50,000 individually
addressable memory locations with one that works with a delimited
infinite-length string wouldn't you expect to see a wee difference? I
think you might be an idiot.

#9

Quote:

Homer L. Hazel wrote:
Steve,

Your code samples appear to be identical except for the word
Dynamic or Dimensioned at the top?

SteveB wrote:

Umm, yeah. Sorry. Here is the dynamic array version cut and pasted
from my test program.

YY='ABCDEFGHIJK'
NI=1000
NJ=50
ZZ=''
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ<RND(NI)+1,RND(NJ)+1>=STR('X',RND(10))
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Steve

Mike Preece wrote:
Well gee. If you compare code that works with 50,000 individually
addressable memory locations with one that works with a delimited
infinite-length string wouldn't you expect to see a wee difference?
I think you might be an idiot.

Ah, but is he self-aggrandizing? Or even vociferous? =`:^>

--
frosty

#10

Steve:

Perhaps a little reworking of the tests become illuminating. Here's a
simple program:

** Test the difference between a dimensioned array and a relatively small
** dynamic array.
*
** define the arrays
*
ArrayCnt = 5000
MyDynArray = ''
DIM MyDimArray(ArrayCnt)
*
** Build both arrays
CRT "Dimensioned Load Start-: " : TIMEDATE()
FOR X = 1 TO ArrayCnt
MyDimArray(X) = "Dim"
NEXT X
CRT "Dimensioned Load Stop--: " : TIMEDATE()
*
CRT "Dynamic Load Start-----: " : TIMEDATE()
FOR X = 1 TO ArrayCnt
MyDynArray<-1> = "Dyn"
NEXT X
CRT "Dynamic Load Stop------: " : TIMEDATE()
*
** Read both arrays
CRT "Dimensioned Read Start-: " : TIMEDATE()
FOR X = 1 TO ArrayCnt
dummy = MyDimArray(X)
NEXT X
CRT "Dimensioned Read Stop--: " : TIMEDATE()
*
CRT "Dynamic Read Start-----: " : TIMEDATE()
FOR X = 1 TO ArrayCnt
dummy = MyDynArray<X>
NEXT X
CRT "Dynamic Read Stop------: " : TIMEDATE()
*
** End of program
END

The output is illustrative of what has been said about reasonably small
dynamic arrays:

1 Demo (0)-> RUN DTABP TEST
Dimensioned Load Start-: 13:20:03 Jul 01 2006
Dimensioned Load Stop--: 13:20:03 Jul 01 2006
Dynamic Load Start-----: 13:20:03 Jul 01 2006
Dynamic Load Stop------: 13:20:03 Jul 01 2006
Dimensioned Read Start-: 13:20:03 Jul 01 2006
Dimensioned Read Stop--: 13:20:03 Jul 01 2006
Dynamic Read Start-----: 13:20:03 Jul 01 2006
Dynamic Read Stop------: 13:20:03 Jul 01 2006
1 Demo (0)->

This pretty much tells us the difference in loading and reading small
dimensioned and dynamic arrays is, as a practical matter, negligible.

Bill

"SteveB" <steve.bush (AT) neosys (DOT) com> wrote

Quote:

Bill H wrote:
Whenever I do I normally just use the dbms. However, I normally use
dynamic
arrays for up to about 1-2Mb of data, which meets your (1000x50)
requirement, which is very small.

Hmm I thought I better get back to basics and do a real test why I
cannot use dynamic arrays. According to the tests below, for the case
being discussed, it takes around 160 SECONDS to use a dynamic array
versus 70 MILLISECONDS to use a DIMENSIONED array in the following
reasonable test.

Dimensioned arrays in this simple case are more than 2000 times faster.

I think this is very strong evidence that dynamic arrays are totally
unsuitable as a memory allocation method for very small 1-2Mb arrays.

Since dynamic arrays are no use for high performance structures one
needs dimensioned arrays and this is my point that due to the lack of
OO data structures in the typical MVBASIC they are of limited use.

DIMENSIONED ARRAY TEST
------------------------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 70 MILLISECONDS

DYNAMIC ARRAY
----------------------
YY='ABCDEFGHIJK'
NI=1000
NJ=50
DIM ZZ(NI,NJ)
STARTTIME=SYSTEM(1020)
FOR II=1 TO NI
FOR JJ=1 TO NJ
ZZ(RND(NI)+1,RND(NJ)+1)=YY
NEXT JJ
NEXT II
STOPTIME=SYSTEM(1020)
PRINT (STOPTIME-STARTTIME) 'MD00':'ms'

Takes around 160 SECONDS

(XP Professional SP2 Intel T2400 1.83Ghz
using OpenQM under Cygwin and QM for PC from Ladybridge)

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Re: Arrays of Arrays is useful, a hallmark of OO, and missing inMVBASIC

comp.databases.pick comp.databases.pick

Re: Arrays of Arrays is useful, a hallmark of OO, and missing inMVBASIC - 06-30-2006 , 06:15 PM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 12:04 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 12:08 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 01:46 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 05:03 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 09:26 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing inMVBASIC - 07-01-2006 , 10:05 AM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 12:52 PM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 01:12 PM

Re: Arrays of Arrays is useful, a hallmark of OO, and missing in MVBASIC - 07-01-2006 , 03:25 PM