dbTalk Databases Forums  

How wasteful is READV... WRITEV...?

comp.databases.pick comp.databases.pick


Discuss How wasteful is READV... WRITEV...? in the comp.databases.pick forum.



Reply
 
Thread Tools Display Modes
  #21  
Old   
Jeff K
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-29-2010 , 11:36 AM






On Jan 29, 2:36*am, "Peter McMurray" <excalibu... (AT) bigpond (DOT) com> wrote:

Quote:
It is ridiculous to ever use writev or readv as it leads to chaos.
I agree 100%. Especially since buggy programs can WRITEV a single
attribute to a record that didn't previously exist, leading to
"mystery" data. Plus, these buggy programs will usually WRITEV using
a record ID of 0, an empty string, or some ridiculous key that
includes system delimiters, making it next to impossible to clean up.
Chaos!

Quote:
The simple rule to follow is always dimension all items as an array and
always equate every element of the array to a standard variable then always
refer to all variables by name.
Simple bug free and obvious.
Not bad, but this approach causes trouble if you ever need to handle
*two* records from the same file at once. You handle one with the
MATREAD, but what about the other one? The EQUs don't help now, and
you fall back to integer field references. I prefer to let the EQUs
just be numeric, and then always use dynamic arrays:

* in an INCLUDEd file definition item:
EQU TRNS.SUBGROUP.ID TO 10

* In code using the file
THIS.SUBGROUP = TRNS.REC< TRNS.SUBGROUP.ID >

While theoretically slower, in my work I've never seen it make a
measurable difference.

The additional benefit of this approach is that, per the original
poster's issue, you could scan the code for TRNS.SUBGROUP.ID to find
all programs that touch that field. Again, theoretical, because very
few applications that have kept a consistent coding style throughout
their maintenance history.

Reply With Quote
  #22  
Old   
Kevin Powick
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-29-2010 , 12:12 PM






On Jan 29, 11:36*am, Jeff K <jeffk75... (AT) verizon (DOT) net> wrote:

Quote:
*I prefer to let the EQUs
just be numeric, and then always use dynamic arrays:

* ** in an INCLUDEd file definition item:
* *EQU TRNS.SUBGROUP.ID TO 10

* ** In code using the file
* *THIS.SUBGROUP = TRNS.REC< TRNS.SUBGROUP.ID

While theoretically slower, in my work I've never seen it make a
measurable difference.
Also, this method can be used for both dimensioned and dynamic arrays.

Another benefit is that, in D3 at least, if you spell the equated
variable incorrectly, it will raise a runtime error as an unassigned
variable.

However, if your variable is EQUated to a dimensioned array variable,
and you spell it incorrectly, it simply becomes a variable
assignment. No error is raised and your dimensioned array ends up
missing data.

--
Kevin Powick

Reply With Quote
  #23  
Old   
Peter McMurray
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-29-2010 , 02:49 PM



Hi
The issue of two reads at the same time is of course what object oriented
programming solves simply.
I expect that the most used programs in a commercial suite do this all the
time. An order involves a delivery address and a billing address. In our
case we stick to the rule and simply add a prefix to the base schema for the
second layout. So you might have DELADD1 AND BILLADD1 or DEL_ADD1 and
BILL_ADD1. we actually use the same names in the Access/Recall/English
dictionary so we tend to short names.
The schema must be included and never fiddled with by anyone in the actual
final program. We actually write the business logic in a separate file so
the schema is sacrosanct,
Kevin pointed out that read rather than matread could be missed. Not in our
case because we also set that in a schema and unpack the read to the defined
variable names. However there is so little advantage with this and the use
of standard names with arrays presents such a massive advantage we do not
bother anymore.
As for finding places that the items are used that is simplicity itself one
just selects the items that contain a given schema and recompile the lot
whenever a schema changes. Obviously this is pretty rare on a settled
system.
This is our thirty third year of programming this way and I have had the
misfortune to work on some of the horrors this thread has raised which
convinces me that we were right from the beginning. We can ignore 90% of a
program when debugging as the error is always in that little bit of logic
that you added.
Peter McMurray
"Jeff K" <jeffk75067 (AT) verizon (DOT) net> wrote

On Jan 29, 2:36 am, "Peter McMurray" <excalibu... (AT) bigpond (DOT) com> wrote:

Quote:
It is ridiculous to ever use writev or readv as it leads to chaos.
I agree 100%. Especially since buggy programs can WRITEV a single
attribute to a record that didn't previously exist, leading to
"mystery" data. Plus, these buggy programs will usually WRITEV using
a record ID of 0, an empty string, or some ridiculous key that
includes system delimiters, making it next to impossible to clean up.
Chaos!

Quote:
The simple rule to follow is always dimension all items as an array and
always equate every element of the array to a standard variable then
always
refer to all variables by name.
Simple bug free and obvious.
Not bad, but this approach causes trouble if you ever need to handle
*two* records from the same file at once. You handle one with the
MATREAD, but what about the other one? The EQUs don't help now, and
you fall back to integer field references. I prefer to let the EQUs
just be numeric, and then always use dynamic arrays:

* in an INCLUDEd file definition item:
EQU TRNS.SUBGROUP.ID TO 10

* In code using the file
THIS.SUBGROUP = TRNS.REC< TRNS.SUBGROUP.ID >

While theoretically slower, in my work I've never seen it make a
measurable difference.

The additional benefit of this approach is that, per the original
poster's issue, you could scan the code for TRNS.SUBGROUP.ID to find
all programs that touch that field. Again, theoretical, because very
few applications that have kept a consistent coding style throughout
their maintenance history.

Reply With Quote
  #24  
Old   
Tony Gravagno
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-29-2010 , 09:47 PM



Um, did anyone here understand any of that?
Don't worry about asking which part I didn't understand.
I didn't understand anything past "Hi" or before the sig.

Reply With Quote
  #25  
Old   
RJ
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 01:50 AM



It's actually not that hard to understand. What is hard to understand is
why bother? With cheap iron that is faster than believable there is not
much of a hit to most inefficient programming practices. It's much like the
old days of dial-in. If you had a 2400 baud or faster line you really
didn't notice the delay in most cases. I guess the only really good reason
for good practices is pride. But that IS a good reason.
BobJ

Aside to Tony - The OP to which I am responding appears to be attributed to
you but I suspect that is false. The wordy but informative and accurate
original, original is surely yours but the succinct question about
understanding it is surely not yours. And perhaps should be signed with a
smiley
BobJ

"Tony Gravagno" <address.is.in.posts (AT) removethis (DOT) com.invalid> wrote in
message news:dh47m5lcnqhvlpuc64e9kg2lnp7e9iqnp1 (AT) 4ax (DOT) com...
Quote:
Um, did anyone here understand any of that?
Don't worry about asking which part I didn't understand.
I didn't understand anything past "Hi" or before the sig.

Reply With Quote
  #26  
Old   
Kevin King
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 11:13 AM



Quote:
It is ridiculous to ever use writev or readv as it leads to chaos.
Agreed.

Quote:
The simple rule to follow is always dimension all items as an array and
always equate every element of the array to a standard variable then always
refer to all variables by name.
With all due respect, the assumption that dimensioned arrays are more
efficient than dynamic arrays is unfortunately not a universal truth.

Reply With Quote
  #27  
Old   
Tony Gravagno
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 04:03 PM



Thanks for the way out Bob, but I really didn't get any of Peter's
post at the time. After a night's sleep, some coffee, three more
reads, and a slightly more open mind, I understood more.

Except for this statement:

Quote:
The issue of two reads at the same time is of course what
object oriented programming solves simply.
I'm picking a nit and it's OT from READV/WRITEV. Never mind.

T

"RJ" wrote:
Quote:
Aside to Tony - The OP to which I am responding appears to be attributed to
you but I suspect that is false. The wordy but informative and accurate
original, original is surely yours but the succinct question about
understanding it is surely not yours. And perhaps should be signed with a
smiley
BobJ

"Tony Gravagno" wrote
Um, did anyone here understand any of that?
Don't worry about asking which part I didn't understand.
I didn't understand anything past "Hi" or before the sig.

Reply With Quote
  #28  
Old   
Peter McMurray
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 04:50 PM



Hi
According to the manual it is wherever any more than one element is
concerned.
However the major benefit in even that singular scenario is the use of
standard variable names.
Peter McMurray
"Kevin King" <kevin (AT) precisonline (DOT) com> wrote

Quote:
It is ridiculous to ever use writev or readv as it leads to chaos.

Agreed.

The simple rule to follow is always dimension all items as an array and
always equate every element of the array to a standard variable then
always
refer to all variables by name.

With all due respect, the assumption that dimensioned arrays are more
efficient than dynamic arrays is unfortunately not a universal truth.

Reply With Quote
  #29  
Old   
Peter McMurray
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 04:50 PM



Hi Tony
Perhaps you should try reading the post to which I was responding which is
part of my reponse
I'll highlight it for you
"Not bad, but this approach causes trouble if you ever need to handle
*two* records from the same file at once. You handle one with the
MATREAD, but what about the other one? "

Following on the original writev topicI actually disagree with using offsets
into dynamc arrays of titems for many reasons not least of which is that
people always remember the wrong one and wreck the file.
Of course I forgot you hate generated code because you went and played with
the output instead of using the proper tool and as a result lost all of your
work. Check your posts from a year or two back.
Peter McMurray

"Tony Gravagno" <address.is.in.posts (AT) removethis (DOT) com.invalid> wrote in
message news:dh47m5lcnqhvlpuc64e9kg2lnp7e9iqnp1 (AT) 4ax (DOT) com...
Quote:
Um, did anyone here understand any of that?
Don't worry about asking which part I didn't understand.
I didn't understand anything past "Hi" or before the sig.

Reply With Quote
  #30  
Old   
Scott Ballinger
 
Posts: n/a

Default Re: How wasteful is READV... WRITEV...? - 01-30-2010 , 05:58 PM



Back to the OP....

On CUBS systems there is (used to be?) a memo file of textual notes;
the last note is appended to the end of the memo record as the last
attribute, and the number of that last attribute is stored in
attribute 1 of the same memo record. CUBS used to use a trick to speed
up (circa R83) adding note lines to this file by using a READV +
WRITEV + WRITEV which was faster than a standard READ + WRITE,
particularly as the number of memo lines increases.

Here are two test programs to demonstrate:

:ct bp memo.test (double writev)

memo.test
001 open "sb.temp" to sb.temp else stop 201,"sb.temp"
002
003 id = "12345"
004 rec = 1
005 t = time()
006 d = date()
007 tot = 10000
008 a0 = @(0)
009
010 write rec on sb.temp,id
011
012 for n = 1 to tot
013 read rec from sb.temp,id else rec = 1
014 pos = rec<1> +1
015 rec<1> = pos
016 rec<pos> = "the quick brown fox jumped over the lazy dog"
017 write rec on sb.temp,id
018 if mod(pos,100) eq 0 then
019 call counter.sub(n,t,d,tot,txt)
020 print a0:n:txt:
021 end
022 next n
023
024 call final.count(n,t,d)
025

:ct bp memo.test2 (single write)

memo.test2
001 open "sb.temp" to sb.temp else stop 201,"sb.temp"
002
003 id = "12345"
004 rec = 1
005 t = time()
006 d = date()
007 tot = 10000
008 a0 = @(0)
009
010 write rec on sb.temp,id
011
012 for n = 1 to tot
013 readv pos from sb.temp,id,1 else pos = 1
014 pos +=1
015 writev pos on sb.temp,id,1
016 writev "the quick brown fox jumped over the lazy dog" on
sb.temp,id,pos
017 if mod(pos,100) eq 0 then
018 call counter.sub(n,t,d,tot,txt)
019 print a0:n:txt:
020 end
021 next n
022
023 call final.count(n,t,d)
024


And the results....

D3 7.5.1 Linux (with MFT drives!)
Regular compile (non-flash)

:memo.test
10001 records processed in 00:31 = 323 per second.
:memo.test2
10001 records processed in 00:24 = 417 per second.
:memo.test
10001 records processed in 00:30 = 333 per second.
:memo.test2
10001 records processed in 00:22 = 455 per second.

You can see why CUBS did it that way.

That was then, this is now; Flash-compiling negates the writev
advantage, I guess because of Flash doesn't change the I/O speed but
it does radically improve the dynamic array (record) handling. Still,
the fact that you can do two WRITEVs in the same amount of time as a
single WRITE implies that WRITEV is quite a bit faster.

Flash-compiled:

:memo.test
10001 records processed in 00:23 = 435 per second.
:memo.test2
10001 records processed in 00:22 = 455 per second.
:memo.test
10001 records processed in 00:23 = 435 per second.
:memo.test2
10001 records processed in 00:23 = 435 per second.


Here are the results from a UV/Linux system (I shortened the loop to
3000 on the UV machine to keep the times similar).

RELLEVEL
X
10.2.7
PICK
PICK.FORMAT
10.2.7

Quote:
MEMO.TEST
3001 records processed in 00:16 = 179 per second.
MEMO.TEST2
3001 records processed in 00:35 = 85.0 per second.
MEMO.TEST
3001 records processed in 00:16 = 179 per second.
MEMO.TEST2
3001 records processed in 00:35 = 85.7 per second.

On this UV system, it looks like two WRITEVs are about twice as slow
as a single WRITE, so there does not seem to be any advantage at all
to using WRITEV here.

/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.