Just when I thought things could not get more complicated.

I've been having some problems with ACCESS selections on my database. This
is d3 7.4.0 on RH Linux 2.4.20

SELECT TOWN.REF WITH LAT OR WITH LAT.DEC

9m items selected out of 13m [paraphrase!]

now to refine the numbers some more & knock out items flagged for deletion

SELECT TOWN.REF WITH COUNTY [DEL] OR WITH CHK DEL]

60K items selected out of 80K

Now this is WRONG! There should be around 3.5m items.

So I've copied those 60K and noted the last one to be copied in a file.

Then done an ITEM on the record ID.

The last or next to last record ID in the group is very long. I know where
it came from, but it does not seem to have caused any problems previously.
Only since d3 and Linux were upgraded as month or so ago.

Luckily I have an independent index file and can find these very long IDs.

In each case I can retrieve the record and chop down the ID to something
more normal. When I do this, I can get further though the file. The
problem is that I don't know just how many there are.

In one case that I am looking at now, the ID has a length of 101. There
may be longer ones, even.

I thought at first that I had somehow got real corruption in the file that
was causing file truncation. This is a pain when I am trying to get a
large job out quickly, but I can cope with it.

What I am interested in, though, is whether the assembled experts would
expect this pattern of behaviour to occur with very long record IDs of
this type?

Alan

Best wishes
Alan Pritchard MPhil FCLIP MBCS
The GLOBAL GAZETTEER™: the world on file
http://www.allm-geodata.com
Tel: +44 (0) 1202 417 477

Please reply to: alan.pritchard (AT) gmail (DOT) com

Glen B wrote:
WTF?!? *crash* <--- The sound of frosty being knocked over by a feather.

--
frosty

In R83 days, there was a 50 byte workspace carved out of the users PCB that
held the item ID. I thought they expanded that when they went to 4K frames.
I'd have said there was no limit, or, if there was, it would be 50 bytes.

Is this file indexed? Long ID's might screw you up there, too.

Mark Brown

"Glen B" <no$pamwebmaster@no$pamforallspec.com> wrote

Alan,

I have often found that when I have bizarre selection problems checking
the D3 index is very useful. If you have an index try rebuilding it.
If that fails try deleting the index altogether and see if the problem
clears up.

Thanks for the various comments.

Picking up on some of the points.

1. I do have d3 indexes, but was not using them for this selection. The
index I mentioned in my OP is a local inverted file of an index key that I
maintain.

2. I am now going through each of these large-ID records, chopping the
ID down (or deleting the whole record). What is so odd, though, is that
some of these have been on for over two years with no problem at all. It
is only since the d3/Linux upgrade that problems have arisen.

Best wishes
Alan Pritchard MPhil FCLIP MBCS
The GLOBAL GAZETTEER™: the world on file
http://www.allm-geodata.com
Tel: +44 (0) 1202 417 477

Please reply to: alan.pritchard (AT) gmail (DOT) com

How easy would it be to re-engineer things so that the IDs are of a
more sensible length?

Who is responsible for designing a system with enormous IDs in the
first place?

These things can and should be a doddle to fix if everyone called
generic IO subroutines instead of doing direct
read/writes/deletes/releases in the code - sorta like a CALLX but
activated for reads etc., not just writes.

Alan Pritchard wrote:

In article <1152710480.312734.154890 (AT) m79g2000cwm (DOT) googlegroups.com>,
michael (AT) preece (DOT) net (Mike Preece) wrote:

Very difficult.

I am. The record key is a place or organisation name. I have never had a
problem in 11 years with these until a couple days ago
I am not sure that this would work in my case. I take in data from a wide
range of census, gazetteer, telephone area code etc data files, and each
one is different, so I have to write specific programs to reformat each
type into my standard database structure.

Alan


Best wishes
Alan Pritchard MPhil FCLIP MBCS
The GLOBAL GAZETTEER™: the world on file
http://www.allm-geodata.com
Tel: +44 (0) 1202 417 477

Please reply to: alan.pritchard (AT) gmail (DOT) com

Donno about D3 7.4 but in my experimentation 7.2.1 has a item id length
limit of 99 or 100 characters.

Regards,

Dale


"Alan Pritchard" <alan.pritchard (AT) gmail (DOT) com> wrote

01 execute "create-file keysize-file 1 7"
02 open "","keysize-file" to file else stop
03 mykey = ""
04 loop
05 mykey := "A"
06 while len(mykey) < 1000 do
07 read rec from file,mykey then
08 execute "delete-file keysize-file"
09 print len(mykey)-1 :" is max keysize"
10 stop
11 end
12 write "" on file,mykey
13 print @(0,24):len(mykey):
14 repeat

Gives 101 on D3/Linux 7.4.0, which corresponds "nicely" to the
observation. Numerous ways to make keys smaller, especially in
situations like I imagine you have here (we used to get very "tricky"
with data warehouse structures. This limit used to be much smaller, but
now we "cheat" with Visage.BIT structures and have "keys" much longer)

Dale wrote:

I cannot help but think about an post a few days old where 'collections'
where being emulated via temp files! Whoops... only works with certain data
sets (short data, no AM marks ..etc) - But hey! We don't need no 'new' ways
of doing stuff...

(Loving the move to 'modern' languages!)
dave

"Dale" <benedictknowspam (AT) silk (DOT) net> wrote