IDS 10FC6 HP-UX 11.11


My server went into DDR Block mode last night.. in the process of getting the rest of my systems back online, the instance got re-started. There was a bunch of data in the recv queue on the recieving server that finally got processed, but it appears that no data is replicating now. I think it is because the instance got rebooted while in DDR Block, so it kinda lost its mind and didnt know it was supposed to go into catch up mode.. I can deal with that -

My question has to do with the onstat -g ddr output (see below..) just what is the value under Tossed (LBC full) Im thinking that is all my data that is just being tossed and not replicating. Also the snoopy and replay position are not moving at all...

I have already resolved to stop/delete and recreate all my replicates (good thing I saved my scripts!) - but any additional light someone can shed on this would be appreciated.


IBM Informix Dynamic Server Version 10.00.FC6 -- On-Line -- Up 11:21:51 -- 1949156 Kbytes

DDR -- Running --

# Event Snoopy Snoopy Replay Replay Current Current
Buffers ID Position ID Position ID Position
3088 38464 c7fe700 38464 c7f919c 38501 4c654000

Log Pages Snooped:
From From Tossed
Cache Disk (LBC full)
0 51199 2538285

Total dynamic log requests: 0

DDR events queue

Type TX id Partnum Row id




Thanks

Laurie!

The value under Tossed is that as your current system is generating
log records and putting them in the logical log buffer, when the
logical log buffer is getting flushed to disk, CDR will attempt to
copy that logical log buffer from memory into a cdr specific log
record buffer cache for the snoopy thread so it wouldn't have to read
those records from disk. However, there is a limited amount of those
buffers and if that cache is already full, then we increment the
tossed count and then those records would just have to be snooped from
disk. I don't believe that would be a reason why stuff isn't
replicating. It looks to be more of a performance sort of value where
ideally the snooper would be running more quickly if it was able to
get it's work from this cache and not have to retrieve it from disk.

Did the replay postion get over written? When the server was bounced
was there any messages in the online.log concerning the ability of CDR
to start backup? If logical log id 38464 (uniqid from onstat -l
output) is no longer on disk (ie the reply position getting over
written), then yeah CDR is in big trouble, as that would be the spot
at which the snoopy thread needs to get to start back up snooping the
logs, and if it can't then I don't believe anything will get
replicated since the snooper isn't reading any log records. However,
if logical log id 38464 is still on disk and the replay position
hasn't been over written, without seeing other information it would be
hard to say why the replay position isn't advancing.

Jacques
IBM Informix

Hmmm - the replay position hasnt been overwritten. I have attached onstat -l and onstat -g ddr outputs.

I added a bunch of logs last night to make sure we didnt block again until I could do more research this morning (and think more clearly..)
as you can see from the onstat -l output, there are tons of new logs out there, but the system just dynamically created log #89. That really confused me.

so maybe the problem is just with my logs...

Thanks.
Laurie

The value under Tossed is that as your current system is generating
log records and putting them in the logical log buffer, when the
logical log buffer is getting flushed to disk, CDR will attempt to
copy that logical log buffer from memory into a cdr specific log
record buffer cache for the snoopy thread so it wouldn't have to read
those records from disk. However, there is a limited amount of those
buffers and if that cache is already full, then we increment the
tossed count and then those records would just have to be snooped from
disk. I don't believe that would be a reason why stuff isn't
replicating. It looks to be more of a performance sort of value where
ideally the snooper would be running more quickly if it was able to
get it's work from this cache and not have to retrieve it from disk.

Did the replay postion get over written? When the server was bounced
was there any messages in the online.log concerning the ability of CDR
to start backup? If logical log id 38464 (uniqid from onstat -l
output) is no longer on disk (ie the reply position getting over
written), then yeah CDR is in big trouble, as that would be the spot
at which the snoopy thread needs to get to start back up snooping the
logs, and if it can't then I don't believe anything will get
replicated since the snooper isn't reading any log records. However,
if logical log id 38464 is still on disk and the replay position
hasn't been over written, without seeing other information it would be
hard to say why the replay position isn't advancing.

Jacques
IBM Informix
_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Laurie Gustin wrote:
Do a cdr list server

onstat -g rqm brief

onstat -g nif

Perhaps attach an onstat -g cat

With the above output, may be able to determine if the problem is on the
source or target(s) of this server.

Laurie Gustin wrote:
onstat -g stk of the ddr thread?

UPDATE - The engine went into DDR Block mode again - taking down all my systems (again...) This time I restarted the engine, cdr stop on both servers.. then when I started cdr again, the snoopy and replay position appeared to be correct, and they were 'moving'. However, I am trying to cdr check-repair one replicate and it is taking forever (there are only 348 rows in the table and it has been running for over an hour.

How do I know which CDR thread? I did onstat -g stk all and just deleted all but the CDR stuff

Thing appear to be moving through... but my sync jobs still dont seem to be working.

Thanks in advance for any insight..

Laurie

Laurie Gustin
IT Programmer Analyst
Department of Public Safety
lgustin (AT) utah (DOT) gov
801-965-4410

onstat -g stk of the ddr thread?

Laurie Gustin wrote:
The cdr check --repair runs as an sqlexec thread, not as a CDR thread.
Are there blobs/sblobs ? That's the only reason that I can think of
that the repair would take so long.

maybe a trigger which updates a table without where clause which is
also replicated or executing an spl which does a lot of work...
no long trx rollbacks??????

i would check the schema of the tables...

Superboer