Hi All,



I'm running two installations, both using II9.2.0+p13556. Call the two
installations A and B.



I'm using incremental recovery to keep a warm spare copy of database
A::ace_pack_live on B.



The hosts differ in the settings of paths for the database locations but
I generally solve that problem with a few symlinks.



I do an initial recovery of A::ace_pack_live on B by:

1. Create an empty database on B which looks like A::ace_pack_live.

2. Copying in all the journals, dumps, checkpoints and configuration
files from A to the correct locations on B.

3. rollforwarddb ace_pack_live +c -j -incremental -norollback



At the end of which the database is accessible and all appears to be OK.



I do subsequent journals by copying in the new journals from A to the
correct locations on B and doing:

rollforwarddb ace_pack_live -c +j -incremental -norollback



However, after a few of these, although I get no error on the
rollforwarddb command, I can no longer connect to the B::ace_pack_live.
Instead it fails at the front end with:



E_QE0086 Error beginning a transaction.

E_SC0206 An internal error prevents further processing of this query.



And in the errlog we see:

E_DM9004_BAD_FILE_OPEN Disk file open error on database:ace_pack_live
table:Not a table
pathname:/ace_vol_1/II/ingres/data/default/ace_pack_live
filename:aaaaaaaa.cnf

open() failed with operating system error 2 (No such file or directory)

E_DM012A_CONFIG_NOT_FOUND The database configuration file was not found.

E_DM9413_RCP_COUNT_OPENS Error tracking open/close database operations
in the recovery process.

E_CL0F0F_LG_DB_INCONSISTENT One of the databases has become
inconsistent. No new transactions may be begun against the inconsistent
database, and no new update operations against the inconsistent database
may be performed by transactions already underway.



But I can easily do an 'ls' on the config file path. Its there!



infodb on ace_pack_live shows a status of:
VALID,JOURNAL,DUMP,ROLL_FORWARD,CFG_BACKUP,INCR_RF P



infodb over the entire installation suggests there are no inconsistent
databases, yet logstat believes there are.



Am I doing something wrong or is this a bug of some kind?



Martin Bowes

Hi Martin

Can you please tell which is the exact sequence of events ?
Can you please post the infodb for ace_pack_live both A and B along
with the relevant section from errlog.log/iircp.log?
I assume symlinks are ok (did not change). The output from B, of
logdump -v along with auditdb -v -all may help as well.

Thanks
Armand


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile: http://community.ingres.com/forum/me...hp?userid=2932
View this thread: http://community.ingres.com/forum/sh...ad.php?t=11556

Hi Armand,

I've just raised a sev 2 item with tech support on this. Issue number is
142110.

The problem appears to be related to the symlinked path. Furthermore, it
only effects the incremental recovery.

This is a little tricky to follow but bear with me...
You need two installations. Call them A and B. We will recover
A::bowtest to B using incremental recovery.

On A we have a location ii_database_6 whose path starts at /ace_vol_1/II
On B we have a location ii_database_6 whose path starts at
/san/dbdata1/II

To make the paths equivalent, on B we install a symlink named ace_vol_1
which points at /san/dbdata1.

Note that both A and B have the same path for II_DATABASE.

1. Lets try an incremental recovery using the symlinked path...
So on A...
createdb bowtest -dii_database_6
ckpdb -v +j bowtest
sql bowtest <<SQL_END
create table xx (a integer, b long varchar) with journaling; commit\g
\q
SQL_END
ckpdb -v +j bowtest

Now transfer all the checkpoints, dumps, journals, config files and
saved config files from A to holding directories on B. These directories
on B are named as follows:
The checkpoint stuff is kept in: /dbbackup/II/ingres/ckp/A.II/bowtest
The journal stuff (and data area config file) is kept in:
/dbbackup/II/ingres/jnl/A.II/bowtest
The dump stuff is in: /dbbackup/II/ingres/dmp/A.II/bowtest

So on B lets set up the incremental recovery...
"/dbsystem/II/ingres/bin/createdb" bowtest -dii_database_6
"/dbsystem/II/ingres/bin/ckpdb" +j bowtest
cd /ace_vol_1/II/ingres/data/default/bowtest
mv aaaaaaaa.cnf aaaaaaaa.cnf.orig
cp -p /dbbackup/II/ingres/jnl/A.II/bowtest/aaaaaaaa.cnf .

cd /dbbackup/II/ingres/dmp/default/bowtest
mv aaaaaaaa.cnf aaaaaaaa.cnf.orig
cp -p /dbbackup/II/ingres/jnl/A.II/bowtest/aaaaaaaa.cnf .

cp -p /dbbackup/II/ingres/dmp/A.II/bowtest/c0000002.* .

cd /dbbackup/II/ingres/jnl/default/bowtest
cp -p /dbbackup/II/ingres/jnl/A.II/bowtest/j0000002.* .

cd /dbbackup/II/ingres/ckp/default/bowtest
ln -s /dbbackup/II/ingres/ckp/A.II/bowtest/c0002001.ckp

cd /user/ingres/recovery/bowtest
"/dbsystem/II/ingres/bin/rollforwarddb" +c -j bowtest -incremental
-norollback

This completes with NO error.

But an attempt to connect to the database will fail...
sql bowtest
INGRES TERMINAL MONITOR Copyright 2008 Ingres Corporation
E_QE0086 Error beginning a transaction.
etc
And the errlog bitches about not being abl to find the config file....
Yet it clearly exists...
ls -l /ace_vol_1/II/ingres/data/default/bowtest/aaaaaaaa.cnf
-rw------- 1 ingres ingres 8192 Jan 14 10:55
/ace_vol_1/II/ingres/data/default/bowtest/aaaaaaaa.cnf

And at the same time we can do an infodb and get Status:
VALID,JOURNAL,ROLL_FORWARD,CFG_BACKUP,INCR_RFP

2. Now destroy the database on A and B and repeat without the
-dii_database_6.
This will have the efect of using ii_database which is the same path on
both A and B. This time you will be able to connect to the database on B
after the incremental recovery.

BTW. If I do the symlinked recovery as a normal recovery (not
incremental) then B::bowtest is perfectly OK.

Marty


-----Original Message-----
From: info-ingres-bounces (AT) kettleriver...ting (DOT) com
[mailto:info-ingres-bounces (AT) kettleriverconsulting (DOT) com] On Behalf Of
Ingres Forums
Sent: 13 January 2010 17:53
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: Re: [Info-Ingres] incremental recovery failure


Hi Martin

Can you please tell which is the exact sequence of events ?
Can you please post the infodb for ace_pack_live both A and B along
with the relevant section from errlog.log/iircp.log?
I assume symlinks are ok (did not change). The output from B, of
logdump -v along with auditdb -v -all may help as well.

Thanks
Armand


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile:
http://community.ingres.com/forum/me...hp?userid=2932
View this thread:
http://community.ingres.com/forum/sh...ad.php?t=11556

_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Hi Martin

Thanks for your info.
Are there two machines or two installations on the same machine ?
I noticed this "Note that both A and B have the same path for
II_DATABASE." So I can assume two machines each with it's own
installation. The only difference being ii_database_6 which on each
machine points to a different directory.
Will see if I can reproduce and keep you updated.

Thanks
Armand


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile: http://community.ingres.com/forum/me...hp?userid=2932
View this thread: http://community.ingres.com/forum/sh...ad.php?t=11556

Hi Martin

This is reproducible and unless I am missing something, hence I believe
this should be reported as a bug or at least discussed as such in the
ticket you have opened. IMHO incremental or not, the behavior should be
the same across flavors. On the same subject it could be some
architectural limit , therefore if this is determined to be such, it
should be the subject of a software improvement request.
I was wondering if incremental rollforwarddb is happy with other data
locations than the default (i.e. used with -d) but testing shows that
this is not the case.
I also considered the db_id (which could be different among different
installations even if we have the same db name) and testing shows that
this is not the case either.


E_DM9004_BAD_FILE_OPEN Disk file open error on database:rolldb
table:Not a table
pathname:/home/ingres/ingresb1/ingres/data/default/rolldb
filename:aaaaaaaa.cnf
E_DM012A_CONFIG_NOT_FOUND The database configuration file was not
found.
E_DM9413_RCP_COUNT_OPENS Error tracking open/close database operations
in the recovery process.
E_CL0F0F_LG_DB_INCONSISTENT One of the databases has become
inconsistent. No new transactions may be begun against the inconsistent
database, and no new update operations against the inconsistent database
may be performed by transactions already underway.
E_DM900C_BAD_LOG_BEGIN Error trying to begin a transaction on the
database 00440012.
E_DM9500_DMXE_BEGIN Error occurred beginning a transaction.
E_QE0025_USER_ERROR A user error has occurred.
E_CL0F0F_LG_DB_INCONSISTENT One of the databases has become
inconsistent. No new transactions may be begun against the inconsistent
database, and no new update operations against the inconsistent database
may be performed by transactions already underway.

ls -l /home/ingres/ingresb1/ingres/data/default/rolldb/*cnf
-rw------- 1 ingres ingres 8192 2010-01-14 12:25
/home/ingres/ingresb1/ingres/data/default/rolldb/aaaaaaaa.cnf

Hope this helps

Thanks
Armand
Database : (rolldb,ingres) ID : 0x4B4F57FF Default collation :
default
Unicode enabled : No
Extents : 5 Last Table Id : 239
Config File Version Id : 0x00060001 Database Version Id : 8
Mode : DDL ALLOWED, ONLINE CHECKPOINT ENABLED
Status : VALID,JOURNAL,ROLL_FORWARD,CFG_BACKUP,INCR_RFP


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile: http://community.ingres.com/forum/me...hp?userid=2932
View this thread: http://community.ingres.com/forum/sh...ad.php?t=11556

Hi Armand,

Two machines.

Marty

-----Original Message-----
From: info-ingres-bounces (AT) kettleriver...ting (DOT) com
[mailto:info-ingres-bounces (AT) kettleriverconsulting (DOT) com] On Behalf Of
Ingres Forums
Sent: 14 January 2010 17:36
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: Re: [Info-Ingres] incremental recovery failure


Hi Martin

Thanks for your info.
Are there two machines or two installations on the same machine ?
I noticed this "Note that both A and B have the same path for
II_DATABASE." So I can assume two machines each with it's own
installation. The only difference being ii_database_6 which on each
machine points to a different directory.
Will see if I can reproduce and keep you updated.

Thanks
Armand


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile:
http://community.ingres.com/forum/me...hp?userid=2932
View this thread:
http://community.ingres.com/forum/sh...ad.php?t=11556

_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Hi Martin

As I stated before, this is reproducible at will. IMHO this should be
reported as a bug. But if there is some architectural consideration in
incremental rollforwarddb, this should be then the subject of a software
improvement request.

Hope this helps

Thanks
Armand


--
armand.pirvu (AT) ingres (DOT) com
------------------------------------------------------------------------
armand.pirvu (AT) ingres (DOT) com's Profile: http://community.ingres.com/forum/me...hp?userid=2932
View this thread: http://community.ingres.com/forum/sh...ad.php?t=11556