dbTalk Databases Forums  

Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1]

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-24-2004 , 02:56 PM






=?UTF-8?Q?=E5=A7=9C?= =?UTF-8?Q?=E7=BB=B4?= <jiangwei_1976 (AT) yahoo (DOT) com.cn> writes:
Quote:
BEGIN;
...
...
...
END;

PANIC: invalid xlog record length 236052
server closed the connection unexpectedly
This is quite unhelpful, if you're not going to show us what you did to
cause it.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings


Reply With Quote
  #2  
Old   
Alvaro Herrera
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-24-2004 , 03:08 PM






On Sun, Aug 22, 2004 at 09:39:07AM +0800, ?????? wrote:
Quote:
BEGIN;
...
...
...
END;

PANIC: invalid xlog record length 236052
Huh, so what kind of operations did you execute within the transaction?

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
Voy a acabar con todos los humanos / con los humanos yo acabaré
voy a acabar con todos / con todos los humanos acabaré (Bender)


---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match


Reply With Quote
  #3  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-24-2004 , 05:12 PM



Alvaro Herrera <alvherre (AT) dcc (DOT) uchile.cl> writes:
Quote:
On Sun, Aug 22, 2004 at 09:39:07AM +0800, ?????? wrote:
PANIC: invalid xlog record length 236052

Huh, so what kind of operations did you execute within the transaction?
I found one possible explanation, though I don't know if it's the
submitter's problem or not. Create a SQL file that generates a whole
lot of subtransactions, like more than 16000. I used

begin;
create table foo(d1 int);
drop table foo;
savepoint x;
release x;
-- repeat above 2 lines 20000 times
commit;

First try gave me

WARNING: out of shared memory
ERROR: out of shared memory
HINT: You may need to increase max_locks_per_transaction.
WARNING: StartAbortedSubTransaction while in START state
ERROR: current transaction is aborted, commands ignored until end of transaction block
ERROR: current transaction is aborted, commands ignored until end of transaction block
.... etc ...

which is fine except for the StartAbortedSubTransaction warning; that
may indicate a problem. (What do you think about it, Alvaro?)

I bumped up max_locks_per_transaction to 1000 and tried again, and got

psql:zzbig.sql:40004: PANIC: invalid xlog record length 80024
server closed the connection unexpectedly

What is happening of course is that more than 16K subtransaction IDs
won't fit in a commit record (since XLOG records have a 16-bit length
field). We're gonna have to rethink the representation of subxact
commit in XLOG.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html


Reply With Quote
  #4  
Old   
Alvaro Herrera
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-24-2004 , 11:10 PM



On Tue, Aug 24, 2004 at 06:10:40PM -0400, Tom Lane wrote:

Quote:
WARNING: out of shared memory
ERROR: out of shared memory
HINT: You may need to increase max_locks_per_transaction.
WARNING: StartAbortedSubTransaction while in START state
ERROR: current transaction is aborted, commands ignored until end of transaction block
ERROR: current transaction is aborted, commands ignored until end of transaction block
... etc ...

which is fine except for the StartAbortedSubTransaction warning; that
may indicate a problem. (What do you think about it, Alvaro?)
I think the problem here is that we can't "safely" call
StartAbortedSubTransaction when in TRANS_START state. This is because
we could have some subsystems already initialized, and we will be
initializing them again. For example, AtSubStart_Memory() will be
called twice, which will lead to the loss of an (empty) memory context.
It doesn't seem a big problem, only a memory leak.

(We also lose a ResourceOwner, but since it doesn't own anything yet, it
isn't a problem.)

Fortunately I think we are good regarding other subsystems --- for
example if we happened to do something with sinval.c list-of-lists, it
could get out of sync with the transaction stack and it could get ugly.
But we don't.

I don't think we should get rid of the warning. It shows that there's a
problem, but it's not critical. We could set a flag indicating that
memory and resource owner are already initialized (another TRANS state?
a static bool?), but I don't know if it's worth the trouble.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"The West won the world not by the superiority of its ideas or values
or religion but rather by its superiority in applying organized violence.
Westerners often forget this fact, non-Westerners never do."
(Samuel P. Huntington)


---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


Reply With Quote
  #5  
Old   
˙ffffce˙ffffac ˙ffffbd˙ffffaa
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-25-2004 , 08:00 PM




--- Alvaro Herrera <alvherre (AT) dcc (DOT) uchile.cl> wrote:

Quote:
On Sun, Aug 22, 2004 at 09:39:07AM +0800, ??????
wrote:
BEGIN;
...
...
...
END;

PANIC: invalid xlog record length 236052

Huh, so what kind of operations did you execute
within the transaction?

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
Voy a acabar con todos los humanos / con los humanos
yo acabar?> voy a acabar con todos / con todos los
humanos
acabar?(Bender)


-------------------example 1--------------------
$ echo "BEGIN;" > backup.sql
$ pg_dump -o >> backup.sql
$ echo "END;" >> backup.sql
....
$ psql -f backup.sql

PANIC: invalid xlog record length 236052


----------------example 2 ------------------------

There are 1600 tables in database 'db1', I wrote a
pl/pgsql function "update_tables" like

"
FOR table IN SELECT relname FROM pg_class
LOOP
...
DROP INDEX ON ... ;
ALTER TABLE DROP CONSTRAINT ...;
...
CREATE INDEX xxx ON TABLE xxx;
...
ALTER TABLE xxx ADD PRIMARY KEY...
ALTER TABLE xxx ADD ...
...
END LOOP
....
"

$ select update_tables();

PANIC: invalid xlog record length 236052




__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo (AT) postgresql (DOT) org)


Reply With Quote
  #6  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-25-2004 , 09:27 PM



"˙ffffce˙ffffac" "˙ffffbd˙ffffaa" <jiangwei_1976 (AT) yahoo (DOT) com.cn> writes:
Quote:
--- Alvaro Herrera <alvherre (AT) dcc (DOT) uchile.cl> wrote:
Huh, so what kind of operations did you execute
within the transaction?

There are 1600 tables in database 'db1', I wrote a
pl/pgsql function "update_tables" like

"
FOR table IN SELECT relname FROM pg_class
LOOP
...
DROP INDEX ON ... ;
ALTER TABLE DROP CONSTRAINT ...;
...
CREATE INDEX xxx ON TABLE xxx;
...
ALTER TABLE xxx ADD PRIMARY KEY...
ALTER TABLE xxx ADD ...
...
END LOOP
Okay, so it was the number-of-deleted-files issue and not the
number-of-subtransactions issue. Still says we have to allow
commit records to be bigger than 64K ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend


Reply With Quote
  #7  
Old   
Bruce Momjian
 
Posts: n/a

Default Re: [BUGS] server crash in very big transaction [postgresql 8.0beta1] - 08-28-2004 , 10:01 PM




Is this fixed?

---------------------------------------------------------------------------

Tom Lane wrote:
Quote:
"˙ffffce˙ffffac" "˙ffffbd˙ffffaa" <jiangwei_1976 (AT) yahoo (DOT) com.cn> writes:
--- Alvaro Herrera <alvherre (AT) dcc (DOT) uchile.cl> wrote:
Huh, so what kind of operations did you execute
within the transaction?

There are 1600 tables in database 'db1', I wrote a
pl/pgsql function "update_tables" like

"
FOR table IN SELECT relname FROM pg_class
LOOP
...
DROP INDEX ON ... ;
ALTER TABLE DROP CONSTRAINT ...;
...
CREATE INDEX xxx ON TABLE xxx;
...
ALTER TABLE xxx ADD PRIMARY KEY...
ALTER TABLE xxx ADD ...
...
END LOOP

Okay, so it was the number-of-deleted-files issue and not the
number-of-subtransactions issue. Still says we have to allow
commit records to be bigger than 64K ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

--
Bruce Momjian | http://candle.pha.pa.us
pgman (AT) candle (DOT) pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.