server auto-restarts and ipcs

#11

Greg Stark <gsstark (AT) MIT (DOT) EDU> writes:

Quote:

At least in 2.6 it seems to avoid reuse of ids by keeping a global counter
that is incremented every time a segment is created which ranges from 0..128k
that it multiplies by 32k and adds to the array index (which is reused
quickly).

So it doesn't seem plausible that there was an id collision unless this was
different in 2.4.20. However looking at his list of ids they're all separated
by multiples of 32769 which is what you would expect from this algorithm at
least until they start being reused.

Oh I missed the fact that you were talking about after a reboot. So the
algorithm I described would produce exactly the same sequence of ids after any
reboot given the same sequence of creation and deletions. Even if there's a
different sequence as long as the n'th creation is for the m'th array slot it
would get the same id. So collisions would be very common.

(though it seems the sequence is shared across all the ipc objects.)

--
greg

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

#12

Greg Stark <gsstark (AT) MIT (DOT) EDU> writes:

Quote:

Oh I missed the fact that you were talking about after a reboot. So the
algorithm I described would produce exactly the same sequence of ids after any
reboot given the same sequence of creation and deletions. Even if there's a
different sequence as long as the n'th creation is for the m'th array slot it
would get the same id. So collisions would be very common.

This seems to square with Ed's complaint that he frequently sees a
collision after a reboot. I've just committed some code that makes a
more extensive check as to whether a pre-existing segment actually has
any relevance to our data directory; should fix the problem.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

#13

On Tuesday November 9 2004 1:37, Tom Lane wrote:

Quote:

The shared memory block would certainly not still exist after a system
reboot, so what we have here is a misleading error message. Looking
at the code, the most plausible explanation appears to be that
shmctl(IPC_STAT) is failing (which it ought to) and returning some
errno code different from EINVAL (which is the case we are expecting
to see).

I believe the attached patch will fix this problem for you, at least on
the assumption that you are starting only one postmaster at system boot.

Just realizing we do start multiple postmasters under same user id when
upgrading a cluster (one on old port, one on new).

I noticed that ipcs on my linux box has a command-line option to list the
pid that created the segment. Not sure if such a library exists in usable
form, but looking for segments owned by the downed postmaster's pid would
seem to be what is needed. Just a thought...

Ed

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo (AT) postgresql (DOT) org so that your
message can get through to the mailing list cleanly

#14

"Ed L." <pgsql (AT) bluepolka (DOT) net> writes:

Quote:

I noticed that ipcs on my linux box has a command-line option to list the
pid that created the segment. Not sure if such a library exists in usable
form, but looking for segments owned by the downed postmaster's pid would
seem to be what is needed. Just a thought...

[ thinks about it... ] Nah, it's still not bulletproof, because in a
system reboot situation you can't trust the old PID either. It could
easy be that the other guy gets both the PID and the shmem ID that
belonged to you last time.

I've committed changes for 8.0 that mark a shmem segment with the inode
of the associated data directory; that should be a stable enough ID to
handle all routine-reboot cases. (If you had to restore your whole
filesystem from backup tapes, it might be wrong, but you're going to be
doing such recovery manually anyway ...)

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

#15

On Tuesday November 9 2004 4:35, Tom Lane wrote:

Quote:

"Ed L." <pgsql (AT) bluepolka (DOT) net> writes:
I noticed that ipcs on my linux box has a command-line option to list
the pid that created the segment. Not sure if such a library exists in
usable form, but looking for segments owned by the downed postmaster's
pid would seem to be what is needed. Just a thought...

[ thinks about it... ] Nah, it's still not bulletproof, because in a
system reboot situation you can't trust the old PID either. It could
easy be that the other guy gets both the PID and the shmem ID that
belonged to you last time.

I see. Ipcs on my box also lists the date/time of shared memory segment
attach/detach/change (ipcs -t), but ...

Quote:

I've committed changes for 8.0 that mark a shmem segment with the inode
of the associated data directory; that should be a stable enough ID to
handle all routine-reboot cases. (If you had to restore your whole
filesystem from backup tapes, it might be wrong, but you're going to be
doing such recovery manually anyway ...)

....that will remove a major hassle for us and lots of other. Thanks.

Ed

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

server auto-restarts and ipcs

comp.databases.postgresql.general comp.databases.postgresql.general

Re: server auto-restarts and ipcs - 11-09-2004 , 02:52 PM

Re: server auto-restarts and ipcs - 11-09-2004 , 03:36 PM

Re: server auto-restarts and ipcs - 11-09-2004 , 05:28 PM

Re: server auto-restarts and ipcs - 11-09-2004 , 05:35 PM

Re: server auto-restarts and ipcs - 11-09-2004 , 06:20 PM