Folks,

I've a 7.2.4 report-generation database that has been growing for some time,
resuting in the nightly VACUUM FULL ANALYZE taking longer and longer. (most
of the data is copied nightly from other systems, so use of FSM is not very
effective).

The problem is that the nightly admin scripts are programmed to check for a
locked up nightly maintainence, and to "pg_ctl -m fast stop" it. As the
VACUUM FULL now takes over an hour, it falsely detected a lockup and shutdown
the database in the middle of VACUUM.

On restarting the database, I manually VACUUM FULLed it, and the VACUUM would
speed through until hitting the spot where the database was shutdown, at
which point the VACUUM process went "D", and apparently locked up for 10
minutes. No error messages were written to the logs. Unfortunately, I could
not give it longer to see if it recovered because this is a production system
and I had to get it up and running from backup by 9am.

Does this sound like a crash during VACUUM, or just like it needed more time?

If anyone wants to analyze, I have a complete backup of the post-problem
PGDATA directory. The host system is RH Linux 8.0.

--
Josh Berkus
Aglio Database Solutions
San Francisco

---------------------------(end of broadcast)---------------------------
TIP 6: Have you searched our list archives?

http://archives.postgresql.org

Josh Berkus <josh (AT) agliodbs (DOT) com> writes:
I think it just needed more time. VACUUM goes to great lengths to be
crash-safe. I doubt that a "fast stop" could have left the database
in a corrupted state.

It is entirely likely that an interrupted VACUUM FULL would leave the
next VACUUM FULL with more, not less, work to do --- all of the tuples
the first one tried to move before being killed would now be dead and
need to be cleaned up.

Are you saying that you delete most or all of the rows, then vacuum?
You might consider TRUNCATE if you delete all the rows, or CLUSTER
if you delete most, as a substitute for VACUUM FULL. (You'd still want
to run ANALYZE, after you load fresh data.) VACUUM FULL is really
designed for the case where there are not a huge number of dead rows
--- it gets awfully slow if it has to move lots of data.

Also, I think you have probably not given the FSM enough chance.
If the FSM settings are adequate then it should work fine to do
-- delete unwanted rows
VACUUM (not FULL, not ANALYZE)
-- load new rows
ANALYZE

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Josh Berkus <josh (AT) agliodbs (DOT) com> writes:
Disk I/O wait state, no? Pretty much what I'd expect for VACUUM ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings