dbTalk Databases Forums  

One MySQL table is marked as crashed periodically

comp.databases.mysql comp.databases.mysql


Discuss One MySQL table is marked as crashed periodically in the comp.databases.mysql forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Ignoramus14242
 
Posts: n/a

Default One MySQL table is marked as crashed periodically - 02-10-2011 , 10:10 AM






I have a website algebra.com, and it uses mysql 5.0.51a on Ubuntu
Hardy. In the database, I have a sessions table to hold apache session
data. I have about 40,000 visitors per day, give or take. I try not to
delete old sessions to please users and to keep them logged on. I
think that I delete sessions twice a year or some such. So, the
sessions table is a little under 400 megabytes.

I noticed that a couple of times, the sessions table becomes marked as
"crashed" around 2am. That, naturally, screws up my website.

To go around it, I wrote a perl script that verifies tables and
repairs them as necessary, but it still pisses me off.

There are no system update jobs that run at 2am. The only thing that
runs at 2am is a backup job to save algebra.com's database into a
file.

The command is as follows:

mysqldump -u$MYSQL_USER -p$MYSQL_PASSWORD --opt $DBNAME \
--ignore-table algebra.sessions \
--ignore-table algebra.bincache \
--ignore-table algebra.usertrack \
--skip-lock-tables \
Quote:
gzip -9 > $FILE.new
Despite being asked to ignore the sessions table, it is still marked
as crashed. I am afraid that the table being marked as crashed, as
such, is even the correct action to take and the table indeed is
corrupted.

My question is, how can I track down what exactly corrupts this table?

Just one thing that I may go by, is that the database sits on a
mirrored pair of drives, behind a 3ware RAID controller. Once in a
great while, I see these messages in logs:

Feb 9 00:40:08 myserver syslogd 1.5.0#1ubuntu1: restart.
Feb 9 02:40:44 myserver kernel: [238455.984407] sd 6:0:3:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 02:41:16 myserver kernel: [238477.290460] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 02:41:16 myserver kernel: [238477.410343] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 02:41:16 myserver kernel: [238477.530214] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 02:42:39 myserver kernel: [238538.778075] sd 6:0:3:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 02:42:39 myserver kernel: [238559.846239] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 02:42:39 myserver kernel: [238559.966112] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 02:42:39 myserver kernel: [238560.085988] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 23:42:56 myserver kernel: [314109.742591] sd 6:0:2:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 23:43:32 myserver kernel: [314134.347098] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 23:43:32 myserver kernel: [314134.466971] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 23:43:32 myserver kernel: [314134.586847] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 23:50:50 myserver kernel: [314583.259057] sd 6:0:3:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 23:51:24 myserver kernel: [314604.380288] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 23:51:24 myserver kernel: [314604.500159] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 23:51:24 myserver kernel: [314604.620036] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=3.
Feb 10 01:07:28 myserver syslogd 1.5.0#1ubuntu1: restart.
Feb 10 02:41:23 myserver kernel: [324773.510168] sd 6:0:3:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 10 02:41:23 myserver kernel: [324794.926211] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.
Feb 10 02:41:23 myserver kernel: [324795.046083] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=2.
Feb 10 02:41:23 myserver kernel: [324795.165959] 3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=3.

I wonder if that stuff could be blamed for this?

i

Reply With Quote
  #2  
Old   
Lawrence D'Oliveiro
 
Posts: n/a

Default Re: One MySQL table is marked as crashed periodically - 02-10-2011 , 04:47 PM






In message <JYCdnS6P6qaNk8nQnZ2dnUVZ_hudnZ2d (AT) giganews (DOT) com>, Ignoramus14242
wrote:

Quote:
Just one thing that I may go by, is that the database sits on a
mirrored pair of drives, behind a 3ware RAID controller. Once in a
great while, I see these messages in logs:

Feb 9 02:40:44 myserver kernel: [238455.984407] sd 6:0:3:0: WARNING:
(0x06:0x002C): Command (0x2a) timed out, resetting card.
I wouldn’t trust hardware RAID. Use the Linux built-in (mdadm) software
RAID.

Reply With Quote
  #3  
Old   
andre
 
Posts: n/a

Default Re: One MySQL table is marked as crashed periodically - 02-11-2011 , 03:28 AM



Ignoramus14242 wrote:

Quote:
I have a website algebra.com, and it uses mysql 5.0.51a on Ubuntu
Hardy. In the database, I have a sessions table to hold apache session
data. I have about 40,000 visitors per day, give or take. I try not to
delete old sessions to please users and to keep them logged on. I
think that I delete sessions twice a year or some such. So, the
sessions table is a little under 400 megabytes.

I noticed that a couple of times, the sessions table becomes marked as
"crashed" around 2am. That, naturally, screws up my website.

To go around it, I wrote a perl script that verifies tables and
repairs them as necessary, but it still pisses me off.

There are no system update jobs that run at 2am. The only thing that
runs at 2am is a backup job to save algebra.com's database into a
file.

The command is as follows:

mysqldump -u$MYSQL_USER -p$MYSQL_PASSWORD --opt $DBNAME \
--ignore-table algebra.sessions \
--ignore-table algebra.bincache \
--ignore-table algebra.usertrack \
--skip-lock-tables \
| gzip -9 > $FILE.new

Despite being asked to ignore the sessions table, it is still marked
as crashed. I am afraid that the table being marked as crashed, as
such, is even the correct action to take and the table indeed is
corrupted.

My question is, how can I track down what exactly corrupts this table?

Just one thing that I may go by, is that the database sits on a
mirrored pair of drives, behind a 3ware RAID controller. Once in a
great while, I see these messages in logs:

Feb 9 00:40:08 myserver syslogd 1.5.0#1ubuntu1: restart.
Feb 9 02:40:44 myserver kernel: [238455.984407] sd 6:0:3:0: WARNING:
(0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 02:41:16 myserver kernel: [238477.290460] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 02:41:16 myserver kernel: [238477.410343] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 02:41:16 myserver kernel: [238477.530214] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 02:42:39 myserver kernel: [238538.778075] sd 6:0:3:0: WARNING:
(0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 02:42:39 myserver kernel: [238559.846239] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 02:42:39 myserver kernel: [238559.966112] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 02:42:39 myserver kernel: [238560.085988] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 23:42:56 myserver kernel: [314109.742591] sd 6:0:2:0: WARNING:
(0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 23:43:32 myserver kernel: [314134.347098] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 23:43:32 myserver kernel: [314134.466971] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 23:43:32 myserver kernel: [314134.586847] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=3.
Feb 9 23:50:50 myserver kernel: [314583.259057] sd 6:0:3:0: WARNING:
(0x06:0x002C): Command (0x2a) timed out, resetting card.
Feb 9 23:51:24 myserver kernel: [314604.380288] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=0.
Feb 9 23:51:24 myserver kernel: [314604.500159] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=2.
Feb 9 23:51:24 myserver kernel: [314604.620036] 3w-9xxx: scsi6: AEN: INFO
(0x04:0x005E): Cache synchronization completed:unit=3. Feb 10 01:07:28
myserver syslogd 1.5.0#1ubuntu1: restart. Feb 10 02:41:23 myserver kernel:
[324773.510168] sd 6:0:3:0: WARNING: (0x06:0x002C): Command (0x2a) timed
out, resetting card. Feb 10 02:41:23 myserver kernel: [324794.926211]
3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization
completed:unit=0. Feb 10 02:41:23 myserver kernel: [324795.046083]
3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization
completed:unit=2. Feb 10 02:41:23 myserver kernel: [324795.165959]
3w-9xxx: scsi6: AEN: INFO (0x04:0x005E): Cache synchronization
completed:unit=3.

I wonder if that stuff could be blamed for this?

i
Look like you have disk problems on your RAID.
André

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.