Using myisamchk for Table Maintenance and Crash Recovery

Starting with MySQL Version 3.23.13, you can check MyISAM tables with the CHECK TABLE command. See CHECK TABLE. You can repair tables with the REPAIR TABLE command. See REPAIR TABLE.

To check/repair MyISAM tables (.MYI and .MYD) you should use the myisamchk utility. To check/repair ISAM tables (.ISM and .ISD) you should use the isamchk utility. See Table types.

In the following text we will talk about myisamchk, but everything also applies to the old isamchk.

You can use the myisamchk utility to get information about your database tables, check and repair them, or optimize them. The following sections describe how to invoke myisamchk (including a description of its options), how to set up a table maintenance schedule, and how to use myisamchk to perform its various functions.

You can, in most cases, also use the command OPTIMIZE TABLES to optimize and repair tables, but this is not as fast or reliable (in case of real fatal errors) as myisamchk. On the other hand, OPTIMIZE TABLE is easier to use and you don't have to worry about flushing tables. See OPTIMIZE TABLE.

Even though the repair in myisamchk is quite secure, it's always a good idea to make a backup before doing a repair (or anything that could make a lot of changes to a table)

myisamchk Invocation Syntax

myisamchk is invoked like this:

shell> myisamchk [options] tbl_name

The options specify what you want myisamchk to do. They are described here. (You can also get a list of options by invoking myisamchk --help.) With no options, myisamchk simply checks your table. To get more information or to tell myisamchk to take corrective action, specify options as described here and in the following sections.

tbl_name is the database table you want to check/repair. If you run myisamchk somewhere other than in the database directory, you must specify the path to the file, because myisamchk has no idea where your database is located. Actually, myisamchk doesn't care whether the files you are working on are located in a database directory; you can copy the files that correspond to a database table into another location and perform recovery operations on them there.

You can name several tables on the myisamchk command-line if you wish. You can also specify a name as an index file name (with the .MYI suffix), which allows you to specify all tables in a directory by using the pattern *.MYI. For example, if you are in a database directory, you can check all the tables in the directory like this:

shell> myisamchk *.MYI

If you are not in the database directory, you can check all the tables there by specifying the path to the directory:

shell> myisamchk /path/to/database_dir/*.MYI

You can even check all tables in all databases by specifying a wildcard with the path to the MySQL data directory:

shell> myisamchk /path/to/datadir/*/*.MYI

The recommended way to quickly check all tables is:

myisamchk --silent --fast /path/to/datadir/*/*.MYI
isamchk --silent /path/to/datadir/*/*.ISM

If you want to check all tables and repair all tables that are corrupted, you can use the following line:

myisamchk --silent --force --fast --update-state -O key_buffer=64M \
          -O sort_buffer=64M -O read_buffer=1M -O write_buffer=1M \
          /path/to/datadir/*/*.MYI
isamchk --silent --force -O key_buffer=64M -O sort_buffer=64M \
        -O read_buffer=1M -O write_buffer=1M /path/to/datadir/*/*.ISM

The above assumes that you have more than 64 M free.

Note that if you get an error like:

myisamchk: warning: 1 clients is using or hasn't closed the table properly

This means that you are trying to check a table that has been updated by another program (like the mysqld server) that hasn't yet closed the file or that has died without closing the file properly.

If mysqld is running, you must force a sync/close of all tables with FLUSH TABLES and ensure that no one is using the tables while you are running myisamchk. In MySQL Version 3.23 the easiest way to avoid this problem is to use CHECK TABLE instead of myisamchk to check tables.

General Options for myisamchk

myisamchk supports the following options.

-# or --debug=debug_options Output debug log. The debug_options string often is 'd:t:o,filename'.
-? or --help Display a help message and exit.
-O name=value, --set-variable=name=value Set the value of a variable. Please note that --set-variable=name=value and -O name=value syntax is deprecated as of MySQL 4.0. Use --name=value instead. The possible variables and their default values for myisamchk can be examined with myisamchk --help:
VariableValue
key_buffer_size 523264
read_buffer_size 262136
write_buffer_size 262136
sort_buffer_size 2097144
sort_key_blocks 16
decode_bits 9
-s or --silent Silent mode. Write output only when errors occur. You can use -s twice (-ss) to make myisamchk very silent.  
-v or --verbose Verbose mode. Print more information. This can be used with -d and -e. Use -v multiple times (-vv, -vvv) for more verbosity!  
-V or --version Print the myisamchk version and exit.  
-w or, --wait Instead of giving an error if the table is locked, wait until the table is unlocked before continuing. Note that if you are running mysqld on the table with --skip-external-locking, the table can only be locked by another myisamchk command.  

Check Options for myisamchk

-c or --check Check table for errors. This is the default operation if you are not giving myisamchk any options that override this.
-e or --extend-check Check the table very thoroughly (which is quite slow if you have many indexes). This option should only be used in extreme cases. Normally, myisamchk or myisamchk --medium-check should, in most cases, be able to find out if there are any errors in the table. If you are using --extended-check and have much memory, you should increase the value of key_buffer_size a lot!
-F or --fast Check only tables that haven't been closed properly.
-C or --check-only-changed Check only tables that have changed since the last check.
-f or --force Restart myisamchk with -r (repair) on the table, if myisamchk finds any errors in the table.
-i or --information Print informational statistics about the table that is checked.
-m or --medium-check Faster than extended-check, but only finds 99.99% of all errors. Should, however, be good enough for most cases.
-U or --update-state Store in the .MYI file when the table was checked and if the table crashed. This should be used to get full benefit of the --check-only-changed option, but you shouldn't use this option if the mysqld server is using the table and you are running mysqld with --skip-external-locking.
-T or --read-only Don't mark table as checked. This is useful if you use myisamchk to check a table that is in use by some other application that doesn't use locking (like mysqld --skip-external-locking).

Repair Options for myisamchk

The following options are used if you start myisamchk with -r or -o:

-B or --backup Make a backup of the .MYD file as filename-time.BAK
--correct-checksum Correct checksum information for table.
-D # or --data-file-length=# Max length of datafile (when re-creating datafile when it's 'full').
-e or --extend-check Try to recover every possible row from the datafile. Normally this will also find a lot of garbage rows. Don't use this option if you are not totally desperate.
-f or --force Overwrite old temporary files (table_name.TMD) instead of aborting.
-k # or --keys-used=# If you are using ISAM, tells the ISAM storage engine to update only the first # indexes. If you are using MyISAM, tells which keys to use, where each binary bit stands for one key (first key is bit 0). This can be used to get faster inserts! Deactivated indexes can be reactivated by using myisamchk -r.
-l or --no-symlinks Do not follow symbolic links. Normally myisamchk repairs the table a symlink points at. This option doesn't exist in MySQL 4.0, as MySQL 4.0 will not remove symlinks during repair.
-p or --parallel-recover Uses the same technique as -r and -n, but creates all the keys in parallel, in different threads. This option was added in MySQL 4.0.2. This is alpha code. Use at your own risk!
-r or --recover Can fix almost anything except unique keys that aren't unique (which is an extremely unlikely error with ISAM/MyISAM tables). If you want to recover a table, this is the option to try first. Only if myisamchk reports that the table can't be recovered by -r, you should then try -o. (Note that in the unlikely case that -r fails, the datafile is still intact.) If you have lots of memory, you should increase the size of sort_buffer_size!
-o or --safe-recover Uses an old recovery method (reads through all rows in order and updates all index trees based on the found rows); this is an order of magnitude slower than -r, but can handle a couple of very unlikely cases that -r cannot handle. This recovery method also uses much less disk space than -r. Normally one should always first repair with -r, and only if this fails use -o. If you have lots of memory, you should increase the size of key_buffer_size!
-n or --sort-recover Force myisamchk to use sorting to resolve the keys even if the temporary files should be very big.
--character-sets-dir=... Directory where character sets are stored.
--set-character-set=name Change the character set used by the index
-t or --tmpdir=path Path for storing temporary files. If this is not set, myisamchk will use the environment variable TMPDIR for this. Starting from MySQL 4.1, tmpdir can be set to a list of paths separated by colon : (semicolon ; on Windows). They will be used in round-robin fashion.
-q or --quick Faster repair by not modifying the datafile. One can give a second -q to force myisamchk to modify the original datafile in case of duplicate keys
-u or --unpack Unpack file packed with myisampack.

Other Options for myisamchk

Other actions that myisamchk can do, besides repair and check tables:

-a or --analyze Analyze the distribution of keys. This improves join performance by enabling the join optimizer to better choose in which order it should join the tables and which keys it should use: myisamchk --describe --verbose table_name' or using SHOW KEYS in MySQL.
-d or --description Prints some information about table.
-A or --set-auto-increment[=value] Force AUTO_INCREMENT to start at this or higher value. If no value is given, then sets the next AUTO_INCREMENT value to the highest used value for the auto key + 1.
-S or --sort-index Sort the index tree blocks in high-low order. This will optimize seeks and will make table scanning by key faster.
-R or --sort-records=# Sorts records according to an index. This makes your data much more localized and may speed up ranged SELECT and ORDER BY operations on this index. (It may be very slow to do a sort the first time!) To find out a table's index numbers, use SHOW INDEX, which shows a table's indexes in the same order that myisamchk sees them. Indexes are numbered beginning with 1.

myisamchk Memory Usage

Memory allocation is important when you run myisamchk. myisamchk uses no more memory than you specify with the -O options. If you are going to use myisamchk on very large files, you should first decide how much memory you want it to use. The default is to use only about 3M to perform repairs. By using larger values, you can get myisamchk to operate faster. For example, if you have more than 32M RAM, you could use options such as these (in addition to any other options you might specify):

shell> myisamchk -O sort=16M -O key=16M -O read=1M -O write=1M ...

Using -O sort=16M should probably be enough for most cases.

Be aware that myisamchk uses temporary files in TMPDIR. If TMPDIR points to a memory filesystem, you may easily get out of memory errors. If this happens, set TMPDIR to point at some directory with more space and restart myisamchk.

When repairing, myisamchk will also need a lot of disk space:

  • Double the size of the record file (the original one and a copy). This space is not needed if one does a repair with --quick, as in this case only the index file will be re-created. This space is needed on the same disk as the original record file!

  • Space for the new index file that replaces the old one. The old index file is truncated at start, so one usually ignore this space. This space is needed on the same disk as the original index file!

  • When using --recover or --sort-recover (but not when using --safe-recover), you will need space for a sort buffer for: (largest_key + row_pointer_length)*number_of_rows * 2. You can check the length of the keys and the row_pointer_length with myisamchk -dv table. This space is allocated on the temporary disk (specified by TMPDIR or --tmpdir=#).

If you have a problem with disk space during repair, you can try to use --safe-recover instead of --recover.

Using myisamchk for Crash Recovery

If you run mysqld with --skip-external-locking (which is the default on some systems, like Linux), you can't reliably use myisamchk to check a table when mysqld is using the same table. If you can be sure that no one is accessing the tables through mysqld while you run myisamchk, you only have to do mysqladmin flush-tables before you start checking the tables. If you can't guarantee the above, then you must take down mysqld while you check the tables. If you run myisamchk while mysqld is updating the tables, you may get a warning that a table is corrupt even if it isn't.

If you are not using --skip-external-locking, you can use myisamchk to check tables at any time. While you do this, all clients that try to update the table will wait until myisamchk is ready before continuing.

If you use myisamchk to repair or optimize tables, you must always ensure that the mysqld server is not using the table (this also applies if you are using --skip-external-locking). If you don't take down mysqld you should at least do a mysqladmin flush-tables before you run myisamchk. Your tables may be corrupted if the server and myisamchk access the tables simultaneously.

This section describes how to check for and deal with data corruption in MySQL databases. If your tables get corrupted frequently you should try to find the reason for this! See Crashing.

The MyISAM table section contains reason for why a table could be corrupted. See MyISAM table problems.

When performing crash recovery, it is important to understand that each table tbl_name in a database corresponds to three files in the database directory:

FilePurpose
tbl_name.frmTable definition (form) file
tbl_name.MYDDatafile
tbl_name.MYIIndex file

Each of these three file types is subject to corruption in various ways, but problems occur most often in datafiles and index files.

myisamchk works by creating a copy of the .MYD (data) file row by row. It ends the repair stage by removing the old .MYD file and renaming the new file to the original file name. If you use --quick, myisamchk does not create a temporary .MYD file, but instead assumes that the .MYD file is correct and only generates a new index file without touching the .MYD file. This is safe, because myisamchk automatically detects if the .MYD file is corrupt and aborts the repair in this case. You can also give two --quick options to myisamchk. In this case, myisamchk does not abort on some errors (like duplicate key) but instead tries to resolve them by modifying the .MYD file. Normally the use of two --quick options is useful only if you have too little free disk space to perform a normal repair. In this case you should at least make a backup before running myisamchk.

How to Repair Tables

In the following section we only talk about using myisamchk on MyISAM tables (extensions .MYI and .MYD). If you are using ISAM tables (extensions .ISM and .ISD), you should use isamchk instead.

Starting with MySQL Version 3.23.14, you can repair MyISAM tables with the REPAIR TABLE command. See REPAIR TABLE.

The symptoms of a corrupted table include queries that abort unexpectedly and observable errors such as these:

  • tbl_name.frm is locked against change

  • Can't find file tbl_name.MYI (Errcode: ###)

  • Unexpected end of file

  • Record file is crashed

  • Got error ### from table handler

    To get more information about the error you can run perror ###. Here is the most common errors that indicates a problem with the table:

      shell> perror 126 127 132 134 135 136 141 144 145
      126 = Index file is crashed / Wrong file format
      127 = Record-file is crashed
      132 = Old database file
      134 = Record was already deleted (or record file crashed)
      135 = No more room in record file
      136 = No more room in index file
      141 = Duplicate unique key or constraint on write or update
      144 = Table is crashed and last repair failed
      145 = Table was marked as crashed and should be repaired
      

    Note that error 135 (no more room in record file), is not an error that can be fixed by a simple repair. In this case you have to do:

      ALTER TABLE table MAX_ROWS=xxx AVG_ROW_LENGTH=yyy;
      

    You can also use this technique for error 136 (no more room in index file).

In the other cases, you must repair your tables. myisamchk can usually detect and fix most problems that occur.

The repair process involves up to four stages, described here. Before you begin, you should cd to the database directory and check the permissions of the table files. Make sure they are readable by the Unix user that mysqld runs as (and to you, because you need to access the files you are checking). If it turns out you need to modify files, they must also be writable by you.

If you are using MySQL Version 3.23.16 and above, you can (and should) use the CHECK and REPAIR commands to check and repair MyISAM tables. See CHECK TABLE. See REPAIR TABLE.

The manual section about table maintenance includes the options to isamchk/myisamchk. See Table maintenance.

The following section is for the cases where the above command fails or if you want to use the extended features that isamchk/myisamchk provides.

If you are going to repair a table from the command-line, you must first take down the mysqld server. Note that when you do mysqladmin shutdown on a remote server, the mysqld server will still be alive for a while after mysqladmin returns, until all queries are stopped and all keys have been flushed to disk.

Stage 1: Checking your tables

Run myisamchk *.MYI or myisamchk -e *.MYI if you have more time. Use the -s (silent) option to suppress unnecessary information.

If the mysqld server is done you should use the --update option to tell myisamchk to mark the table as 'checked'.

You have to repair only those tables for which myisamchk announces an error. For such tables, proceed to Stage 2.

If you get weird errors when checking (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 2: Easy safe repair

Note: If you want repairing to go much faster, you should add: -O sort_buffer=# -O key_buffer=# (where # is about 1/4 of the available memory) to all isamchk/myisamchk commands.

First, try myisamchk -r -q tbl_name (-r -q means ``quick recovery mode''). This will attempt to repair the index file without touching the datafile. If the datafile contains everything that it should and the delete links point at the correct locations within the datafile, this should work, and the table is fixed. Start repairing the next table. Otherwise, use the following procedure:

  1. Make a backup of the datafile before continuing.

  2. Use myisamchk -r tbl_name (-r means ``recovery mode''). This will remove incorrect records and deleted records from the datafile and reconstruct the index file.

  3. If the preceding step fails, use myisamchk --safe-recover tbl_name. Safe recovery mode uses an old recovery method that handles a few cases that regular recovery mode doesn't (but is slower).

If you get weird errors when repairing (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 3: Difficult repair

You should only reach this stage if the first 16K block in the index file is destroyed or contains incorrect information, or if the index file is missing. In this case, it's necessary to create a new index file. Do so as follows:

  1. Move the datafile to some safe place.

  2. Use the table description file to create new (empty) data and index files:

     shell> mysql db_name
     mysql> SET AUTOCOMMIT=1;
     mysql> TRUNCATE TABLE table_name;
     mysql> quit
     

    If your SQL version doesn't have TRUNCATE TABLE, use DELETE FROM table_name instead.

  3. Copy the old datafile back onto the newly created datafile. (Don't just move the old file back onto the new file; you want to retain a copy in case something goes wrong.)

Go back to Stage 2. myisamchk -r -q should work now. (This shouldn't be an endless loop.)

As of MySQL 4.0.2 you can also use REPAIR ... USE_FRM which performs the whole procedure automatically.

Stage 4: Very difficult repair

You should reach this stage only if the description file has also crashed. That should never happen, because the description file isn't changed after the table is created:

  1. Restore the description file from a backup and go back to Stage 3. You can also restore the index file and go back to Stage 2. In the latter case, you should start with myisamchk -r.

  2. If you don't have a backup but know exactly how the table was created, create a copy of the table in another database. Remove the new datafile, then move the description and index files from the other database to your crashed database. This gives you new description and index files, but leaves the datafile alone. Go back to Stage 2 and attempt to reconstruct the index file.

Table Optimization

To coalesce fragmented records and eliminate wasted space resulting from deleting or updating records, run myisamchk in recovery mode:

shell> myisamchk -r tbl_name

You can optimize a table in the same way using the SQL OPTIMIZE TABLE statement. OPTIMIZE TABLE does a repair of the table and a key analysis, and also sorts the index tree to give faster key lookups. There is also no possibility of unwanted interaction between a utility and the server, because the server does all the work when you use OPTIMIZE TABLE. See OPTIMIZE TABLE.

myisamchk also has a number of other options you can use to improve the performance of a table:

  • -S, --sort-index

  • -R index_num, --sort-records=index_num

  • -a, --analyze

For a full description of the option. See myisamchk syntax.