<?xml version="1.0" encoding="utf-8"?>
<launchpad-bug id="989055">
  <date_last_updated>2012-05-24 13:35:37.714521+00:00</date_last_updated>
  <api_links>
    <bug_api_link>https://api.launchpad.net/1.0/bugs/989055</bug_api_link>
    <bug_owner_link>https://api.launchpad.net/1.0/~jparrott</bug_owner_link>
    <milestone_link>https://api.launchpad.net/1.0/maria/+milestone/5.1</milestone_link>
    <linked_branches_collection_link>https://api.launchpad.net/1.0/bugs/989055/linked_branches</linked_branches_collection_link>
    <activity_link>https://api.launchpad.net/1.0/bugs/989055/activity</activity_link>
  </api_links>
  <bug_web_link>https://bugs.launchpad.net/bugs/989055</bug_web_link>
  <owner>Jason Parrott</owner>
  <assignee>Michael Widenius</assignee>
  <milestone_title>Maria 5.1</milestone_title>
  <duplicate_link></duplicate_link>
  <duplicate_bug_id></duplicate_bug_id>
  <title>Querying myisam table metadata may corrupt the table</title>
  <status>New</status>
  <importance>High</importance>
  <created>2012-04-26 17:37:37.573774+00:00</created>
  <description>
<![CDATA[This is related to bug #925377, but confirmed on the 5.3.6 release.

Jervin R (viiin) wrote a reproducer which I have attached (myisam_crash.rb) and confirm causes the corrupt table message.  We're seeing corrupt tables on our live databases as well.

This bug appears to be present still on 5.3.6 - possibly not during ENABLE|DISABLE keys but immediately before drop. see my.cnf below and Ruby script for test case.

[mysqld]
basedir=/home/revin/Downloads/mariadb-5.3.6-Linux-x86_64/
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=revin
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

collation_server = utf8_unicode_ci
character_set_server = utf8
skip-external-locking
#fast_index_creation = false

# 10MB ramdisk - can be ignored
tmpdir = /tmpfs

key_buffer = 276M
innodb_buffer_pool_size = 276M
max_allowed_packet = 16M
thread_stack = 256K
thread_cache_size = 128
thread_concurrency = 8
max_connections = 512
table_cache = 2048
myisam-recover = BACKUP
query_cache_limit = 1M
query_cache_size = 128M
expire_logs_days = 10
max_binlog_size = 100M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid


Produces the output:
120426 11:32:39 [ERROR] mysqld: Table './test/customerXX' is marked as crashed and should be repaired]]>  </description>
  <activities>
    <activity datechanged="2012-04-26T17:37:37.573774+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[]]>      </newvalue>
      <whatchanged>bug</whatchanged>
      <person>Jason Parrott</person>
      <message>added bug</message>
    </activity>
    <activity datechanged="2012-04-26T17:37:37.573774+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[myisam_crash.rb https://bugs.launchpad.net/bugs/989055/+attachment/3105750/+files/myisam_crash.rb]]>      </newvalue>
      <whatchanged>attachment added</whatchanged>
      <person>Jason Parrott</person>
      <message></message>
    </activity>
    <activity datechanged="2012-04-26T19:53:45.152710+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[]]>      </newvalue>
      <whatchanged>bug</whatchanged>
      <person>Peter (Stig) Edwards</person>
      <message>added subscriber Peter (Stig) Edwards</message>
    </activity>
    <activity datechanged="2012-04-27T00:50:25.334147+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[5.1]]>      </newvalue>
      <whatchanged>maria: milestone</whatchanged>
      <person>Elena Stepanova</person>
      <message></message>
    </activity>
    <activity datechanged="2012-04-27T00:50:30.510649+00:00">
      <oldvalue>
<![CDATA[Undecided]]>      </oldvalue>
      <newvalue>
<![CDATA[High]]>      </newvalue>
      <whatchanged>maria: importance</whatchanged>
      <person>Elena Stepanova</person>
      <message></message>
    </activity>
    <activity datechanged="2012-04-27T00:50:38.936589+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[Michael Widenius (monty)]]>      </newvalue>
      <whatchanged>maria: assignee</whatchanged>
      <person>Elena Stepanova</person>
      <message></message>
    </activity>
  </activities>
  <comments>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/1" datecreated="2012-04-26T17:37:37.573774+00:00">
      <person>Jason Parrott</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/2" datecreated="2012-04-27T00:50:14.971031+00:00">
      <person>Elena Stepanova</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[The problem is also reproducible with the MTR test case provided in the original bug report (I will paste it again here). 
The behavior is different on 5.1/5.2 and 5.3 now. On 5.3 the assertion failure seems to have been fixed, although the corruption messageis still reproducible. On 5.1 and 5.2 debug versions the original assertion failure (Assertion `strcmp(share->unique_file_name,filename) || share->last_version' failed) is still observed, and there is no commit for bug #925377 in the tree history, apparently it was only pushed into 5.3.

# MTR test case:

--connect(con1,localhost,root,,)
--perl
open( DATA, ">bug925377.txt" )
  || die "Couldn't open file bug925377.txt for writing: $!";
foreach my $i ( 1..100000 )
{
  print DATA "$i,line number $i\n";
}
close( DATA );
EOF
--disable_warnings
DROP TABLE IF EXISTS bug925377;
--enable_warnings
CREATE TABLE bug925377 (
  id INT PRIMARY KEY,
  a VARCHAR(100),
  INDEX(a)
) ENGINE=MyISAM;
ALTER TABLE bug925377 DISABLE KEYS;
LOAD DATA LOCAL INFILE 'bug925377.txt'
  INTO TABLE bug925377
  FIELDS TERMINATED BY ',';
--send
  ALTER TABLE bug925377 ENABLE KEYS;

--connection default
--let $wait_timeout=10
--let $show_statement= SHOW PROCESSLIST
--let $field= State
--let $condition= = 'Repair by sorting'
--source include/wait_show_condition.inc

SHOW TABLE STATUS LIKE 'bug925377';

--connection con1
--reap
DROP TABLE bug925377;

# End of MTR test case]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/3" datecreated="2012-05-15T15:46:15.717351+00:00">
      <person>Peter (Stig) Edwards</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[I created a bash version of the ruby reproducer, the table has fewer fields.
It reproduces the table marked as crashed error reliably (for me), using MariaDB 5.3.6, I will try it with the latest 5.3 next

#!/bin/sh

HOST=localhost
PORT=3306
USER=root
PASSWORD=
DATABASE=test
MYSQL_CLIENT=/usr/bin/mysql

function run_cmd {
  $MYSQL_CLIENT --host=$HOST --user=$USER --password=$PASSWORD --database=$DATABASE --exec="$1" > /dev/null
}

function dbs_size {
  for i in {1..200}
  do
    run_cmd "SELECT table_schema , data_length , index_length FROM information_schema.TABLES WHERE table_schema='$DATABASE'"
  done
}

function table_run {
    IDX="$1"
  run_cmd "CREATE TABLE table_$IDX (\`id\` int,  PRIMARY KEY (\`id\`)) ENGINE=MyISAM CHARSET=latin1"
  run_cmd "alter table table_$IDX disable keys"
  run_cmd "alter table table_$IDX enable keys"
  run_cmd "drop table table_$IDX"
}

function many_table_run {
  for i in {1..20}
  do
    table_run $i &
  done
}

dbs_size &
many_table_run

# produces in mysqld error log:
# 120515 11:31:23 [ERROR] mysqld: Table './test/table_5' is marked as crashed and should be repaired
# 120515 11:31:23 [ERROR] mysqld: Table 'table_5' is marked as crashed and should be repaired]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/4" datecreated="2012-05-15T16:42:52.080270+00:00">
      <person>Peter (Stig) Edwards</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[The bash reproducer in the comment above reproduces the table marked as crashed when using:
  http://terrier.askmonty.org/archive/pack/5.3/build-2163/kvm-bintar-hardy-amd64/mariadb-5.3.7-Linux-x86_64.tar.gz
from
  http://buildbot.askmonty.org/buildbot/builders/kvm-bintar-hardy-amd64/builds/1634
which is revision 3523 of ~maria-captains/maria/5.3]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/5" datecreated="2012-05-23T16:58:51.999526+00:00">
      <person>Peter (Stig) Edwards</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[I created a MTR version of the ruby and bash reproducers.  I suspect it could be simplified even more:

# MTR test case:

# https://bugs.launchpad.net/maria/+bug/989055

--disable_warnings
let $h= 200;
while ($h)
{
  eval DROP TABLE IF EXISTS t_$h;
  dec $h;
}
--enable_warnings

--connect(con1,localhost,root,,)
--connect(con2,localhost,root,,)

let $i= 200;
while ($i)
{
  eval CREATE TABLE t_$i ( id INT PRIMARY KEY, a VARCHAR(100), INDEX(a) ) ENGINE=MyISAM;
  dec $i;
}

let $j= 200;
while ($j)
{
  connection con1;
    --send
      eval ALTER TABLE t_$j DISABLE KEYS;
  connection con2;
    --send
      eval SHOW TABLE STATUS LIKE 't_$j';
  connection con1;
    reap;
    --send
      eval ALTER TABLE t_$j ENABLE KEYS;
  connection con1;
    reap;
  connection con2;
    reap;
  dec $j;
}
--connection default
disconnect con1;
disconnect con2;

let $k= 200;
while ($k)
{
  eval SHOW TABLE STATUS LIKE 't_$k';
  dec $k;
}

# Fails because of warnings found in the error log:
# YYMMDD HH:MM:SS [ERROR] mysqld: Table './test/t_XX' is marked as crashed and should be repaired

# End of MTR test case]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/6" datecreated="2012-05-23T17:14:58.479822+00:00">
      <person>Elena Stepanova</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[Hi Peter, 

There was an MTR test case in comment #2, does it not work for you?]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/989055/comments/7" datecreated="2012-05-24T13:35:37.362353+00:00">
      <person>Peter (Stig) Edwards</person>
      <subject>
<![CDATA[Re: Querying myisam table metadata may corrupt the table]]>      </subject>
      <content>
<![CDATA[Oops, sorry, in place of the "SHOW TABLE.." I intended to use "SELECT table_schema , data_length , index_length...", so:

< eval SHOW TABLE STATUS LIKE 't_$j';
> SELECT table_schema , data_length , index_length FROM information_schema.TABLES WHERE table_schema='test';

This fails because of the the table mark as crashed error in the mysqld err log.

The MTR test case in comment #2 works for me, in that it reproduces the problem of table marked as crashed.

Even though I suspect the bug may be the same, I wanted to try and produce a MTR test case for the 2nd reproducer
as it didn't involve "LOAD DATA ..." (or even writing any data to the tables), in the #2 comment MTR the query
running while keys are disabled/enabled is SHOW TABLE STATUS LIKE 'bug925377' while in the 2nd reproducer the query
is less specific (I imagine it ends up "opening for status").

I have not run it with a debug binary, if it is the same bug then please disregard it, thanks.]]>      </content>
    </comment>
  </comments>
  <messages>
    <message created="2012-04-26 17:37:37.573774+00:00" owner="Jason Parrott">
<![CDATA[]]>      <attachment link="https://bugs.launchpad.net/bugs/989055/+attachment/3105750" type="Unspecified">
        <title>myisam_crash.rb</title>
        <file>LPexportBug989055_myisam_crash.rb</file>
      </attachment>
    </message>
  </messages>
</launchpad-bug>
