<?xml version="1.0" encoding="utf-8"?>
<launchpad-bug id="470590">
  <date_last_updated>2011-09-05 13:29:10.806234+00:00</date_last_updated>
  <api_links>
    <bug_api_link>https://api.launchpad.net/1.0/bugs/470590</bug_api_link>
    <bug_owner_link>https://api.launchpad.net/1.0/~arjen-lentz</bug_owner_link>
    <milestone_link></milestone_link>
    <linked_branches_collection_link>https://api.launchpad.net/1.0/bugs/470590/linked_branches</linked_branches_collection_link>
    <activity_link>https://api.launchpad.net/1.0/bugs/470590/activity</activity_link>
  </api_links>
  <bug_web_link>https://bugs.launchpad.net/bugs/470590</bug_web_link>
  <owner>Arjen Lentz</owner>
  <assignee>Kristian Nielsen</assignee>
  <milestone_title></milestone_title>
  <duplicate_link></duplicate_link>
  <duplicate_bug_id></duplicate_bug_id>
  <title>Multiple fsync()s when sync_binlog=N (&gt;0)</title>
  <status>Invalid</status>
  <importance>Wishlist</importance>
  <created>2009-11-02 11:53:36.354987+00:00</created>
  <description>
<![CDATA[Depending on the setup and the choice of speed vs integrity in case of a server crash, configs need sync_binlog=1.
This slows down the server, but actually unnecessarily so. The setting apparently causes *separate* fsync() calls in various places in the code (binlog, relay log/index, possibly the master.info file), and that's ridiculous.

Can someone (Monty?) please fix this, and make the logic such that we get only a single fsync() per commit?
If this is a logistical problem given the XtraDB/InnoDB plugin, I can cope with one extra fsync() in the main server code in addition to the one in InnoDB... that can be a good intermediate step and would already reduce the overhead and slow-down significantly.

Thanks!]]>  </description>
  <activities>
    <activity datechanged="2009-11-02T11:53:36.354987+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[]]>      </newvalue>
      <whatchanged>bug</whatchanged>
      <person>Arjen Lentz</person>
      <message>added bug</message>
    </activity>
    <activity datechanged="2009-11-02T11:54:43.195815+00:00">
      <oldvalue>
<![CDATA[Undecided]]>      </oldvalue>
      <newvalue>
<![CDATA[High]]>      </newvalue>
      <whatchanged>maria: importance</whatchanged>
      <person>Arjen Lentz</person>
      <message></message>
    </activity>
    <activity datechanged="2009-11-24T08:16:30.562308+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[performance replication]]>      </newvalue>
      <whatchanged>tags</whatchanged>
      <person>Hakan Küçükyılmaz</person>
      <message></message>
    </activity>
    <activity datechanged="2009-12-02T13:22:35.727685+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[Arjen Lentz (arjen-lentz)]]>      </newvalue>
      <whatchanged>maria: assignee</whatchanged>
      <person>Hakan Küçükyılmaz</person>
      <message></message>
    </activity>
    <activity datechanged="2009-12-02T13:22:47.516810+00:00">
      <oldvalue>
<![CDATA[New]]>      </oldvalue>
      <newvalue>
<![CDATA[Incomplete]]>      </newvalue>
      <whatchanged>maria: status</whatchanged>
      <person>Hakan Küçükyılmaz</person>
      <message></message>
    </activity>
    <activity datechanged="2009-12-03T01:55:50.615094+00:00">
      <oldvalue>
<![CDATA[Arjen Lentz (arjen-lentz)]]>      </oldvalue>
      <newvalue>
<![CDATA[]]>      </newvalue>
      <whatchanged>maria: assignee</whatchanged>
      <person>Arjen Lentz</person>
      <message></message>
    </activity>
    <activity datechanged="2009-12-08T16:08:18.733124+00:00">
      <oldvalue>
<![CDATA[High]]>      </oldvalue>
      <newvalue>
<![CDATA[Medium]]>      </newvalue>
      <whatchanged>maria: importance</whatchanged>
      <person>Hakan Küçükyılmaz</person>
      <message></message>
    </activity>
    <activity datechanged="2010-11-30T15:58:01.128281+00:00">
      <oldvalue>
<![CDATA[Medium]]>      </oldvalue>
      <newvalue>
<![CDATA[Wishlist]]>      </newvalue>
      <whatchanged>maria: importance</whatchanged>
      <person>Sergei</person>
      <message></message>
    </activity>
    <activity datechanged="2010-11-30T15:58:41.593038+00:00">
      <oldvalue>
<![CDATA[Incomplete]]>      </oldvalue>
      <newvalue>
<![CDATA[Confirmed]]>      </newvalue>
      <whatchanged>maria: status</whatchanged>
      <person>Sergei</person>
      <message></message>
    </activity>
    <activity datechanged="2011-09-05T12:45:14.054117+00:00">
      <oldvalue>
<![CDATA[]]>      </oldvalue>
      <newvalue>
<![CDATA[Kristian Nielsen (knielsen)]]>      </newvalue>
      <whatchanged>maria: assignee</whatchanged>
      <person>Philip Stoev</person>
      <message></message>
    </activity>
    <activity datechanged="2011-09-05T13:29:10.080648+00:00">
      <oldvalue>
<![CDATA[Confirmed]]>      </oldvalue>
      <newvalue>
<![CDATA[Invalid]]>      </newvalue>
      <whatchanged>maria: status</whatchanged>
      <person>Kristian Nielsen</person>
      <message></message>
    </activity>
  </activities>
  <comments>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/470590/comments/1" datecreated="2009-11-24T09:24:09.311157+00:00">
      <person>Sergey Petrunia</person>
      <subject>
<![CDATA[Re: Multiple fsync()s when sync_binlog=N (>0)]]>      </subject>
      <content>
<![CDATA[Arjen,

Could you please point at the offending fsync() calls or provide some evidence that there are multiple syncs being done for one transaction? (i.e. what has led you to suspect that there are more fsync calls done when necessary?)]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/470590/comments/2" datecreated="2009-12-03T01:55:18+00:00">
      <person>Arjen Lentz</person>
      <subject>
<![CDATA[Re: [Bug 470590] Re: Multiple fsync()s when sync_binlog=N (>0)]]>      </subject>
      <content>
<![CDATA[Hi Sergey

On 24/11/2009, at 7:24 PM, Sergey Petrunia wrote:
> Could you please point at the offending fsync() calls or provide some
> evidence that there are multiple syncs being done for one transaction?
> (i.e. what has led you to suspect that there are more fsync calls done
> when necessary?)


Hearsay from people with a clue. Hadn't browsed the code for this  
myself.
However, just doing a quick grep/less finds them within a few seconds.
There's mysys/my_sync.c that contains my_sync()

my_sync() gets called from sql/log.cc in several places, inside  
functions that do something... for instance when binlog is written, if  
sync_binlog is 1 or reaches its configured number, it's fsynced. This,  
by definition, implies that it's a separate sync from an InnoDB  
transaction or MariaDB action. That's two syncs already, that should  
in fact be one.

And the binlog index should really also be part of that, to be  
resilient to crashes. It would also be less overhead than it is now,  
provided fsync/my_sync is only called once for a commit.

Similarly, the relay log and relay index should be synced at the same  
time, and possibly master.info with it.


I hope this provides you with sufficient detail to address the problem.
Thanks

Cheers,
Arjen.
]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/470590/comments/3" datecreated="2009-12-08T16:09:39.169652+00:00">
      <person>Hakan Küçükyılmaz</person>
      <subject>
<![CDATA[Re: Multiple fsync()s when sync_binlog=N (>0)]]>      </subject>
      <content>
<![CDATA[We have to investigate the use of fsync(s)  and whether it is feasable to reduce the number of fsync() calls.]]>      </content>
    </comment>
    <comment commentlink="https://api.launchpad.net/1.0/maria/+bug/470590/comments/4" datecreated="2011-09-05T13:28:48.774853+00:00">
      <person>Kristian Nielsen</person>
      <subject>
<![CDATA[Re: Multiple fsync()s when sync_binlog=N (>0)]]>      </subject>
      <content>
<![CDATA[The issue is more complex than just saying that things are "ridiculous".

There are several worklogs that describe ideas for improving this. For example:

    http://askmonty.org/worklog/Server-RawIdeaBin/?tid=164
    http://askmonty.org/worklog/Server-RawIdeaBin/?tid=187
    http://askmonty.org/worklog/Server-Sprint/?tid=175]]>      </content>
    </comment>
  </comments>
</launchpad-bug>
