Messages in this thread | | | Date | Mon, 30 May 2011 11:02:05 -0700 (PDT) | From | david@lang ... | Subject | Re: [rfc] Ignore Fsync Calls in Laptop_Mode |
| |
On Mon, 30 May 2011, D. Jansen wrote:
> On Mon, May 30, 2011 at 3:53 AM, <david@lang.hm> wrote: >> On Sun, 29 May 2011, D. Jansen wrote: >>> On Fri, May 27, 2011 at 4:17 PM, Theodore Tso <tytso@mit.edu> wrote: >>>> On May 27, 2011, at 3:12 AM, D. Jansen wrote: >>>>> That reordering is exactly what I'm talking about. It wasn't my idea. >>>>> But if I understood it correctly, it's possible that the kernel >>>>> commits writes of an application, _to one and the same file_, in a >>>>> non-FIFO order, if the application does not fsync. And this _afaiu_ >>>>> could result in the loss not only of new data, but complete corruption >>>>> of previously existing data in laptop mode without fsync. >>>> >>>> No, you're not understanding the problem. All layers of the storage >>>> stack -- including the hard drive -- is allowed to reorder writes. So >>>> even if the kernel sends data to the disk in the exact same order that >>>> the application wrote it, it could still get written in a different >>>> order, >>>> because the hard drive itself can reorder writes. This is necessary >>>> for performance; if you didn't have this, the storage stack would be >>>> dog slow, and would consume even more power. >>>> >>>> So at least level, the only thing you can count upon is that if you want >>>> to make sure everything is flushed to stable store, you need to send >>>> an fsync() command at the application to file system level, or a barrier >>>> or flush command at the OS to hard drive level. >>> (...) >>>> Ordering doesn't matter, because nothing, including the hard drive, >>>> guarantees ordering. What does matter is that the fsync() commands >>>> act like barriers; writes before the fsync() command are guaranteed >>>> to be written to the disk, and survive a reboot, before any writes after >>>> the fsync() are processed. See? >>> >>> Ok, thanks a lot! I understand a lot better now! >>> So we can't live without the fsyncs. >>> >>> So what if we would queue the fsyncs along with the writes - we would >>> just fsync later instead of immediately, in between the writes as they >>> came in. Then by design previous data could not be corrupted, right? >>> We would do exactly the same thing, just later. >>> It'd be kind of a disk write time distortion field. >> >> the problem is that the spec for fsync says that your program stops until >> fsync finishes. If you don't do that then you will corrupt and loose data. >> >> so if you delay fsync you will have your application (or desktop manager) >> freeze until the fsync completes. > > So that would not be an option. Freezing until the end of the write > window is not what we want. > Neither is ignoring the fsync because that could corrupt data, esp. in > databases like sqlite. >> >> if what you are wanting is the ability to say 'these things must be written >> before these other things to keep them from being corrupted, but I don't >> care when they get written (or if they get lost in a crash)' then what you >> want isn't fsync, it's a barrier. > > That sounds great! > So an fsync call in laptop mode could be interpreted as a barrier > and we would be reasonably save from corrupting old existing data?
no, you cannot just change a fsync to a barrier, in some cases the data absolutly needs to be saved, not just ordered (remember the example of a mail server telling the other system that the data can be deleted after a fsync returns)
David Lang | |