Messages in this thread | | | From | Alberto Gonzalez <> | Subject | Re: Ext4 and the "30 second window of death" | Date | Wed, 1 Apr 2009 03:25:43 +0200 |
| |
On Wednesday 01 April 2009 01:22:19 Andreas T.Auer wrote: > On 01.04.2009 00:02 Alberto Gonzalez wrote: > > In fact, thinking about it, this option would be the ideal one for > > desktops and especially laptops (servers running databases are a > > different thing). What we need is that _no_ application uses fsync. The > > decision as to when the data should be written to disk should be left to > > the filesystem. And then the user can choose how often they want this to > > happen (every 5, 15, 30, 60... seconds). So if Ext4 could have a > > "nofsync" mount option that would disable fsync from applications (i.e, > > it wouldn't honor an fsync call), that would be wonderful. But then of > > course we have to make sure that if the kernel crashes (or there's a > > power-off, etc..), we will just lose the new data that hasn't been > > written to disk, but the old data will still be there. So maybe this > > could be achieved with mounting the filesystem with nofsync, nodelalloc? > > You are always thinking about the few seconds/minutes of work you gonna > lose, but there are different situations, too. > > E.g. your POP3 client receives a very important mail, saves it to disk, > uses fsync to make sure it is out and tells the server to delete it. If > you are gonna delay the fsync, you will have a long window in which the > mail can get lost instead of a minimum window. Or are there any POP3 > clients, which can synchronize the mail-polling with a spinning a disk?
Yes, I guess this is a clear example of data that needs to be written to disk straight away.
> > There are tasks that are not very important, that should not spin up the > disk and there are tasks, that might better do so. It is the preference > of the user, which tasks should or should not spin up the disk, but the > application developer has to decide globally, whether or not to use > fsync() and the filesystem can't even distinguish the tasks at all, > except that it receives fsyncs or not. > > So fine-tuning the system to the ideal disk-writing policy is really > problematic, especially given a lot of different people turning knobs: > - different filesystem developers using different methods and default > behaviors, which can be changed by distros and sys admins. > - different applications trying to use or not use fsync() and other > methods to get the best policies for any kind of fs. Or the developers > are incompetent enough to expect features from the filesystem which are > not always given, whether trained by ext3 data=ordered or trained by > reiserfs or just bare of any better fs knowledge. > - different users having different preferences on what data is how > important, but usually they can not change the fsync-policy of the > applications.
Yes, I agree with all the above. There's no magic recipe for any filesystem, and honestly, I've never had problems with reiserfs in the past or ext3 later on. I don't know why I got scared with all this "ext4 will give you zero- length files on every crash unless all applications start to fsync like crazy and kill your hard drive in a year time" thing. Filesystem developers must have a bit of bit of knowledge about how this works to not do something too stupid.
> Andreas
Alberto.
| |