lkml.org 
[lkml]   [2000]   [Feb]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: 2.4 Features
Hi Ted.

>> I don't by any means know all of the patches against the e2fs
>> file system, but of the ones I do know, only e2comp is even
>> close to being ready for inclusion in the standard kernel.
>> However, it's long past time that e2comp was included therein
>> in my opinion.

> Well, see Al Viro's comments about the porting work needed to
> get it ready for Linux 2.3.latest.

I've read them. However, considering Al's previous comments regarding
what I and several others see as reasonable tweaks to the VFAT file
system to make it simpler to use with Linux (which comments can
basically be summarised as "I don't fancy that so I'm going to rubbish
the whole idea and stick with what I fancy doing"), I tend to take
Al's comments with a very large pinch of salt.

>>>> Note that the *reason* why most of these patches haven't been
>>>> merged into the e2fsprogs is that it's not yet 100% certain
>>>> that the format filesystem is stable yet.

>> Ted: What exactly do you mean by "the format filesystem" ???

>> If you are referring to "the format of the filesystem", then as
>> far as the e2comp patch is concerned, that is essentially
>> unchanged, as the ONLY change is that the file will take up less
>> blocks than its UNCOMPRESSED size indicates. Were this not the
>> case, then it would not have been possible to produce the patch
>> in the first place.

> I'm referring to the format of the filesystem change. E2compr
> makes filesytem changes. Backwards compatible changes, yes, but
> filesystem changes nonetheless.

> Basically, I arbitrate ext2 format changes. If someone wants to
> use a space in the ext2 inode, we need to make sure that it's a
> good use of a very precious resource. For example, one of the
> ext2 extensions wanted to use the space reserved for extended
> the uid to 32 bits. I suggested that they not do that, so that
> when the 32 uid work got done, there was space for it.

Nodz.

> As another example, e2compr 0.3 used a number of extra fields in
> the inode, and so I reviewed the format as suggeted by Peter
> Moulder, and after some back and forth, we managed to keep the
> number of bits reserved to e2compr to be 11 bits in i_flags.

I don't plan on changing it at this stage, but will be looking through
it before I make any definate statements regarding that.

> If the *format* for e2compr is stable, then I'm much more
> willing to include support it into the mainline e2fsprogs
> sources.

Fair enough.

> (Of course, then I will have code correctness issues.
> Currently the e2compr e2fsprogs patches don't check the
> superblock compatibility flag before they do there different
> thing. This is bad; they should only make allowances for e2compr
> if the e2compr compatibility flag is set. That's not a problem;
> I can fix most of that when I accept the patch.

I could certainly take a look at that, but if you've already done so,
perhaps you could suggest a patch to fix that?

> I also don't want to support 0.3 users in the mainline, since it
> adds to the grot and makes it harder to do long-term
> maintenance.)

Fair enough.

> Speaking of format, though, consider carefully. Before you merge
> into the 2.3 tree, this is your last chance to drop grot that
> maybe *you* don't want to support in the long term.

> Do you really want to support all those many different
> compression algorithms, for example? It may be painful to
> existing e2compr users if you have to force some fraction of
> them to convert (although you could provide conversion
> programs), but after it hits the mainline kernel, you'll
> *really* have a hard time getting rid of some deprecated
> compression format.

I've been thinking about that myself, and I can definately see a need
to separate the actual compression process from the logistics of
storing compressed files on disk.

> The ability to add additional compression methods is good, for
> future expansion --- but do you really expect everyone to use
> low these many different types of compression algorithms?

Can I presume 's/low/all/' applies to the first word of that last
line?

> Each bit of kernel bloat (and potential maintenance headache on
> your part) should be carefully scrutinized and justified. This
> is your last chance to remove excess code bloat!

Don't worry, it will be.

>>>> If it turns out that some change is needed to more efficienctly
>>>> or robustly support ACL or Capabilities, there may not be
>>>> (almost certainly will not be) backwards compatibility with the
>>>> old format.

>> Linus made it clear some time ago that ANY time a file system
>> changed in ways that meant that it could no longer access older
>> formats, it would be given a new name. That's the reason why ext
>> became ext2 in the first place, and also why ext3 has its own
>> name. As a result, that argument is a non-starter.

I note that you ignore the claim that "ext became ext2" and
concentrate on ext3 here...

> No. Ext3 is currently fully backwards compatible with ext2.

From experience, no it isn't. However, I think what you meant to
say was that the ext3 driver is fully capable of reading ext2 file
systems, in which case you are of course right.

However, take a system with the root partition in ext3 format (as I
once managed to do) and mangle the only kernel image with ext3 in it
such that said image won't boot, then have the system lock up - then
watch the fun and games as one tries to boot the system !!! If it was
backwards compatible, that would cause no problems at all...

> Once you unmount an ext3 filesystem cleanly, the journal bit is
> cleared, and a standard ext2 filesystem code and standard ext2
> utilities will work just fine with the filesystem.

First, your use of the word "cleanly" invalidates your claim that it's
backwards compatible since the latter claim is meaningless unless it's
true with unclean unmounts.

Secondly, I had occasions where even with a clean unmount, the journal
bit was NOT cleared, so the claim isn't necessarily true anyway.

> The reason why ext3 has its own name is more about making it
> easy to have two different versions of the filesystem code in
> the tree at once; one which is in production and is stable, and
> one which is in testing.

My memory may be wrong here, but it seems to remember SCT stating that
he chose to rename to ext3 since the journalling facility was likely
to make the driver not be backwards compatible with the current code.

SCT: Am I wrong here?

> I generally use ext2dev as a module-loaded filesystem when I'm
> doing my own testing, for example.

I have to admit to not having heard of ext2dev, so I can't comment
thereon.

> What I was talking about is the backwards incompatible changes.
> For example, the change between e2compr 0.3 and e2compr 0.4.
> While that sort of change is still a possibility, IMHO it
> shouldn't go into the kernel, since we *don't* want to encourage
> widespread adoption of something that we might need to change
> later.

That's presumably the reason why there was massive file system
breakage early in the 2.3 kernel series when Linus made changes that
stopped most of the file systems from compiling?

Having said that, I'm as loathe as anybody to put something in the
kernel knowing that there will soon be violent changes of that sort of
nature, and would never do so.

> (And yes, we could put lots of backwards compatibility support
> code into the kernel, but that starts getting real ugly and it's
> a real maintenance headache).

The only 'backwards compatibility' code that I'm interested in seeing
in the kernel would be code that decompresses depreciated compression
algorithms and forcibly labels any files using those algorithms as
read-only. As I see it, that's the ONLY sensible way to deal with such
algorithms - and to prevent them taking up unnecessary kernel memory,
force them to be modules rather than hard compiled in.

>> Better still, let's get it merged into e2fs where it belongs.
>> That way, we can do precicely what Linus has said he wants: Get
>> it in wide circulation so any problems can be located and ironed
>> out.

> If you're committed to spending the time necessary to port
> e2compr to 2.3.latest, and committed to tracking down the
> resulting bugs, great.

I'll email you separately regarding this point...

> What would be a really bad thing would be if the code gets
> folded in, and folks lose interest to fix the problems.

Very true. I don't anticipate losing interest in this though, as I
have already met several cases where its use is the only practical way
to install Linux.

> P.S. I just downloaded the e2compr patches, and some comments on
> the actual e2compr code:

Thanks.

> 1) Suggestion: find the comment that's in French, and run it
> through babelfish, and the put the comments in English in the
> ext2 code. Long-term maintainability is your friend.

I have to admit that I hadn't noticed it. However, given my experience
of BabelFish in the past, I would tend to use a slightly more accurate
translation source...

> 2) The code that checks for e2compr flags and does something
> different if they are set should check to see if compression was
> enabled in the superblock, and call ext2_error() if not. That
> way inodes that are incorrectly set as using compression when
> the overall filesystem aren't can be caught.

True.

> 3) There is space to support at most 32 algorithms. Do you
> really need to chew up that space with 9 different GZIP variants
> (for gzip -1 through gzip -9) and 3 different Lempel-Ziv
> variants?

Based on what I've looked through so far, at least one of the L-Z
variants only exists in the documentation as it has already been
depreciated and withdrawn. I can also see reason to reduce the number
of valid gzip levels to at most 3, along the lines of the following
entries in the Config.in file:

choose ' GZIP compression strategy' \
"SPEED 'Favour speed over compression' \
NORMAL 'Speed equal to compression' \
SPACE 'Favour compression over speed'" NORMAL

The code would then decide to use -1 or -5 or -9 depending on which of
those three was chosen. The other levels are effectively meaningless
in this context, in my opinion.

> The gzip variants are particularly interesting since you don't
> need to know the gzip level to decompress, and it may be that a
> global system-wide gzip compression level is sufficient --- at
> which point, you only need one gzip variant instead of 9.

Whilst the level isn't needed to decompress, I'm not so sure that the
same is true when an application just appends to an existing file
without reading it. However, I will be looking into that in the near
future.

> (This is an example of a format change where now's the time to
> make such changes if you're going to make them.)

True. However, at this stage, the only part of the system I intend to
offer for inclusion is the documentation and the tweaks to the
Config.in file, and on their own, they can't actually affect anything
in the kernel.

Best wishes from Riley.


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:56    [W:0.070 / U:0.264 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site