[lkml]   [1998]   [Feb]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: New Feature Idea: Compress swap file
    >Remember y'all the kernel swaps PAGES. On Intel Machines, a PAGE is 8192

    On Intel it's 4096, it's 8192 on Alpha. Architectures such as M68K and PPC
    allow you to choose the page size from a range.

    >bytes. Suppose it got compressed to 14 bytes using RLE. What do you do with
    >it? --er where do you put it until you have accumulated enough compressed
    >pages to make a Disc write efficient? Then, suppose you need to get back a

    You have 1 or 2 pages of buffer to accumulate compressed pages before
    writing them to disk.

    >compressed page that's been concatenated into others to make an efficient
    >write. There's a LOT of overhead extracting that 14 bytes from an accumulated
    >buffer. Now, you've got a hole..

    I think that the best thing to do is not worry about filling holes. If you
    shuffle data around to fill holes then you're unlikely to end up saving disk

    >Digital spent a lot of time working on modified-page writers, and other
    >paging/swapping schemes while trying to optimize VMS. They had small pages,
    >512 bytes. The fastest scheme remained, if you have a dirtied page you have
    >to free, write it, and write it now -- but don't write a page unless you
    >absolutely, positively have to. This means stealing unwritten pages from
    >others if necessary.

    At that time CPUs were 3 orders of magnitude slower than they are now and
    hard drives were only 2 orders of magnitude slower. The change in the
    relative speeds of CPUs and mass storage changes the cost/benefit trade-offs.
    Also the size and use of programs has changed dramatically in recent decades,
    this IMHO makes the old research into swapping techniques outdated and
    therefore makes it worth experimenting with new things.

    >RLE is quite efficient for sparse data, blocks of 0s get compressed to
    >char(token), word(how-much), char(what kind). Therefore a long will
    >substitute for 64k of such sparse data. The problem remains with what-to-do
    >with it after you've compressed it!

    Put combine it with a few other compressed pages and store it in the same
    amount of disk space as a single uncompressed page.

    >FYI, in the days of CP/M, 43k, and 3.3 MHz clocks, I made the first
    >compressed file utilities so one could download a whole "user-area", i.e.,
    >"directory" at 300 baud. Anybody remember FILENAME.LBR and FILENAME.LQR
    >(Library file, Compressed library file)?

    Yes, I remember it well. However I doubt that such programs would have
    been developed if 33K6 modems were available then. Todays modems can transfer
    data fast enough that a CP/M user wouldn't save any time by compressing the

    >This was quite a few years before ".ARC" and ".ZIP" on PCs.

    Yes. I also remember that .ARC offered noticably better compression than
    anything on CP/M machines, mainly due to the technique of having several
    compression algorithms and trying each in turn to find the one that's best for
    each file. Early versions of PKZIP did the same thing. These weren't options
    for CP/M systems because of limits on program size, CPU speed, and RAM. The
    extra capabilities of more powerful PCs (16bit CPU and 256K or more of
    directly addressable RAM) made approaches feasible which had been found to be
    non-viable on CP/M machines. We may find this to be an analogy to compressed
    swap for Linux.

    Russell Coker

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to

     \ /
      Last update: 2005-03-22 13:41    [W:0.032 / U:52.804 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site