Messages in this thread | | | From | "Russell Coker - mailing lists account" <> | Date | Sun, 22 Feb 98 20:38:53 +1000 | Subject | Re: New Feature Idea: Compress swap file |
| |
>Remember y'all the kernel swaps PAGES. On Intel Machines, a PAGE is 8192
On Intel it's 4096, it's 8192 on Alpha. Architectures such as M68K and PPC allow you to choose the page size from a range.
>bytes. Suppose it got compressed to 14 bytes using RLE. What do you do with >it? --er where do you put it until you have accumulated enough compressed >pages to make a Disc write efficient? Then, suppose you need to get back a
You have 1 or 2 pages of buffer to accumulate compressed pages before writing them to disk.
>compressed page that's been concatenated into others to make an efficient >write. There's a LOT of overhead extracting that 14 bytes from an accumulated >buffer. Now, you've got a hole..
I think that the best thing to do is not worry about filling holes. If you shuffle data around to fill holes then you're unlikely to end up saving disk access.
>Digital spent a lot of time working on modified-page writers, and other >paging/swapping schemes while trying to optimize VMS. They had small pages, >512 bytes. The fastest scheme remained, if you have a dirtied page you have >to free, write it, and write it now -- but don't write a page unless you >absolutely, positively have to. This means stealing unwritten pages from >others if necessary.
At that time CPUs were 3 orders of magnitude slower than they are now and hard drives were only 2 orders of magnitude slower. The change in the relative speeds of CPUs and mass storage changes the cost/benefit trade-offs. Also the size and use of programs has changed dramatically in recent decades, this IMHO makes the old research into swapping techniques outdated and therefore makes it worth experimenting with new things.
>RLE is quite efficient for sparse data, blocks of 0s get compressed to >char(token), word(how-much), char(what kind). Therefore a long will >substitute for 64k of such sparse data. The problem remains with what-to-do >with it after you've compressed it!
Put combine it with a few other compressed pages and store it in the same amount of disk space as a single uncompressed page.
>FYI, in the days of CP/M, 43k, and 3.3 MHz clocks, I made the first >compressed file utilities so one could download a whole "user-area", i.e., >"directory" at 300 baud. Anybody remember FILENAME.LBR and FILENAME.LQR >(Library file, Compressed library file)?
Yes, I remember it well. However I doubt that such programs would have been developed if 33K6 modems were available then. Todays modems can transfer data fast enough that a CP/M user wouldn't save any time by compressing the data.
>This was quite a few years before ".ARC" and ".ZIP" on PCs.
Yes. I also remember that .ARC offered noticably better compression than anything on CP/M machines, mainly due to the technique of having several compression algorithms and trying each in turn to find the one that's best for each file. Early versions of PKZIP did the same thing. These weren't options for CP/M systems because of limits on program size, CPU speed, and RAM. The extra capabilities of more powerful PCs (16bit CPU and 256K or more of directly addressable RAM) made approaches feasible which had been found to be non-viable on CP/M machines. We may find this to be an analogy to compressed swap for Linux.
Russell Coker
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu
| |