[lkml]   [2003]   [Dec]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
Successfully tested on a Thinkpad T21 and 32GB NUMA-Q. Any feedback
regarding performance would be very helpful. Desktop users should
notice top(1) is faster, kernel hackers that kernel compiles are faster,
and highmem users should see much less per-process lowmem overhead
(but nothing truly radical like pgcl or 4/4).

This release features 4KB stack fixes for task_pt_regs(), and is vastly
cleaned up compared to the test8 release. It also features the wchan
and major/minor fault accounting fixes. I still need to fold the
anobjrmap fixes into the original patches and find some alternative way
for smbfs and ncpfs to do whatever d_validate() was doing for them (they
won't compile until I do; for now, they depend on CONFIG_BROKEN). I
also need to find a coherent way to incorporate the cleanups for pte
caching suggested by akpm without bloating the pte cache on small boxen.

It's also worth noting that the patches "originally due" to someone
were just plucked off of the list by me (in some cases, they just first
implemented of the idea), so be sure to ascribe all the credit to
them, and all the bugs to me. I wasn't sure of a complete (or even
partial) list of those to credit for top-down vma allocation, hence
"various", though it was jejb who described the glibc workaround.

Available from:

336 files changed, 4668 insertions(+), 3054 deletions(-)

-- wli

1: highpmd
-- originally due to aa as part of pte-highmem
-- from scratch implementation based on 2.6 highpte
Shove middle level pagetables into highmem PAE process
scalability, for a lowmem savings of 20KB/process. This
is a large fraction of the per-process lowmem overhead.

2: O(1) buffered_rmqueue()
Make zone->lock hold time independent of batch counts
with a revised page allocator algorithm.

3: i386 pte caching
Highpte-compatible implementation of leaf pagetable
node caching. Should eliminate pte_alloc_one() from
profiles as well as conserve cache.

4: O(lg(n)) proc_pid_readdir()/proc_task_readdir()
Make /proc/ task enumeration scalable by organizing
lists as rbtrees for O(lg(n)) list seeks. To see how
effective this is, try doing the following:
(a) start up top(1)
(b) for n in `seq 1 1024`; do sleep 60; done
... and watch -wli whale on mainline with vastly
reduced cpu consumption.

5: O(1) proc_pid_statm()
-- originally due to bcrl
proc_pid_statm() involves a vma list walk for every
process on the system. This patch keeps count of the
statistics instead for extreme process scalability:
as they're integers, no per-process locks are required.

6: kmap_atomic() microoptimizations
Trivial microoptimization that have been in RHAS for years.

7: compile-time mapping->page_lock type selection
Use rwlocks for mapping->page_lock on larger systems for
greater concurrency. Nice improvement for SDET.

8: 4KB i386 stacks
-- originally due to bcrl
Even more PAE process scalability: cut per-process lowmem
overhead from kernel stacks in half.

9: objrmap
-- originally due to dmc/mbligh
Teach the VM about how to use accounting done for file-
backed mmap() to avoid creating some pte_chains.

10: RCU mapping->i_shared_lock
Yes, the first VM RCU patch.
mapping->i_shared_sem was a contention point with some
catastrophic contention overheads. This goes all the way
and RCU's it.

11: anobjrmap, phase 1
-- originally due to hugh
Prepare the VM to change how some of the fields of struct
page are interpreted.

12: anobjrmap, phase 2
-- originally due to hugh
Remove pte_chains and temporarily disable swapout.

13: anobjrmap, phase 3
-- originally due to hugh
Set up new accounting structures for anonymous pages to
perform PTE shootdowns in an analogous way to file-backed.

14: anobjrmap, phase 4
-- originally due to hugh
Teach the VM how to handle remap_file_pages() and mremap()
and reinstate paging of anonymous memory.

15: anobjrmap, phase 5
-- originally due to hugh
Micro-optimize the new rmap functions.

11-15 are also known as "full object-based rmap" or "anonobjrmap"
11-15 mean pte_chains are *NEVER* created, *EVER*, except for two
rare instances:
(1) remap_file_pages()
(2) a rare (nonfatal) race during an in-flight mremap()
that causes pages to appear at two virtual
addresses simultaneously.

16: RCU anon->lock
This RCU's the new anobjrmap accounting structures
analogous to mapping->i_mmap and mapping->i_shared_lock.
VM RCU patch #2.

17: convert copy_strings() to use kmap_atomic() instead of kmap()
Kill off one more instance of the sleeping kmap() and
so rip out one more acquisition of the global kmap_lock
and potential global TLB flush.

18: increase page allocator batch counts
O(1) buffered_rmqueue() makes the algorithm scale with
large batches. This jacks up the batch sizes to exploit
that newfound scalability.

19: node-local i386 per_cpu areas
Bootmem allocation of per_cpu areas implies lowmem
allocation, which furthermore implies they're all
trapped on node 0. This patch utilizes the in-mainline
arch code hook to perform the necessary remapping so
that the per_cpu area belonging to a cpu is node-local.

20: CONFIG_MMAP_TOPDOWN, top-down vma allocation
-- various prior implementations
This compacts i386 virtualspace for reduced pagetable
space usage as well as enabling larger heaps and mmap()'s.
2.9GB mmap()'s and heaps are feasible in a 3:1 split.

21: anobjrmap fixes
Fix some bogons in the anobjrmap code.

22: increase static vfs hashtable and VM array sizes
Certain hashtables in the VM and vfs are statically
sized and worse yet microscopic. This jacks up their
sizes substantially for increased performance with
little code or space impact.

23: intermezzo 4KB stack fixes
-- due to muli
This fixes obscene stack usage by the intermezzo fs.

24: /proc/ BKL gunk plus page wait hashtable sizing adjustment
This nukes some unnecessary BKL acquisitions in /proc/
code and increases page wait hashtable size for better
waitqueue hashing scalability.

25: invalidate_inodes() speedup
-- originally due to Kirill Korotaev
This makes umount(8) less of a catastrophe, particularly
in automount setups typical with academic NFS setups.

26: fix the anobjrmap fix and adjust page_alloc.c inlining
Minor tweaks for the page allocator, and another
anobjrmap bugfix.

27: remove ->valid_addr_bitmap, kern_addr_valid(), and d_validate()
The pgdat->valid_addr_bitmap is unused everywhere. This
removes it and so likely saves a small amount of lowmem,
and nukes d_validate() as collateral damage. If you
realize what d_validate() does, you'll see why. (It tries
to "validate" some random address grabbed off the wire
or some other random place as a dcache entry.)

28: fix wchan accounting
Scheduling internals are still reported as blocking
points by get_wchan(); this uses an ELF section to mark
scheduling internals in a better way than current.

29: fix major/minor fault accounting
The major and minor fault statistics reported by
getrusage(2) are completely bogus; this patch corrects
the kernel's accounting so they are meaningful.

30: remove mm->swap_address
Remove an unused field of the process address space

31: 4KB stack fixes
Fix an oops and oops reporting.

32: patch-2.6.0-test11-bk8
Linus' current bk snapshot, featuring various
critical bugfixes for tmpfs, mmap(), and others.

Let people know this kernel is -wli.
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:59    [W:0.072 / U:7.944 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site