[lkml]   [1999]   [May]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: Bad apache perfomance wtih linux SMP
Juergen Schmidt <> writes:
> Do you have any ideas, what's happening there?
> Or even better, how to fix this?

One culprit is most likely that the data copy for TCP sending runs completely
serialized. This can be fixed by doing replacing the

skb->csum = csum_and_copy_from_user(from,
skb_put(skb, copy), copy, 0, &err);

in tcp.c:tcp_do_sendmsg with

skb->csum = csum_and_copy_from_user(from,
skb_put(skb, copy), copy, 0, &err);

The patch does not violate any locking requirements in the kernel, because
the kerne lock could have been dropped at any time anyways when the copy_from_user
slept to swap a page in.
(I'm not sure if running a published benchmark with such a patch is fair though.
On the other hand Microsoft did some many hidden changes in their service packs
that probably everything is allowed ;)

Another problem is that Linux 2.2 per default uses only 1GB of memory. This can
be tuned by changing the PAGE_OFFSET constant in include/asm/page.h and
arch/i386/ from 0xc0000000 to 0x80000000 or so and recompiling
(the tradeoff is that that it limits the per process virtual memory to ~1.8GB,
but increases the overall physical memory that can be mapped).

> It seems to me, that this might exactly be the factor 4 those mindcraft
> people have measured. Do you think, this is possible?
> bye, juergen
> BTW: In another test with cgi-scripts I set NR_OPEN in the linux kernel
> from 1024 to 2048 (in include/linux/limits.h, fs.h and posix_types.h)
> and recompiled apache.
> So I got rid of the open() errors that occured under heavy load. But
> therefore I get only about half the rps with <=128 processes. Did I
> forget something?

Probably increasing the global file table size.


echo 32768 > /proc/sys/fs/file-max
echo 65536 > /proc/sys/fs/inode-max

Overall it should be clear that the current Linux kernel doesn't scale
to 4CPUs for system load (user load is fine). I blame the Linux vendors
for advertising it, although it is not true.

If you're interested I can send you a profiling patch that shows how much
of the system CPU time is spent in locks. Another easy way is to boot
with profile=2 and to run /usr/sbin/readprofile to see where the time is spent.

Work to fix all these problems is underway.


This is like TV. I don't like TV.

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
Please read the FAQ at

 \ /
  Last update: 2005-03-22 13:51    [W:0.052 / U:3.788 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site