Messages in this thread | | | Subject | Re: Bad apache perfomance wtih linux SMP | From | Andi Kleen <> | Date | 18 May 1999 15:02:08 +0200 |
| |
Juergen Schmidt <ju@ct.heise.de> writes: > Do you have any ideas, what's happening there? > Or even better, how to fix this?
One culprit is most likely that the data copy for TCP sending runs completely serialized. This can be fixed by doing replacing the
skb->csum = csum_and_copy_from_user(from, skb_put(skb, copy), copy, 0, &err);
in tcp.c:tcp_do_sendmsg with
unlock_kernel(); skb->csum = csum_and_copy_from_user(from, skb_put(skb, copy), copy, 0, &err); lock_kernel();
The patch does not violate any locking requirements in the kernel, because the kerne lock could have been dropped at any time anyways when the copy_from_user slept to swap a page in. (I'm not sure if running a published benchmark with such a patch is fair though. On the other hand Microsoft did some many hidden changes in their service packs that probably everything is allowed ;)
Another problem is that Linux 2.2 per default uses only 1GB of memory. This can be tuned by changing the PAGE_OFFSET constant in include/asm/page.h and arch/i386/vmlinux.lds from 0xc0000000 to 0x80000000 or so and recompiling (the tradeoff is that that it limits the per process virtual memory to ~1.8GB, but increases the overall physical memory that can be mapped).
> > It seems to me, that this might exactly be the factor 4 those mindcraft > people have measured. Do you think, this is possible? > > bye, juergen > > BTW: In another test with cgi-scripts I set NR_OPEN in the linux kernel > from 1024 to 2048 (in include/linux/limits.h, fs.h and posix_types.h) > and recompiled apache. > So I got rid of the open() errors that occured under heavy load. But > therefore I get only about half the rps with <=128 processes. Did I > forget something?
Probably increasing the global file table size.
Try: echo 32768 > /proc/sys/fs/file-max echo 65536 > /proc/sys/fs/inode-max
Overall it should be clear that the current Linux kernel doesn't scale to 4CPUs for system load (user load is fine). I blame the Linux vendors for advertising it, although it is not true.
If you're interested I can send you a profiling patch that shows how much of the system CPU time is spent in locks. Another easy way is to boot with profile=2 and to run /usr/sbin/readprofile to see where the time is spent.
Work to fix all these problems is underway.
-Andi
-- This is like TV. I don't like TV.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |