Messages in this thread | | | Date | Mon, 17 Jul 2000 15:09:50 -0700 | From | "J. Robert von Behren" <> | Subject | kernel-level file serving optimizations |
| |
Greetings!
I've been playing around with some minor tweaks to the network and disk buffer layers of linux (kernel 2.3.99-2) to improve user-level file serving performance by avoiding unnecessary data copies (**). The mods I have in mind are relatively small, but I'm not entirely sure they are all safe. ;-) I'm hoping one of the kernel uberhackers on this list can help me out with a couple of questions.....
1) A number of the sk_buff manipulation routines (skb_push, skb_pull, skb_reserve, etc.) assume that the packet data pointed to by skb->data is contiguous. This is not the case when doing scatter/gather communications b/w memory and a NIC. Fortunately, it seems likely that all of these routines are only manipulating packet headers, and shouldn't ever go into the user-specified payload bytes (except when doing checksums - see Q2). Is this actually the case? Are there other problems that might crop up due to non-contiguous sk_buff data, either durring data transmission or reception?
2) For incoming packets, skb->ip_summed is used to tell the network stack that the NIC has already verified the packet's checksum. This flag seems to be ignored when sending packets (presumably b/c it is easy to do the checksum while copying the packet data, so without gather writes to the NIC it wouldn't help). It looks like the right place to check this flag for outgoing TCP packets would be in tcp_sendmsg, in tcp.c. Does anyone see any problems with this?
3) Finally, does anyone have any tips on places I should look, or techniques I should use for better understanding the flow of control through the network stack? My current thought is to set up a serial console with another machine, and use gdb to step through the network processing, but I'd be very interested to hear if people have better suggestions.
Thanks much!
-Rob von Behren
(**) Before anyone jumps on my case for this, I _have_ read the zero copy discussions from the list archives. Using the SGI kernel profiling patches, I consistently see over 50% of the cpu time spent doing copies when I run my test app - so in _this_ case, zero copy will be a win.... ;-) Additionally, I _don't_ need to look at the bytes in user level, so all of the usual page remaping tricks discussed in the research literature are completely unnecessary, as are the complications they entail.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |