Messages in this thread | | | Date | Fri, 31 Mar 2006 20:18:27 +0200 | From | Jens Axboe <> | Subject | Re: [PATCH] splice support #2 |
| |
On Fri, Mar 31 2006, Ingo Molnar wrote: > > * Jens Axboe <axboe@suse.de> wrote: > > > > > with pipe-based buffering this approach has still the very same problems > > > > that sendfile() has with packet boundaries, because it's not enough to > > > > have "large enough" buffering (like a pipe has), the pipe also has to be > > > > drained, and the networking layer has to know the precise boundary of > > > > data. > > > > > > > > the right solution to the packet boundary problem is to pass in a proper > > > > "does userspace expect more data right now" flag, or to let userspace > > > > 'flush' the socket independently - which is independent of the > > > > pipe-in-slice issue. This solution already exists: the MSG_MORE flag. > > > > > > We can add a SPLICE_F_MORE flag for this, right now splice doesn't set > > > the MSG_MORE flag for the end of the pipe. > > > > Ala > > > #define SPLICE_F_MOVE (0x01) /* move pages instead of copying */ > > +#define SPLICE_F_MORE (0x02) /* expect more data */ > > ok, nice - something like this should work. The direction of the flag is > a philosophical question i guess: i believe in Linux we prefer to > default to "buffering enabled", i.e. the default flag should be "expect > more data". So maybe it would be better to pass in PLICE_F_END, to > indicate end-of-data. [it doesnt mean 'permanent end', so all the files > still remain open: this could be something like a HTTP 1.1 pipelined > request.]
Hmm I do prefer a MORE as to an END, as it will always give predictable results. The MORE is a performance hit, where as a missing END wont necessarily be detected until you start looking at latencies.
> furthermore, the internal implementation should also get smarter and do > a flush-socket if it would e.g. block on a pagecache page. [we often > prefer a partial packet in such cases instead of having a half-built > packet hang around.]
That's a really good idea! Wont be too much trouble to add, probably the easiest is to add a 'wait' variable to the ->map() function or something along those lines.
> btw., that 'data boundary' detail is likely lost with the pipe > intermediary solution: there is no direct connection between 'input > file' and 'output socket', so a 'flush now' event doesnt get propagated > in a natural way. (unless we extend pipes with 'data boundary' markers, > or force their flushing, which looks a whole lot of complexity for such > a simple thing.)
Probably not.
-- Jens Axboe
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |