[lkml]   [2004]   [May]   [25]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: [ANNOUNCEMENT PATCH COW] proof of concept impementation of cowlinks
    On Tuesday 25 May 2004 17:08, Pavel Machek wrote:

    > > Doesn't asynchronous sendfile has the little problem your process can
    > > exit before the sendfile is complete?
    > Hmm, it has...
    > > I'm not sure how much of a help it really is, since fork() isn't brain
    > > surgery if you want it to be asynchronous, and the lifetime rules are
    > > really explicit then. (With a ps that does thread grouping, this isn't
    > > too bad from a clutter standpoint, even. And you automatically get a
    > > SIGCHLD when the sendfile is complete, too...)
    > Right.
    > > Of course if the syscall can make the sendfile outlive the process that
    > > fired it off, then by all means it sounds good. I dunno how much extra
    > > work that is for the kernel, though.
    > Well, it would be "interesting" to stop that sendfile then. You could
    > not kill it etc.

    Well, logically what you're doing is redirecting an existing filehandle so it
    points to something else. Instead of reading from this pipe, you're now
    reading from this file or from this network socket, or this other process's
    stdout. So any intermediate processes going away is theoretically okay as
    long as the anchors at each end remain (process/filesystem/network connection
    generating the data, process/filesystem/network connection receiving the

    The easy way to make the semantics work out right is that such an asynchronous
    sendfile would effectively close the file in question from the point of view
    of the process that did the sendfile. It would pretty much have to be part
    of the semantics of any asynchronous sendfile call: welding together the two
    filehandles would behave like a single direction shutdown(2) as far as the
    process that called sendfile is concerned. (That way, if you do an async
    sendfile in each direction, the filehandle is closed both ways, but you don't
    HAVE to if you don't want to. You can feed data to a child process from a
    script file or something, and just deal with the responses coming back.)

    This would mean that in theory the process that did the sendfile could go away
    without too much ambiguity about what should happen. (The bits that are
    already closed from the process's point of view are unaffected by the process
    exiting.) I dunno what's needed to clean that up in the kernel.

    I also don't know if it's a good idea, because as you noticed the fire and
    forget nature of the thing means that killing it afterwards is something we
    haven't got a semantic for if the thing at the other end is NOT a pipe to a
    processes. (We kill processes. We don't have a kill for network connections
    or for files in the filesystem that are no longer associated with any
    process. This is theoretically existing problem, by the way. Check out
    SO_LINGER in man 7 socket...)

    There are a number of interesting "no process involved" cases, actually. Do
    an asynchronous sendfile from one file to another and the disk fills up; who
    do you report the error to? And if you do a sendfile from one network socket
    to another, both of which are outbound connections to the internet, then how
    do you interrupt it later if those other systems decide to keep talking
    through your system for two days?

    Possibly the rule would have to be that a filehandle must remain open in SOME
    process context somewhere, or it gets closed (even if it was in the middle of
    an asynchronous sendfile). That would make sense, really; that's how we
    handle incoming data from a network connection when the process in question
    exits. We don't care that the datastream had data in transit, coming from
    the network into the receive buffer. It go bye-bye when the last process
    that had it open closes it, and the in-flight data is discarded.

    That sounds like a workable semantic to me. Opinions?

    It might even be possible for other I/O on the filehandle to block until the
    async sendfile finishes, but that's getting a bit fancy in the absence of
    known real world use cases asking for that feature. It would let you add in
    something like SIGIO to let you know when you need to close the filehandle
    and get any error code (like "disk full" or whatever). But I'd probably hold
    off on that until somebody actually came up with a real world use case for

    > I guess async sendfile is bad idea after all.

    Well, one thread per sendfile does smell a little bit like the java way of
    doing things, but synchronous sendfile is the 90% solution. I can always
    make my own wrapper to do the fork() and an atexit() hook to detach and
    daemonify if I really care. The kernel doesn't really have to do this for
    me. And since 2.6 already has some infrastructure to group threads so ps
    doesn't look so terrible...

    My opinion is that a synchronous sendfile that works in a generic context
    would be an improvement over what's there now, and something I actually have
    a couple existing uses for. Cleaning it up to work async can remain a to-do
    item for later (including working out under what circumstances, if any, it
    might be a good idea).

    Low hanging fruit, and all that.

    > Pavel


    -- Linux Expo and Science Fiction Convention
    October 8-10, 2004 in Austin Texas. (I'm the con chair.)

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    More majordomo info at
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 14:03    [W:0.028 / U:0.048 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site