[lkml]   [2008]   [Aug]   [12]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: checkpoint/restart ABI
    On Mon, 2008-08-11 at 16:38 -0700, Jeremy Fitzhardinge wrote:
    > This feature, as it currently stands, is essentially useless for any
    > practical purpose. Self-checkpointing a single process with no handling
    > of non-file file descriptors and no proper handling of file
    > file-descriptors is not very useful.
    > My understanding that this is basically a prototype for a more useful
    > multi-process or container-wide checkpoint facility.

    Yes, that's exactly it. We're diverging from discussing the important
    bits as it is, and I think we'd do that more and more with extra
    code. :)

    > While you could try to come up with an extensible file format that would
    > be able to handle any future extensions, the chances are you'd get it
    > wrong and need to break file format compatibility anyway.

    Amen to that. I won't speak for the rest of the whackos interested in
    this stuff, but I *KNOW* I'm not clever enough to pull it off.

    > I'm more interested in seeing a description of how you're doing to
    > handle things like:
    > * multiple processes
    > * pipes
    > * UNIX domain sockets
    > * INET sockets (both inter and intra machine)
    > * unlinked open files
    > * checkpointing file content
    > * closed files (ie, files which aren't currently open, but will be
    > soon, esp tmp files)
    > * shared memory
    > * (Peter, what have I forgotten?)
    > Having gone through this before, I don't think an all-kernel solution
    > can work except for the most simple cases.

    So, there's a lot of stuff there. The networking stuff is way out of my
    league, so I'll cc Daniel and make him answer. :)

    All of the other stuff has been done in various in-kernel
    implementations. OpenVZ, IBM's Metacluster, Zap (Oren's work at
    Columbia). Most of it *can* be done from userspace, but some of it is
    very painful. There are some good OLS papers describing most of these
    things. Zap might have had one or two academic papers written about it.
    Maybe. ;)

    Unlinked files, for instance, are actually available in /proc. You can
    freeze the app, write a helper that opens /proc/1234/fd, then copies its
    contents to a linked file (ooooh, with splice!) Anyway, if we can do it
    in userspace, we can surely do it in the kernel.

    I'm not sure what you mean by "closed files". Either the app has a fd,
    it doesn't, or it is in sys_open() somewhere. We have to get the app
    into a quiescent state before we can checkpoint, so we basically just
    say that we won't checkpoint things that are *in* the kernel.

    Is there anything specific you are thinking of that particularly worries
    you? I could write pages on the list you have there.

    > Which, come to think of it, is an important point. What are the
    > expected use-cases for this feature? Do you really mean
    > checkpoint/restart? Do you expect to be able to checkpoint a process,
    > leave it running, then "rewind" by restoring the image? Or does
    > checkpoint always atomically kill the source process(es)? Are you
    > expecting to be able to resume on another machine?


    We all want different things, and there are a lot of people interested
    in this stuff. So, I think all of what you've mentioned above are
    goals, at least long term. Some, *really* long term.

    I don't want to get into a full virtualization vs. containers debate,
    but we also want it for all the same reasons that you migrate Xen

    > Lightweight filesystem checkpointing, such as btrfs provides, would seem
    > like a powerful mechanism for handling a lot of the filesystem state
    > problems. It would have been useful when we did this...

    Yup. We were just chatting about that with some filesystem folks last
    week. But, as the OpenVZ dudes like to mention, the poor man's way of
    moving filesystem snapshots around is always rsync.

    -- Dave

     \ /
      Last update: 2008-08-12 17:01    [W:0.025 / U:206.496 seconds]
    ©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site