Messages in this thread | | | Date | Thu, 04 Sep 2008 17:05:00 -0400 | From | Oren Laadan <> | Subject | Re: [RFC v3][PATCH 1/9] Create syscalls: sys_checkpoint, sys_restart |
| |
Serge E. Hallyn wrote: > Quoting Oren Laadan (orenl@cs.columbia.edu): >> >> Serge E. Hallyn wrote: >>> Quoting Oren Laadan (orenl@cs.columbia.edu): >>>> Create trivial sys_checkpoint and sys_restore system calls. They will >>>> enable to checkpoint and restart an entire container, to and from a >>>> checkpoint image file descriptor. >>>> >>>> The syscalls take a file descriptor (for the image file) and flags as >>>> arguments. For sys_checkpoint the first argument identifies the target >>>> container; for sys_restart it will identify the checkpoint image. >>>> >>>> Signed-off-by: Oren Laadan <orenl@cs.columbia.edu> >>>> --- >> [...] >> >>>> +/** >>>> + * sys_checkpoint - checkpoint a container >>>> + * @pid: pid of the container init(1) process >>>> + * @fd: file to which dump the checkpoint image >>>> + * @flags: checkpoint operation flags >>>> + */ >>>> +asmlinkage long sys_checkpoint(pid_t pid, int fd, unsigned long flags) >>>> +{ >>>> + pr_debug("sys_checkpoint not implemented yet\n"); >>>> + return -ENOSYS; >>>> +} >>>> +/** >>>> + * sys_restart - restart a container >>>> + * @crid: checkpoint image identifier >>> So can we compare your api to Andrey's? >>> >>> You've explained before that crid is used to tie together multiple >>> calls to checkpoint, but why do you have to specify it for restart? >>> Can't it just come from the fd? Or, the fd will be passed in >>> seek()d to the right position for the data for this task, so the crid >>> won't be available there? >> I added the 'crid' inside to support a mode of operation in which we >> would like the checkpoint data to remain in memory across multiple >> system calls. Here are example scenarios: >> >> 1) We will want to reduce down time by first buffering the checkpoint >> image in memory, then resuming the container, and only then writing >> the data back to a (the) file descriptor. >> So instead of: >> freeze -> checkpoint and write back -> unfreeze >> We want: >> freeze -> checkpoint to buffer -> unfreeze -> write back >> I envision each of these steps to be a separate invocation of a syscall. >> to the 'crid' returned by the sys_checkpoint() at the 2nd step, will be >> used to identify that data in the 4th step. (Note, that between the >> unfreeze and the write-back, another checkpoint may be already taken). >> >> 2) A task may want to take a checkpoint (e.g. of itself, or a whole >> container) and keep that checkpoint in memory; at a later time it may >> want to revert to that checkpoint. Moreover, it may keep multiple such >> checkpoints (to where it may want to return). 'crid' tells sys_restart >> which one to use. >> >> Note that this 'crid' will in fact be tied to resources that are kept >> by the kernel - e.g. references to COW pages (when we add that). >> Louis suggested to use a specialized FD instead of a numeric 'crid' >> (that is: create a anonymous inode and a struct file that represent >> that checkpoint in the kernel, and return an FD to it). This approach >> has pros and cons of 'crid' (see the archives of the containers >> mailing list). For now I kept 'crid', but I'm definitely open to change >> it to a FD. >> >> Oren. > > Oh, so the crid identifies one checkpoint inside the file - the single > file can store multiple checkpoints?
Not quite. Let me rephrase the motivation first:
There are occasions when we would like to keep the checkpoint data in the kernel for some (relatively long) time, between syscalls. By "checkpoint data" I mean references to memory contents (pages) and all the other data.
The two scenarios above are two examples: between the syscall to checkpoint and the syscall to unfreeze and then write-back the data to a file (first example), and for some time until a task may want to "go back in time" (second example, useful for ultra fast "undo" for a task).
Note that in both cases when I say "keep in kernel" I mean before it is written to a file, or to the network. Simply in memory, in some efficient manner.
Subsequent syscalls will need to refer to a specific checkpoint data that is kept in memory - e.g. to write-back to a file-descriptor, or to clean up, or to restart from it. (At any single time a specific container may have multiple checkpoints associated with it - eg. because they have not yet been written back to storage but already taken).
Once the data is written back to a file descriptor, the in-kernel data can be discarded and cleaned-up.
The main reason why I want to keep the data in the kernel and not instead copy to user space, is efficiency: most of the checkpoint data is the memory footprint; by keeping the data in the kernel, one can merely keep a COW reference instead of a whole copy of everything (save space and copy time).
So, if we have keep data in kernel between syscalls, then we must have a way to refer to it. The current implementation uses a very simple 'crid' value to do that - although, clearly, at the moment it isn't used.
I hope this explains better.
Oren.
> >>> Andrey, how will the 'ctid' in your patchset be used? It sounds >>> like it's actually going to set some integer id on the created >>> container? We actually don't have container ids (or even >>> containers) right now, so we probably don't want that in our api, >>> right? > > -serge
| |