Messages in this thread Patch in this message | | | From | Jeffrey Brian Arnold <> | Subject | [RFC PATCH 6/9] Ksplice documentation | Date | Sat, 13 Sep 2008 00:49:54 -0400 |
| |
This document assumes familiarity with the Ksplice design and describes notable implementation details and the interface between the Ksplice kernel component and the Ksplice user space component.
Signed-off-by: Jeffrey Brian Arnold <jbarnold@mit.edu> Signed-off-by: Tim Abbott <tabbott@mit.edu> --- Documentation/ksplice.txt | 251 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 251 insertions(+), 0 deletions(-) create mode 100644 Documentation/ksplice.txt
diff --git a/Documentation/ksplice.txt b/Documentation/ksplice.txt new file mode 100644 index 0000000..5ec9173 --- /dev/null +++ b/Documentation/ksplice.txt @@ -0,0 +1,251 @@ +Ksplice +------- + +CONTENTS: + +1. Concepts: updates, packs, helper modules, primary modules +2. What changes can Ksplice handle? +3. Dependency model +4. Locking model +5. altinstructions, smplocks, and parainstructions +6. sysfs interface +7. debugfs interface + +0. Design Description +--------------------- + +For a description of the Ksplice design, please see the Ksplice technical +overview document: <http://web.mit.edu/ksplice/doc/ksplice.pdf>. For usage +examples and the Ksplice man pages, please see <http://web.mit.edu/ksplice/>. + +The document below assumes familiarity with the Ksplice design and describes +notable implementation details and the interface between the Ksplice kernel +component and the Ksplice user space component. + +1. Updates, packs, helper modules, primary modules +-------------------------------------------------- + +A Ksplice update (struct update) contains one or more Ksplice packs, one for +each "target" kernel module that should be changed by the update. Ksplice packs +are grouped together into a Ksplice update in order to allow multiple +compilation units to be changed atomically. + +The contents of a Ksplice pack are documented via kernel-doc in +include/linux/ksplice.h. To construct a new Ksplice update to be performed +atomically, one needs to: + 1. Populate the fields of one or more ksplice_pack structures. + 2. Call the Ksplice function init_ksplice_pack() on each pack to register + the packs with the Ksplice kernel component. When init_ksplice_pack() + is called on a pack, that pack will be associated with the other packs + that share the same Ksplice identifier (KID) field. + 3. After all of the packs intended for a particular Ksplice update have + been loaded, that update can be applied via the sysfs interface + (described in Section 7 below). + +In order to save memory, each Ksplice pack has a "helper" module and a "primary" +module associated with it. + +The pack's "helper" module contains materials needed only for preparing for the +update. Specifically, the helper module contains a copy of the pre-patch +version of each of the compilation units changed by the Ksplice pack. The +helper module can be unloaded after the update has been applied. + +The pack's "primary" module contains the new sections to be inserted by the +update; it needs to remain loaded for as long as the update is applied. + +Here's an example: + +Let's say that the Ksplice user space component wants to update the core kernel +and the isdn module. The user space component will select a KID for this update +(let's say 123abc) and generate four modules: + +ksplice_123abc_vmlinux (the "primary" module for the vmlinux pack) +ksplice_123abc_vmlinux_helper (the "helper" module for the vmlinux pack) +ksplice_123abc_isdn (the "primary" module for the vmlinux pack) +ksplice_123abc_isdn_helper (the "helper" module for the vmlinux pack) + +Once both of the vmlinux modules have been loaded, one of the modules calls +init_ksplice_pack on a pack corresponding to the desired vmlinux changes. + +Similarly, once both of the isdn modules have been loaded, one of the modules +calls init_ksplice_pack on a pack corresponding to the desired isdn changes. + +Once all modules are loaded (in this example, four modules), the update can be +applied atomically using the Ksplice sysfs interface. Once the update has been +applied, the helper modules can be unloaded safely to save memory. + +2. What changes can Ksplice handle? +----------------------------------- + +The Ksplice user space component takes a source code patch and uses it to +construct appropriate Ksplice packs for an update. Ksplice can handle source +code patches that add new functions, modify the text or arguments of existing +functions, delete functions, move functions between compilation units, change +functions from local to global (or vice versa), add exported symbols, rename +exported symbols, delete exported symbols. Ksplice can handle patches that +modify either C code or assembly code. + +As described in the Ksplice technical overview document, Ksplice cannot handle +semantic changes to kernel data structures. Some other limitations also apply: + +Ksplice does not support changes to __init functions that been unloaded from +kernel memory. Ksplice also does not support changes to functions in .exit.text +sections since Ksplice currently requires that all Ksplice updates affecting a +module be reversed before that module can be unloaded. + +Exported symbols: + +Ksplice can handle arbitrary changes to exported symbols in the source code +patch. + +Ksplice deletes exported symbols by looking up the relevant struct kernel_symbol +in the kernel's exported symbol table and replacing the name field with a +pointer to a string that begins with DISABLED. + +Ksplice adds new exported symbols through the same mechanism; the relevant +primary module will have a ksymtab entry containing a symbol with a name +beginning with DISABLED, and Ksplice will replace that with the name of the +symbol to be exported when the update is atomically applied. + +Because the struct kernel_symbol for a newly exported symbol is contained in the +Ksplice primary module, if a module using one of the newly exported symbols is +loaded, that module will correctly depend on the Ksplice primary module that +exported the symbol. + +Read-only data: + +The current Ksplice user space implementation does not support changes to global +read-only data structures (changes to read-only data structures that are local +to a compilation unit are fine). The user space implementation currently +handles changes to read-only data structures by patching all functions that +reference those data structures. + +3. Dependency model +------------------- + +Because Ksplice resolves symbols used in the post code using Ksplice +relocations, Ksplice must enforce additional dependencies. Ksplice uses the +use_module function to directly add dependencies on all the modules that the +post code references. + +4. Locking model +---------------- + +From a locking perspective, Ksplice treats applying or removing a Ksplice update +as analogous to loading or unloading a new version of the kernel modules patched +by the update. Ksplice uses module_mutex to protect against a variety of race +conditions related to modules being loaded or unloaded while Ksplice is applying +or reversing an update; this approach also protects against race conditions +involving multiple Ksplice updates being loaded or unloaded simultaneously as +well. + +5. altinstructions, smplocks, and parainstructions +-------------------------------------------------- + +There are currently several mechanisms through which the Linux kernel will +modify executable code at runtime. + +These mechanisms sometimes overwrite the storage unit of a relocation, which +would cause problems if not handled properly by Ksplice. + +Ksplice solves this problem by writing "canary" bytes (e.g., 0x77777777) in the +storage unit of the relocation in user space. Ksplice then checks whether the +canary has been overwritten before using a Ksplice relocation to detect symbol +values or to write a value to the storage unit of a Ksplice relocation. + +6. sysfs interface +------------------ + +Ksplice exports four sysfs files per Ksplice update in order to communicate with +user space. For each update, these four files are located in a directory of the +form /sys/kernel/ksplice/$kid, with $kid replaced by the KID of the Ksplice +update. + +A. /sys/kernel/ksplice/$kid/stage (mode 0600) + +This file contains one of three strings: +preparing: Indicates that this update has not yet been applied +applied: Indicates that this update has been applied and has not been reversed +reversed: Indicates that this update has been reversed + +When the stage is "preparing", the superuser can write "applied" to the stage +file in order to instruct Ksplice to apply the update. When the stage is +"applied", the superuser can write "reversed" to the stage file in order to +instruct Ksplice to reverse the update. After an update has been reversed, its +stage cannot change again (although the update can be removed from the kernel +entirely, in which case its $kid directory will be removed). + +B. /sys/kernel/ksplice/$kid/debug (mode 0600) + +The file contains a single number: 1 if debugging is enabled for this Ksplice +update and 0 otherwise. + +The superuser can write a new value to this file to enable or disable debugging. + +C. /sys/kernel/ksplice/$kid/abort_cause (mode 0400) + +This file contains a value indicating either 1) that Ksplice successfully +completed the most recently requested stage transition or 2) why Ksplice aborted +the most recently requested stage transition. + +Each abort_code string is described below, along with the stage transitions that +might potentially trigger each possible abort code. The stage transitions are +abbreviated as follows: preparing->applied (P->A), applied->reversed (A->R). + +ok (P->A, A->R): The most recent stage transition succeeded. + +no_match (P->A): Ksplice aborted the update because Ksplice was unable to match +the helper module's object code against the running kernel's object code. + +failed_to_find (P->A): Ksplice aborted the update because Ksplice was unable to +resolve some of the symbols used in the update. + +missing_export (P->A): Ksplice aborted the update because the symbols exported +by the kernel did not match Ksplice's expectations based on the ksplice_export +structures provided to Ksplice. + +already_reversed (P->A): Ksplice aborted the update because once an update has +been reversed, it cannot be applied again (without first being cleaned up and +reinitialized). + +module_busy (A->R): Ksplice aborted the undo operation because the target +Ksplice update is in use by another kernel module; specifically, either the +target Ksplice update exports a symbol that is in use by another module or +another Ksplice update depends on this Ksplice update. + +out_of_memory (P->A, A->R): Ksplice aborted the operation because a call to +kmalloc or vmalloc failed. + +code_busy (P->A, A->R): Ksplice aborted the operation because Ksplice was +unable to find a moment when one or more of the to-be-patched functions was not +a thread's kernel stack. + +unexpected_running_task (P->A, A->R): Ksplice aborted the operation because +Ksplice observed a running task during the kernel stack check, at a time when +Ksplice expected all tasks to be stopped by stop_machine. + +unexpected (P->A, A->R): Ksplice aborted the operation because it encountered +an unspecified internal error. This condition can only be caused by an invalid +input to Ksplice or a bug in Ksplice. + +D. /sys/kernel/ksplice/$kid/conflicts (mode 0400) + +This file is empty until Ksplice aborts an operation because of a code_busy +condition (see "abort_code" above). This conflicts file then contains +information about the process(es) that caused the stack check failure. + +Specifically, each line of this file consists of three space-separated values, +describing a single conflict: + +$program_name $program_pid $conflict_label + +$program_name is the name of the program with the conflict. +$program_pid is the pid of the program with the conflict. +$conflict_label is the Ksplice label of the function with the conflict. + +7. debugfs interface +-------------------- + +Ksplice exports a single file to debugfs for each Ksplice update. The file has +a name of the form ksplice_KID, where KID is the unique identifier of the +Ksplice update. It contains debugging information in a human-readable format. -- 1.5.4.3
| |