lkml.org 
[lkml]   [2012]   [Dec]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
SubjectRe: [RFC PATCH v3 0/3] acpi: Introduce prepare_remove device operation
From
Date
On Mon, 2012-12-03 at 12:25 +0800, Hanjun Guo wrote:
> On 2012/11/30 6:27, Toshi Kani wrote:
> > On Thu, 2012-11-29 at 12:48 +0800, Hanjun Guo wrote:
> >> On 2012/11/29 2:41, Toshi Kani wrote:
> >>> On Wed, 2012-11-28 at 19:05 +0800, Hanjun Guo wrote:
> >>>> On 2012/11/24 1:50, Vasilis Liaskovitis wrote:
> >>>> As you may know, the ACPI based hotplug framework we are working on already addressed
> >>>> this problem, and the way we slove this problem is a bit like yours.
> >>>>
> >>>> We introduce hp_ops in struct acpi_device_ops:
> >>>> struct acpi_device_ops {
> >>>> acpi_op_add add;
> >>>> acpi_op_remove remove;
> >>>> acpi_op_start start;
> >>>> acpi_op_bind bind;
> >>>> acpi_op_unbind unbind;
> >>>> acpi_op_notify notify;
> >>>> #ifdef CONFIG_ACPI_HOTPLUG
> >>>> struct acpihp_dev_ops *hp_ops;
> >>>> #endif /* CONFIG_ACPI_HOTPLUG */
> >>>> };
> >>>>
> >>>> in hp_ops, we divide the prepare_remove into six small steps, that is:
> >>>> 1) pre_release(): optional step to mark device going to be removed/busy
> >>>> 2) release(): reclaim device from running system
> >>>> 3) post_release(): rollback if cancelled by user or error happened
> >>>> 4) pre_unconfigure(): optional step to solve possible dependency issue
> >>>> 5) unconfigure(): remove devices from running system
> >>>> 6) post_unconfigure(): free resources used by devices
> >>>>
> >>>> In this way, we can easily rollback if error happens.
> >>>> How do you think of this solution, any suggestion ? I think we can achieve
> >>>> a better way for sharing ideas. :)
> >>>
> >>> Yes, sharing idea is good. :) I do not know if we need all 6 steps (I
> >>> have not looked at all your changes yet..), but in my mind, a hot-plug
> >>> operation should be composed with the following 3 phases.
> >>
> >> Good idea ! we also implement a hot-plug operation in 3 phases:
> >> 1) acpihp_drv_pre_execute
> >> 2) acpihp_drv_execute
> >> 3) acpihp_drv_post_execute
> >> you may refer to :
> >> https://lkml.org/lkml/2012/11/4/79
> >
> > Great. Yes, I will take a look.
>
> Thanks, any comments are welcomed :)

If I read the code right, the framework calls ACPI drivers differently
at boot-time and hot-add as follows. That is, the new entry points are
called at hot-add only, but .add() is called at both cases. This
requires .add() to work differently.

Boot : .add()
Hot-Add : .add(), .pre_configure(), configure(), etc.

I think the boot-time and hot-add initialization should be done
consistently. While there is difficulty with the current boot sequence,
the framework should be designed to allow them consistent, not make them
diverged.

> >>> 1. Validate phase - Verify if the request is a supported operation. All
> >>> known restrictions are verified at this phase. For instance, if a
> >>> hot-remove request involves kernel memory, it is failed in this phase.
> >>> Since this phase makes no change, no rollback is necessary to fail.
> >>
> >> Yes, we have done this in acpihp_drv_pre_execute, and check following things:
> >>
> >> 1) Hot-plugble or not. the instance kernel memory you mentioned is also checked
> >> when memory device remove;
> >
> > Agreed.
> >
> >> 2) Dependency check involved. For instance, if hot-add a memory device,
> >> processor should be added first, otherwise it's not valid to this operation.
> >
> > I think FW should be the one that assures such dependency. That is,
> > when a memory device object is marked as present/enabled/functioning, it
> > should be ready for the OS to use.
>
> Yes, BIOS should do something for the dependency, because BIOS knows the
> actual hardware topology.

Right.

> The ACPI specification provides _EDL method to
> tell OS the eject device list, but still has no method to tell OS the add device
> list now.

Yes, but I do not think the OS needs special handling for add...

> For some cases, OS should analyze the dependency in the validate phase. For example,
> when hot remove a node (container device), OS should analyze the dependency to get
> the remove order as following:
> 1) Host bridge;
> 2) Memory devices;
> 3) Processor devices;
> 4) Container device itself;

This may be off-topic, but how do you plan to delete I/O devices under a
node? Are you planning to delete all I/O devices along with the node?

On other OS, we made a separate step called I/O chassis delete, which
off-lines all I/O devices under the node, and is required before a node
hot-remove. It basically triggers PCIe hot-remove to detach drivers
from all devices. It does not eject the devices so that they do not
have to be on hot-plug slots. This step runs user-space scripts to
verify if the devices can be off-lined without disrupting user's
applications, and provides comprehensive reports if any of them are in
use. Not sure if Linux's PCI hot-remove has such check, but I thought
I'd mention it. :)

> In this way, we can check that all the devices are hot-plugble or not under the
> container device before execute phase, and further more, we can remove devices
> in order to avoid some crash problems.

Yes, we should check if all the resources under the node can be
off-lined at validate phase. (note, all the devices do not have to have
_EJ0 if that's what you meant by hot-pluggable.)

> >> 3) Race condition check. if the device and its dependent device is in hot-plug
> >> process, another request will be denied.
> >
> > I agree that hot-plug operation should be serialized. I think another
> > request should be either queued or denied based on the caller's intent
> > (i.e. wait-ok or no-wait).
> >
> >> No rollback is needed for the above checks.
> >
> > Great.
> >
> >>> 2. Execute phase - Perform hot-add / hot-remove operation that can be
> >>> rolled-back in case of error or cancel.
> >>
> >> In this phase, we introduce a state machine for the hot-plugble device,
> >> please refer to:
> >> https://lkml.org/lkml/2012/11/4/79
> >>
> >> I think we have the same idea for the major framework, but the ACPI based
> >> hot-plug framework implement it differently in detail, right ?
> >
> > Yes, I am surprised with the similarity. What I described is something
> > we had implemented for other OS. I am still studying how best we can
> > improve the Linux hotplug code. :)
>
> Great! your experience is very appreciable for me. I think we can share ideas
> to achieve a better solution for Linux hotplug code. :)

Sounds great.

Thanks,
-Toshi



\
 
 \ /
  Last update: 2012-12-04 02:01    [W:0.384 / U:0.080 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site