lkml.org 
[lkml]   [2018]   [Jul]   [3]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: [PATCH v6 21/21] s390: doc: detailed specifications for AP virtualization
    From
    Date
    On 07/03/2018 07:52 AM, Cornelia Huck wrote:
    > On Tue, 3 Jul 2018 11:22:10 +0200
    > Halil Pasic <pasic@linux.ibm.com> wrote:
    >
    >> On 07/03/2018 09:46 AM, Harald Freudenberger wrote:
    >>> On 02.07.2018 18:28, Halil Pasic wrote:
    >>>>
    >>>> On 06/29/2018 11:11 PM, Tony Krowiak wrote:
    >>>>> This patch provides documentation describing the AP architecture and
    >>>>> design concepts behind the virtualization of AP devices. It also
    >>>>> includes an example of how to configure AP devices for exclusive
    >>>>> use of KVM guests.
    >>>>>
    >>>>> Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
    >>>>> ---
    >>>> [..]
    >>>>> +
    >>>>> +Reserve APQNs for exclusive use of KVM guests
    >>>>> +---------------------------------------------
    >>>>> +The following block diagram illustrates the mechanism by which APQNs are
    >>>>> +reserved:
    >>>>> +
    >>>>> + +------------------+
    >>>>> + remove | | unbind
    >>>>> + +------------------->+ cex4queue driver +<-----------+
    >>>>> + | | | |
    >>>>> + | +------------------+ |
    >>>>> + | |
    >>>>> + | |
    >>>>> + | |
    >>>>> ++--------+---------+ register +------------------+ +-----+------+
    >>>>> +| +<---------+ | bind | |
    >>>>> +| ap_bus | | vfio_ap driver +<-----+ admin |
    >>>>> +| +--------->+ | | |
    >>>>> ++------------------+ probe +---+--------+-----+ +------------+
    >>>>> + | |
    >>>>> + create | | store APQN
    >>>>> + | |
    >>>>> + v v
    >>>>> + +---+--------+-----+
    >>>>> + | |
    >>>>> + | matrix device |
    >>>>> + | |
    >>>>> + +------------------+
    >>>>> +
    >>>>> +The process for reserving an AP queue for use by a KVM guest is:
    >>>>> +
    >>>>> +* The vfio-ap driver during its initialization will perform the following:
    >>>>> + * Create the 'vfio_ap' root device - /sys/devices/vfio_ap
    >>>>> + * Create the 'matrix' device in the 'vfio_ap' root
    >>>>> + * Register the matrix device with the device core
    >>>>> +* Register with the ap_bus for AP queue devices of type 10 devices (CEX4 and
    >>>>> + newer) and to provide the vfio_ap driver's probe and remove callback
    >>>>> + interfaces. The reason why older devices are not supported is because there
    >>>>> + are no systems available on which to test.
    >>>>> +* The admin unbinds queue cc.qqqq from the cex4queue device driver. This results
    >>>>> + in the ap_bus calling the the device driver's remove interface which
    >>>>> + unbinds the cc.qqqq queue device from the driver.
    >>>> What if the queue cc.qqqq is already in use? AFAIU unbind is almost as radical as
    >>>> pulling a cable. What is the proper procedure an admin should follow before doing
    >>>> the unbind?
    >>> What do you mean on this level with 'in use'? A unbind destroys the association
    >>> between device and driver. There is no awareness of 'in use' or 'not in use' on this
    >>> level. This is a hard unbind.
    >>>>
    >>
    >> Let me try to invoke the DASD analogy. If one for some reason wants to detach
    >> a DASD the procedure to follow seems to be (see
    >> https://www.ibm.com/support/knowledgecenter/en/linuxonibm/com.ibm.linux.z.lgdd/lgdd_t_dasd_online.html)
    >> the following:
    >> 1) Unmount.
    >> 2) Offline possibly using safe_offline.
    >> 3) Detach.
    >>
    >> Detaching a disk that is currently doing I/O asks for trouble, so the admin is encouraged
    >> to make sure there is no pending I/O.
    > I don't think we can use dasd (block devices) as a good analogy for
    > every kind of device (for starters, consider network devices).
    >
    >> In case of AP you can interpret my 'in use' as the queue is not empty. In my understanding
    >> unbind is supposed to be hard (I used the word radical). That's why I compared it to pulling
    >> a cable. So that's why I ask is there stuff the admin is supposed to do before doing the
    >> unbind.
    > Are you asking for a kind of 'quiescing' operation? I would hope that
    > the crypto drivers already can deal with that via flushing the queue,
    > not allowing new requests, or whatever. This is not the block device
    > case.

    As I stated in Message ID:
    <89df60be-63d6-3ed9-4724-321e5b55d50d@linux.ibm.com>,
    I believe the queue is flushed when the remove callback is invoked on the
    driver.

    >
    > Anyway, this is an administrative issue. If you don't have a clear
    > concept which devices are for host usage and which for guest usage, you
    > already have problems.
    >
    > Speaking of administrative issues, is there libvirt support for vfio-ap
    > under development? It would be helpful to validate the approach.

    There is libvirt support under development although it is not very far
    along at this point.

    >

    \
     
     \ /
      Last update: 2018-07-03 17:02    [W:4.400 / U:0.044 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site