lkml.org 
[lkml]   [2020]   [Dec]   [16]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: [PATCH v12 14/17] s390/zcrypt: Notify driver on config changed and scan complete callbacks
    From
    Date


    On 11/30/20 4:18 AM, h@d06av26.portsmouth.uk.ibm.com wrote:
    > On Tue, 24 Nov 2020 16:40:13 -0500
    > Tony Krowiak <akrowiak@linux.ibm.com> wrote:
    >
    >> This patch intruduces an extension to the ap bus to notify device drivers
    >> when the host AP configuration changes - i.e., adapters, domains or
    >> control domains are added or removed. To that end, two new callbacks are
    >> introduced for AP device drivers:
    >>
    >> void (*on_config_changed)(struct ap_config_info *new_config_info,
    >> struct ap_config_info *old_config_info);
    >>
    >> This callback is invoked at the start of the AP bus scan
    >> function when it determines that the host AP configuration information
    >> has changed since the previous scan. This is done by storing
    >> an old and current QCI info struct and comparing them. If there is any
    >> difference, the callback is invoked.
    >>
    >> Note that when the AP bus scan detects that AP adapters, domains or
    >> control domains have been removed from the host's AP configuration, it
    >> will remove the associated devices from the AP bus subsystem's device
    >> model. This callback gives the device driver a chance to respond to
    >> the removal of the AP devices from the host configuration prior to
    >> calling the device driver's remove callback. The primary purpose of
    >> this callback is to allow the vfio_ap driver to do a bulk unplug of
    >> all affected adapters, domains and control domains from affected
    >> guests rather than unplugging them one at a time when the remove
    >> callback is invoked.
    >>
    >> void (*on_scan_complete)(struct ap_config_info *new_config_info,
    >> struct ap_config_info *old_config_info);
    >>
    >> The on_scan_complete callback is invoked after the ap bus scan is
    >> complete if the host AP configuration data has changed.
    >>
    >> Note that when the AP bus scan detects that adapters, domains or
    >> control domains have been added to the host's configuration, it will
    >> create new devices in the AP bus subsystem's device model. The primary
    >> purpose of this callback is to allow the vfio_ap driver to do a bulk
    >> plug of all affected adapters, domains and control domains into
    >> affected guests rather than plugging them one at a time when the
    >> probe callback is invoked.
    >>
    >> Please note that changes to the apmask and aqmask do not trigger
    >> these two callbacks since the bus scan function is not invoked by changes
    >> to those masks.
    >>
    >> Signed-off-by: Harald Freudenberger <freude@linux.ibm.com>
    >> Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
    >> ---
    >> drivers/s390/crypto/ap_bus.c | 83 ++++++++++++++++++++++++++-
    >> drivers/s390/crypto/ap_bus.h | 12 ++++
    >> drivers/s390/crypto/vfio_ap_private.h | 14 ++++-
    >> 3 files changed, 106 insertions(+), 3 deletions(-)
    >>
    >> diff --git a/drivers/s390/crypto/ap_bus.c b/drivers/s390/crypto/ap_bus.c
    >> index 593573740981..3a63f6b33d8a 100644
    >> --- a/drivers/s390/crypto/ap_bus.c
    >> +++ b/drivers/s390/crypto/ap_bus.c
    >> @@ -75,6 +75,7 @@ DEFINE_MUTEX(ap_perms_mutex);
    >> EXPORT_SYMBOL(ap_perms_mutex);
    >>
    >> static struct ap_config_info *ap_qci_info;
    >> +static struct ap_config_info *ap_qci_info_old;
    >>
    >> /*
    >> * AP bus related debug feature things.
    >> @@ -1440,6 +1441,52 @@ static int __match_queue_device_with_queue_id(struct device *dev, const void *da
    >> && AP_QID_QUEUE(to_ap_queue(dev)->qid) == (int)(long) data;
    >> }
    >>
    >> +/* Helper function for notify_config_changed */
    >> +static int __drv_notify_config_changed(struct device_driver *drv, void *data)
    >> +{
    >> + struct ap_driver *ap_drv = to_ap_drv(drv);
    >> +
    >> + if (try_module_get(drv->owner)) {
    >> + if (ap_drv->on_config_changed)
    >> + ap_drv->on_config_changed(ap_qci_info,
    >> + ap_qci_info_old);
    >> + module_put(drv->owner);
    >> + }
    >> +
    >> + return 0;
    >> +}
    >> +
    >> +/* Notify all drivers about an qci config change */
    >> +static inline void notify_config_changed(void)
    >> +{
    >> + bus_for_each_drv(&ap_bus_type, NULL, NULL,
    >> + __drv_notify_config_changed);
    >> +}
    >> +
    >> +/* Helper function for notify_scan_complete */
    >> +static int __drv_notify_scan_complete(struct device_driver *drv, void *data)
    >> +{
    >> + struct ap_driver *ap_drv = to_ap_drv(drv);
    >> +
    >> + if (try_module_get(drv->owner)) {
    >> + if (ap_drv->on_scan_complete)
    >> + ap_drv->on_scan_complete(ap_qci_info,
    >> + ap_qci_info_old);
    >> + module_put(drv->owner);
    >> + }
    >> +
    >> + return 0;
    >> +}
    >> +
    >> +/* Notify all drivers about bus scan complete */
    >> +static inline void notify_scan_complete(void)
    >> +{
    >> + bus_for_each_drv(&ap_bus_type, NULL, NULL,
    >> + __drv_notify_scan_complete);
    >> +}
    >> +
    >> +
    >> +
    >> /*
    >> * Helper function for ap_scan_bus().
    >> * Remove card device and associated queue devices.
    >> @@ -1718,15 +1765,43 @@ static inline void ap_scan_adapter(int ap)
    >> put_device(&ac->ap_dev.device);
    >> }
    >>
    >> +static int ap_get_configuration(void)
    > I believe this was Haralds request. I'm OO contaminated, but
    > the signature and the semantic does not mash well with my understanding
    > of a 'getter'. Especially the return value being actually a boolean and
    > 'configuration changed/still the same'. From the signature it looks more
    > like the usual C-stlyle try to do something and return 0 if OK, otherwise
    > error code != 0.

    I know where you're coming from as I spent 20 years as a C++
    then a java programmer. The function does actually get the
    latest host AP configuration and stores both the old and new
    configurations, so in essence, the name is descriptive of
    the function.

    >
    > Since it's Haralds dominion, I'm not asking you to change this, but we
    > could at least document the return value (maybe also the behavior).

    I will document the function in the next version.

    >
    >> +{
    >> + int cfg_chg = 0;
    >> +
    >> + if (ap_qci_info) {
    >> + if (!ap_qci_info_old) {
    >> + ap_qci_info_old = kzalloc(sizeof(*ap_qci_info_old),
    >> + GFP_KERNEL);
    >> + if (!ap_qci_info_old)
    >> + return 0;
    >> + } else {
    >> + memcpy(ap_qci_info_old, ap_qci_info,
    >> + sizeof(struct ap_config_info));
    >> + }
    >> + ap_fetch_qci_info(ap_qci_info);
    >> + cfg_chg = memcmp(ap_qci_info,
    >> + ap_qci_info_old,
    >> + sizeof(struct ap_config_info)) != 0;
    >> + }
    >> +
    >> + return cfg_chg;
    >> +}
    >> +
    >> /**
    >> * ap_scan_bus(): Scan the AP bus for new devices
    >> * Runs periodically, workqueue timer (ap_config_time)
    >> */
    >> static void ap_scan_bus(struct work_struct *unused)
    >> {
    >> - int ap;
    >> + int ap, config_changed = 0;
    >>
    >> - ap_fetch_qci_info(ap_qci_info);
    >> + /* config change notify */
    >> + config_changed = ap_get_configuration();
    >> + if (config_changed)
    >> + notify_config_changed();
    >> + memcpy(ap_qci_info_old, ap_qci_info,
    >> + sizeof(struct ap_config_info));
    > Why is this memcpy needed? Isn't that already take care of in
    > ap_get_configuration()?

    It is not needed and is actually an error since that copy is
    already done in the ap_get_configuration() function.
    Fortunately, it doesn't cause a problem since the vfio_ap
    driver has already been notified of the configuration
    change. Good catch, I'll remove this for the next version.

    >
    >> ap_select_domain();
    >>
    >> AP_DBF_DBG("%s running\n", __func__);
    >> @@ -1735,6 +1810,10 @@ static void ap_scan_bus(struct work_struct *unused)
    >> for (ap = 0; ap <= ap_max_adapter_id; ap++)
    >> ap_scan_adapter(ap);
    >>
    >> + /* scan complete notify */
    >> + if (config_changed)
    >> + notify_scan_complete();
    >> +
    >> /* check if there is at least one queue available with default domain */
    >> if (ap_domain_index >= 0) {
    >> struct device *dev =
    >> diff --git a/drivers/s390/crypto/ap_bus.h b/drivers/s390/crypto/ap_bus.h
    >> index 65edd847c65a..fbfbf6991718 100644
    >> --- a/drivers/s390/crypto/ap_bus.h
    >> +++ b/drivers/s390/crypto/ap_bus.h
    >> @@ -146,6 +146,18 @@ struct ap_driver {
    >> int (*probe)(struct ap_device *);
    >> void (*remove)(struct ap_device *);
    >> int (*in_use)(unsigned long *apm, unsigned long *aqm);
    >> + /*
    >> + * Called at the start of the ap bus scan function when
    >> + * the crypto config information (qci) has changed.
    >> + */
    >> + void (*on_config_changed)(struct ap_config_info *new_config_info,
    >> + struct ap_config_info *old_config_info);
    >> + /*
    >> + * Called at the end of the ap bus scan function when
    >> + * the crypto config information (qci) has changed.
    >> + */
    >> + void (*on_scan_complete)(struct ap_config_info *new_config_info,
    >> + struct ap_config_info *old_config_info);
    >> };
    >>
    >> #define to_ap_drv(x) container_of((x), struct ap_driver, driver)
    >> diff --git a/drivers/s390/crypto/vfio_ap_private.h b/drivers/s390/crypto/vfio_ap_private.h
    >> index 15b7cd74843b..7bd7e35eb2e0 100644
    >> --- a/drivers/s390/crypto/vfio_ap_private.h
    >> +++ b/drivers/s390/crypto/vfio_ap_private.h
    > These changes probably belong to some next patch...
    >
    > With the things I just brought up clarified, you can slap a:
    > Reviewed-by: Halil Pasic <pasic@linux.ibm.com>
    > over it next time.
    >
    >> @@ -36,14 +36,21 @@
    >> * driver, be it using @mdev_list or writing the state of a
    >> * single ap_matrix_mdev device. It's quite coarse but we don't
    >> * expect much contention.
    >> + ** @ap_add: a bitmap specifying the APIDs added to the host AP configuration
    >> + * as notified by the AP bus via the on_cfg_chg callback.
    >> + * @aq_add: a bitmap specifying the APQIs added to the host AP configuration
    >> + * as notified by the AP bus via the on_cfg_chg callback.
    >> */
    >> struct ap_matrix_dev {
    >> struct device device;
    >> atomic_t available_instances;
    >> - struct ap_config_info info;
    >> + struct ap_config_info config_info;
    >> + struct ap_config_info config_info_prev;
    >> struct list_head mdev_list;
    >> struct mutex lock;
    >> struct ap_driver *vfio_ap_drv;
    >> + DECLARE_BITMAP(ap_add, AP_DEVICES);
    >> + DECLARE_BITMAP(aq_add, AP_DEVICES);
    >> };
    >>
    >> extern struct ap_matrix_dev *matrix_dev;
    >> @@ -90,6 +97,8 @@ struct ap_matrix_mdev {
    >> struct kvm_s390_module_hook pqap_hook;
    >> struct mdev_device *mdev;
    >> DECLARE_HASHTABLE(qtable, 8);
    >> + DECLARE_BITMAP(ap_add, AP_DEVICES);
    >> + DECLARE_BITMAP(aq_add, AP_DEVICES);
    >> };
    >>
    >> extern int vfio_ap_mdev_register(void);
    >> @@ -109,4 +118,7 @@ void vfio_ap_mdev_remove_queue(struct ap_device *queue);
    >>
    >> int vfio_ap_mdev_resource_in_use(unsigned long *apm, unsigned long *aqm);
    >>
    >> +void vfio_ap_on_cfg_changed(struct ap_config_info *new_config_info,
    >> + struct ap_config_info *old_config_info);
    >> +
    >> #endif /* _VFIO_AP_PRIVATE_H_ */

    \
     
     \ /
      Last update: 2020-12-16 21:57    [W:4.625 / U:0.376 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site