Messages in this thread | | | Subject | Re: [PATCH v3] pseries/drmem: update LMBs after LPM | From | Laurent Dufour <> | Date | Thu, 29 Apr 2021 13:35:30 +0200 |
| |
Le 29/04/2021 à 13:31, Laurent Dufour a écrit : > Le 29/04/2021 à 12:27, Aneesh Kumar K.V a écrit : >> Laurent Dufour <ldufour@linux.ibm.com> writes: >> >>> After a LPM, the device tree node ibm,dynamic-reconfiguration-memory may be >>> updated by the hypervisor in the case the NUMA topology of the LPAR's >>> memory is updated. >>> >>> This is caught by the kernel, but the memory's node is updated because >>> there is no way to move a memory block between nodes. >>> >>> If later a memory block is added or removed, drmem_update_dt() is called >>> and it is overwriting the DT node to match the added or removed LMB. But >>> the LMB's associativity node has not been updated after the DT node update >>> and thus the node is overwritten by the Linux's topology instead of the >>> hypervisor one. >>> >>> Introduce a hook called when the ibm,dynamic-reconfiguration-memory node is >>> updated to force an update of the LMB's associativity. >>> >>> Cc: Tyrel Datwyler <tyreld@linux.ibm.com> >>> Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com> >>> --- >>> >>> V3: >>> - Check rd->dn->name instead of rd->dn->full_name >>> V2: >>> - Take Tyrel's idea to rely on OF_RECONFIG_UPDATE_PROPERTY instead of >>> introducing a new hook mechanism. >>> --- >>> arch/powerpc/include/asm/drmem.h | 1 + >>> arch/powerpc/mm/drmem.c | 35 +++++++++++++++++++ >>> .../platforms/pseries/hotplug-memory.c | 4 +++ >>> 3 files changed, 40 insertions(+) >>> >>> diff --git a/arch/powerpc/include/asm/drmem.h b/arch/powerpc/include/asm/drmem.h >>> index bf2402fed3e0..4265d5e95c2c 100644 >>> --- a/arch/powerpc/include/asm/drmem.h >>> +++ b/arch/powerpc/include/asm/drmem.h >>> @@ -111,6 +111,7 @@ int drmem_update_dt(void); >>> int __init >>> walk_drmem_lmbs_early(unsigned long node, void *data, >>> int (*func)(struct drmem_lmb *, const __be32 **, void *)); >>> +void drmem_update_lmbs(struct property *prop); >>> #endif >>> static inline void invalidate_lmb_associativity_index(struct drmem_lmb *lmb) >>> diff --git a/arch/powerpc/mm/drmem.c b/arch/powerpc/mm/drmem.c >>> index 9af3832c9d8d..f0a6633132af 100644 >>> --- a/arch/powerpc/mm/drmem.c >>> +++ b/arch/powerpc/mm/drmem.c >>> @@ -307,6 +307,41 @@ int __init walk_drmem_lmbs_early(unsigned long node, >>> void *data, >>> return ret; >>> } >>> +/* >>> + * Update the LMB associativity index. >>> + */ >>> +static int update_lmb(struct drmem_lmb *updated_lmb, >>> + __maybe_unused const __be32 **usm, >>> + __maybe_unused void *data) >>> +{ >>> + struct drmem_lmb *lmb; >>> + >>> + /* >>> + * Brut force there may be better way to fetch the LMB >>> + */ >>> + for_each_drmem_lmb(lmb) { >>> + if (lmb->drc_index != updated_lmb->drc_index) >>> + continue; >>> + >>> + lmb->aa_index = updated_lmb->aa_index; >>> + break; >>> + } >>> + return 0; >>> +} >>> + >>> +/* >>> + * Update the LMB associativity index. >>> + * >>> + * This needs to be called when the hypervisor is updating the >>> + * dynamic-reconfiguration-memory node property. >>> + */ >>> +void drmem_update_lmbs(struct property *prop) >>> +{ >>> + if (!strcmp(prop->name, "ibm,dynamic-memory")) >>> + __walk_drmem_v1_lmbs(prop->value, NULL, NULL, update_lmb); >>> + else if (!strcmp(prop->name, "ibm,dynamic-memory-v2")) >>> + __walk_drmem_v2_lmbs(prop->value, NULL, NULL, update_lmb); >>> +} >>> #endif >>> static int init_drmem_lmb_size(struct device_node *dn) >>> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c >>> b/arch/powerpc/platforms/pseries/hotplug-memory.c >>> index 8377f1f7c78e..672ffbee2e78 100644 >>> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c >>> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c >>> @@ -949,6 +949,10 @@ static int pseries_memory_notifier(struct notifier_block >>> *nb, >>> case OF_RECONFIG_DETACH_NODE: >>> err = pseries_remove_mem_node(rd->dn); >>> break; >>> + case OF_RECONFIG_UPDATE_PROPERTY: >>> + if (!strcmp(rd->dn->name, >>> + "ibm,dynamic-reconfiguration-memory")) >>> + drmem_update_lmbs(rd->prop); >>> } >>> return notifier_from_errno(err); >> >> How will this interact with DLPAR memory? When we dlpar memory, >> ibm,configure-connector is used to fetch the new associativity details >> and set drmem_lmb->aa_index correctly there. Once that is done kernel >> then call drmem_update_dt() which will result in the above notifier >> callback? > > When a memory DLPAR operation is done, the in memory DT property > "ibm,dynamic-memory" or "ibm,dynamic-memory-v2" (if existing) have to be updated > to reflect the added/removed memory part. This is done by calling > drmem_update_dt(). > > This patch is addressing the case where the hypervisor has updated the DT > property mentioned above. In that case, the LMB tree should be updated so the > aa_index fields are matching the DT one. This way the next time a memory DLPAR > operation is done the DT properties "ibm,dynamic-memory" or > "ibm,dynamic-memory-v2" will be rebuilt correctly. > >> IIUC, the call back then will update drmem_lmb->aa_index again?
Oh I missed what you pointed out. Please ignore my previous answer, I need to double check code.
> drmem_update_dt() is not updating drmem_lmb->aa_index, that's the oppposite, it > is rebuilding the in memory DT property "ibm,dynamic-memory" or > "ibm,dynamic-memory-v2" based on the data stored in the LMB tree. > > Laurent.
| |