lkml.org 
[lkml]   [2019]   [Feb]   [26]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
SubjectRe: [RFC] percpu: decrease pcpu_nr_slots by 1
On Tue, Feb 26, 2019 at 12:09:28AM +0000, Peng Fan wrote:
> Hi Dennis,
>
> > -----Original Message-----
> > From: dennis@kernel.org [mailto:dennis@kernel.org]
> > Sent: 2019年2月25日 23:24
> > To: Peng Fan <peng.fan@nxp.com>
> > Cc: tj@kernel.org; cl@linux.com; linux-mm@kvack.org;
> > linux-kernel@vger.kernel.org; van.freenix@gmail.com
> > Subject: Re: [RFC] percpu: decrease pcpu_nr_slots by 1
> >
> > On Sun, Feb 24, 2019 at 09:17:08AM +0000, Peng Fan wrote:
> > > Entry pcpu_slot[pcpu_nr_slots - 2] is wasted with current code logic.
> > > pcpu_nr_slots is calculated with `__pcpu_size_to_slot(size) + 2`.
> > > Take pcpu_unit_size as 1024 for example, __pcpu_size_to_slot will
> > > return max(11 - PCPU_SLOT_BASE_SHIFT + 2, 1), it is 8, so the
> > > pcpu_nr_slots will be 10.
> > >
> > > The chunk with free_bytes 1024 will be linked into pcpu_slot[9].
> > > However free_bytes in range [512,1024) will be linked into
> > > pcpu_slot[7], because `fls(512) - PCPU_SLOT_BASE_SHIFT + 2` is 7.
> > > So pcpu_slot[8] is has no chance to be used.
> > >
> > > According comments of PCPU_SLOT_BASE_SHIFT, 1~31 bytes share the
> > same
> > > slot and PCPU_SLOT_BASE_SHIFT is defined as 5. But actually 1~15 share
> > > the same slot 1 if we not take PCPU_MIN_ALLOC_SIZE into consideration,
> > > 16~31 share slot 2. Calculation as below:
> > > highbit = fls(16) -> highbit = 5
> > > max(5 - PCPU_SLOT_BASE_SHIFT + 2, 1) equals 2, not 1.
> > >
> > > This patch by decreasing pcpu_nr_slots to avoid waste one slot and let
> > > [PCPU_MIN_ALLOC_SIZE, 31) really share the same slot.
> > >
> > > Signed-off-by: Peng Fan <peng.fan@nxp.com>
> > > ---
> > >
> > > V1:
> > > Not very sure about whether it is intended to leave the slot there.
> > >
> > > mm/percpu.c | 4 ++--
> > > 1 file changed, 2 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/mm/percpu.c b/mm/percpu.c index
> > > 8d9933db6162..12a9ba38f0b5 100644
> > > --- a/mm/percpu.c
> > > +++ b/mm/percpu.c
> > > @@ -219,7 +219,7 @@ static bool pcpu_addr_in_chunk(struct pcpu_chunk
> > > *chunk, void *addr) static int __pcpu_size_to_slot(int size) {
> > > int highbit = fls(size); /* size is in bytes */
> > > - return max(highbit - PCPU_SLOT_BASE_SHIFT + 2, 1);
> > > + return max(highbit - PCPU_SLOT_BASE_SHIFT + 1, 1);
> > > }
> >
> > Honestly, it may be better to just have [1-16) [16-31) be separate. I'm working
> > on a change to this area, so I may change what's going on here.
> >
> > >
> > > static int pcpu_size_to_slot(int size) @@ -2145,7 +2145,7 @@ int
> > > __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai,
> > > * Allocate chunk slots. The additional last slot is for
> > > * empty chunks.
> > > */
> > > - pcpu_nr_slots = __pcpu_size_to_slot(pcpu_unit_size) + 2;
> > > + pcpu_nr_slots = __pcpu_size_to_slot(pcpu_unit_size) + 1;
> > > pcpu_slot = memblock_alloc(pcpu_nr_slots * sizeof(pcpu_slot[0]),
> > > SMP_CACHE_BYTES);
> > > for (i = 0; i < pcpu_nr_slots; i++)
> > > --
> > > 2.16.4
> > >
> >
> > This is a tricky change. The nice thing about keeping the additional
> > slot around is that it ensures a distinction between a completely empty
> > chunk and a nearly empty chunk.
>
> Are there any issues met before if not keeping the unused slot?
> From reading the code and git history I could not find information.
> I tried this code on aarch64 qemu and did not meet issues.
>

This change would require verification that all paths lead to power of 2
chunk sizes and most likely a BUG_ON if that's not the case.

So while this would work, we're holding onto an additional slot also to
be used for chunk reclamation via pcpu_balance_workfn(). If a chunk was
not a power of 2 resulting in the last slot being entirely empty chunks
we could free stuff a chunk with addresses still in use.

> > It happens to be that the logic creates
> > power of 2 chunks which ends up being an additional slot anyway.
>
>
> So,
> > given that this logic is tricky and architecture dependent,
>
> Could you share more information about architecture dependent?
>

The crux of the logic is in pcpu_build_alloc_info(). It's been some time
since I've thought deeply about it, but I don't believe there is a
guarantee that it will be a power of 2 chunk.

Thanks,
Dennis

\
 
 \ /
  Last update: 2019-02-26 18:33    [W:0.651 / U:0.744 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site