lkml.org 
[lkml]   [2018]   [Dec]   [6]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCHv2] locking/atomics: build atomic headers as required
On Thu, Dec 06, 2018 at 05:13:44PM +0100, Ingo Molnar wrote:
>
> * Mark Rutland <mark.rutland@arm.com> wrote:
>
> > Andrew and Ingo report that the check-atomics.sh script is simply too
> > slow to run for every kernel build, and it's impractical to make it
> > faster without rewriting it in something other than shell.
> >
> > Rather than committing the generated headers, let's regenerate these
> > as-required, if we change the data or scripts used to generate the
> > atomic headers, or when building from a pristine tree.
> >
> > That ensures they're always up-to-date, allows them to be built in
> > parallel, and avoid redundant rebuilds, which is a 2-8s saving per
> > incremental build. Since the results are not committed, it's very
> > obvious that they should not be modified directly. If we need to
> > generate more headers in future, it's easy to extend Makefile.genheader
> > to permit this.
> >
> > I've verified that this works in the cases we previously had issues with
> > (out-of-tree builds and where scripts have no execute permissions), and
> > have tested these cases for both x86_64 and arm64.
> >
> > The diffstat looks nice, at least...
> >
> > Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> > Cc: Andrew Morton <akpm@linux-foundation.org>
> > Cc: Boqun Feng <boqun.feng@gmail.com>
> > Cc: Borislav Petkov <bp@suse.de>
> > Cc: Ingo Molnar <mingo@kernel.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Will Deacon <will.deacon@arm.com>
> > Cc: linux-kernel@vger.kernel.org
> > ---
> > Kbuild | 18 +-
> > Makefile | 8 +-
> > arch/arm64/include/asm/atomic.h | 2 +-
> > arch/x86/include/asm/atomic.h | 2 +-
> > include/asm-generic/atomic-instrumented.h | 1787 ----------------------
> > include/asm-generic/atomic-long.h | 1012 -------------
> > include/linux/atomic-fallback.h | 2294 -----------------------------
> > include/linux/atomic.h | 4 +-
> > scripts/Makefile.genheader | 26 +
> > scripts/atomic/check-atomics.sh | 19 -
> > 10 files changed, 38 insertions(+), 5134 deletions(-)
> > delete mode 100644 include/asm-generic/atomic-instrumented.h
> > delete mode 100644 include/asm-generic/atomic-long.h
> > delete mode 100644 include/linux/atomic-fallback.h
> > create mode 100644 scripts/Makefile.genheader
> > delete mode 100755 scripts/atomic/check-atomics.sh
>
> So these 'automatically generated' headers are actual and important code,
> and I think it's bad practice to remove these from the git grep search
> space and history as well.

Ok. Let's aim for that, then. :)

> Really, this whole notion of auto-generating the headers should be
> implemented correctly, instead of working around deficiencies in a
> short-term fashion that introduces other deficiencies.

I certainly agree.

> I never got any replies to my previous comments about this:
>
> <20181128083057.GA7879@gmail.com>
>
> Did I miss some mails of yours perhaps?

It appears my mailserver dropped that on the floor, as I can't find it
in any of my mailboxes. :(

Sorry about that; I'll try to figure out what happened there.

This patch covers your suggestion of parallelizing things. On my box
with -j64, that saves ~1.5s for a pristine allnoconfig build which
otherwise takes ~15s.

Practically, optimizing the scripts requires moving away from shell.
The cost is all down to sub-shells resulting in fork+exec, but the
scripts would be illegible without this.

One option is to generate a checksum when the headers are generated and
checked in, and check that at build time. I'll look into this tomorrow.

Thanks,
Mark.

\
 
 \ /
  Last update: 2018-12-06 18:46    [W:0.045 / U:1.324 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site