lkml.org 
[lkml]   [2021]   [Mar]   [9]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH v2 3/4] kbuild: re-implement CONFIG_TRIM_UNUSED_KSYMS to make it work in one-pass
On Tue, 9 Mar 2021, Rasmus Villemoes wrote:

> On 09/03/2021 20.54, Nicolas Pitre wrote:
> > On Wed, 10 Mar 2021, Masahiro Yamada wrote:
> >
>
> >>> I'm not sure I do understand every detail here, especially since it is
> >>> so far away from the version that I originally contributed. But the
> >>> concept looks good.
> >>>
> >>> I still think that there is no way around a recursive approach to get
> >>> the maximum effect with LTO, but given that true LTO still isn't applied
> >>> to mainline after all those years, the recursive approach brings
> >>> nothing. Maybe that could be revisited if true LTO ever makes it into
> >>> mainline, and the desire to reduce the binary size is still relevant
> >>> enough to justify it.
> >>
> >> Hmm, I am confused.
> >>
> >> Does this patch change the behavior in the
> >> combination with the "true LTO"?
> >>
> >> Please let me borrow this sentence from your article:
> >> "But what LTO does is more like getting rid of branches that simply
> >> float in the air without being connected to anything or which have
> >> become loose due to optimization."
> >> (https://lwn.net/Articles/746780/)
> >>
> >> This patch throws unneeded EXPORT_SYMBOL metadata
> >> into the /DISCARD/ section of the linker script.
> >>
> >> The approach is different (preprocessor vs linker), but
> >> we will still get the same result; the unneeded
> >> EXPORT_SYMBOLs are disconnected from the main trunk.
> >>
> >> Then, the true LTO will remove branches floating in the air,
> >> right?
> >>
> >> So, what will be lost by this patch?
> >
> > Let's say you have this in module_foo:
> >
> > int foo(int x)
> > {
> > return 2 + bar(x);
> > }
> > EXPORT_SYMBOL(foo);
> >
> > And module_bar:
> >
> > int bar(int y)
> > {
> > return 3 * baz(y);
> > }
> > EXPORT_SYMBOL(bar);
> >
> > And this in the main kernel image:
> >
> > int baz(int z)
> > {
> > return plonk(z);
> > }
> > EXPORT_SYMBOLbaz);
> >
> > Now we build the kernel and modules. Then we realize that nothing
> > references symbol "foo". We can trim the "foo" export. But it would be
> > necessary to recompile module_foo with LTO (especially if there is
> > some other code in that module) to realize that nothing
> > references foo() any longer and optimize away the reference to bar().
>
> But, does LTO even do that to modules? Sure, the export metadata for foo
> vanishes, so there's no function pointer reference to foo, but given
> that modules are just -r links, the compiler/linker can't really assume
> that the generated object won't later be linked with something that does
> require foo? At least for the simpler case of --gc-sections, ld docs say:
>
> '--gc-sections'
> ...
>
> This option can be set when doing a partial link (enabled with
> option '-r'). In this case the root of symbols kept must be
> explicitly specified either by one of the options '--entry',
> '--undefined', or '--gc-keep-exported' or by a 'ENTRY' command in
> the linker script.
>
> and I would assume that for LTO, --gc-keep-exported would be the only
> sane semantics (keep any external symbol with default visibility).
>
> Can you point me at a tree/set of LTO patches and a toolchain where the
> previous implementation would actually eventually eliminate bar() from
> module_bar?

All that I readily have is a link to the article I wrote with the
results I obtained at the time: https://lwn.net/Articles/746780/.
The toolchain and kernel tree are rather old at this point and some
effort would be required to modernize everything.

I don't remember if there was something special to do LTO on modules.
Maybe Andi Kleen had something in his patchset for that:
https://github.com/andikleen/linux-misc/blob/lto-415-2/Documentation/lto-build
He mentions that LTO isn't very effective with modules enabled, but what
I demonstrated in myarticle is that LTO becomes very effective with or
without modules as long as CONFIG_TRIM_UNUSED_KSYMS is enabled.

Having CONFIG_TRIM_UNUSED_KSYMS in one pass is likely to still be pretty
effective even if possibly not not optimal. And maybe people don't
really care for the missing 10% anyway (I'm just throwing a number in
the air
here).


Nicolas

\
 
 \ /
  Last update: 2021-03-09 21:46    [W:0.056 / U:0.192 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site