[lkml]   [2009]   [Mar]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [PATCH 3/3] utrace-based ftrace "process" engine, v2

* Frank Ch. Eigler <> wrote:

> Hi -
> On Sat, Mar 21, 2009 at 04:45:01PM +0100, Ingo Molnar wrote:
> > [...]
> > To me personally there are two big direct usability issues with
> > SystemTap:
> >
> > 1) It relies on DEBUG_INFO for any reasonable level of utility.
> > Yes, it will limp along otherwise as well, but most of the
> > actual novel capabilities depend on debuginfo. Which is an
> > acceptable constraint for enterprise usage where kernels are
> > switched every few months and having a debuginfo package is not
> > a big issue. Not acceptable for upstream kernel development.
> In my own limited kernel-building experience, I find the debuginfo
> data conveniently and instantly available after every "make". Can
> you elaborate how is it harder for you to incidentally make it
> than for someone to download it?

Four reasons:


I have CONFIG_DEBUG_INFO turned off in 99.9% of my kernel builds,
because it slows down the kernel build times significantly:

without: 4343.31 user 416.39 system 6:09.97 elapsed 1286%CPU
with: 4871.07 user 501.90 system 7:43.22 elapsed 1159 %CPU

( x86 allyesconfig. On an obscenely overpowered Nehalem box
with 12 GB of RAM. )


When the kernel build becomes IO-bound, for example when i build
over a distcc cluster (which is how i generally build my kernels) -
or when others with less RAM build a debuginfo kernel, the ratio
becomes even worse:

without: 870.36 user 292.79 system 3:32.10 elapsed 548% CPU
with: 929.65 user 384.55 system 8:28.70 elapsed 258% CPU


Another metric. Here's an x86 defconfig (i.e. fairly regular config
- not allyesconfig) build's size:

with: 1645 MB
without: 211 MB

Try to build 1.6 GB of dirty data on ext3 and run into an fsync() in
your editor ... you'll sit there twiddling thumbs for a minute or


Or yet another metric - Linux distro package overhead. Try
installing a debuginfo package:

# yum install kernel-debuginfo

Package Arch Version
kernel-debuginfo x86_64 2.6.29-0.258.rc8.git2.fc11
rawhide-debuginfo 294 M
Installing for dependencies:
kernel-debuginfo-common x86_64 2.6.29-0.258.rc8.git2.fc11
rawhide-debuginfo 35 M

Total download size: 329 M

That size of a _compressed_ debuginfo kernel package is obscene. We
can fit 4 years of full Linux kernel Git history into that size -
60,000+ commits, full metadata and full 20 million lines of code
flux included!

Uncompressed it blows up to gigabytes of on-disk data.

And this download has to be repeated for _every_ minor kernel

So when i come into a situation where i could use some debugging
help ... i'd have to rebuild the kernel with DEBUG_INFO=y and i'll
always notice when i have a debuginfo kernel because it's

The solution?)

Dunno - but i definitely think we should think bigger:

The fundamental disconnect i believe seems to come from the fact
that most user-space projects are relatively small, so debuginfo
bloat is a secondary issue there.

But for a project with the size of the kernel, even for moderate
builds (not allyesconfig), it's a _much_ bigger deal. This has been
known for a long time and the situation has become worse over the
last two years, not better. (last time i checked the debuginfo
package overhead it was below 150 MB)

A few random ideas:

Instead of trying to cache 2+GB of debuginfo for a 50 MB kernel
source repo (+50 MB of genuine .o output) - just to be able to debug
one or two source files [which is the typical scope of a debugging
session], why not build debuginfo on the fly, when a debugging
session requires it? Rarely do we need debuginfo for more than a
fraction of the whole kernel.

( Yes, it needs a few smarts like knowing the SHA1 of the source
code module that a particular kernel portion got built with, to
make sure the debuginfo is fresh and relevant - but nothing major. )

I mean, lets _use_ the fact that we have source code available, more
intelligently. It takes zero time to build detailed debuginfo for a
portion of a tree.

If 'download debuginfo' can be replaced with: 'have a recent Git
repository of the distro kernel source', we'll have a _much_ more
efficient use of resources all around.


 \ /
  Last update: 2009-03-22 13:13    [W:0.098 / U:0.212 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site