lkml.org 
[lkml]   [2015]   [May]   [10]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH v3] tags: much faster, parallel "make tags"
[fix Andrew's email]

On Sun, May 10, 2015 at 04:26:34PM +0300, Alexey Dobriyan wrote:
> On Sat, May 09, 2015 at 06:07:18AM +0100, Pádraig Brady wrote:
> > On 08/05/15 14:26, Alexey Dobriyan wrote:
>
> > > exuberant()
> > > {
> > > - all_target_sources | xargs $1 -a \
> > > + rm -f .make-tags.*
> > > +
> > > + all_target_sources >.make-tags.src
> > > + NR_CPUS=$(getconf _NPROCESSORS_ONLN 2>/dev/null || echo 1)
> >
> > `nproc` is simpler and available since coreutils 8.1 (2009-11-18)
>
> nproc was discarded because getconf is standartized.
>
> > > + NR_LINES=$(wc -l <.make-tags.src)
> > > + NR_LINES=$((($NR_LINES + $NR_CPUS - 1) / $NR_CPUS))
> > > +
> > > + split -a 6 -d -l $NR_LINES .make-tags.src .make-tags.src.
> >
> > `split -d -nl/$(nproc)` is simpler and available since coreutils 8.8 (2010-12-22)
>
> -nl/ can't count and always make first file somewhat bigger, which is
> suspicious. What else it can't do right?
>
> > > + sort .make-tags.* >>$2
> > > + rm -f .make-tags.*
> >
> > Using sort --merge would speed up significantly?
>
> By ~1 second, yes.
>
> > Even faster would be to get sort to skip the header lines, avoiding the need for sed.
> > It's a bit awkward and was discussed at:
> > http://lists.gnu.org/archive/html/coreutils/2013-01/msg00027.html
> > Summarising that, is if not using merge you can:
> >
> > tlines=$(($(wc -l < "$2") + 1))
> > tail -q -n+$tlines .make-tags.* | LC_ALL=C sort >>$2
> >
> > Or if merge is appropriate then:
> >
> > tlines=$(($(wc -l < "$2") + 1))
> > eval "eval LC_ALL=C sort -m '<(tail -n+$tlines .make-tags.'{1..$(nproc)}')'" >>$2
>
> Might as well teach ctags to do real parallel processing.
> LC_* are set by top level Makefile.
>
> > p.p.s. You may want to `trap EXIT cleanup` to rm -f .make-tags.*
>
> The real question is how to kill ctags reliably.
> Naive
>
> trap 'kill $(jobs -p); rm -f .make-tags.*' TERM INT
>
> doesn't work.
>
> Files are removed, but processes aren't.


\
 
 \ /
  Last update: 2015-05-10 16:01    [W:0.078 / U:0.392 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site