lkml.org 
[lkml]   [2015]   [Apr]   [14]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
Patch in this message
/
Date
From
Subject[PATCH] tags: much faster, parallel "make tags"
ctags is single-threaded program. Split list of files to be tagged into
equal parts, 1 part for each CPU and then merge the results.

Speedup on one 2-way box I have is ~143 s => ~99 s (-31%).
On another 4-way box: ~120 s => ~65 s (-46%!).

Resulting "tags" files aren't byte-for-byte identical because ctags
program numbers anon struct and enum declarations with "__anonNNN"
symbols. If those lines are removed, "tags" file becomes byte-for-byte
identical with those generated with current code.

Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com>
---

scripts/tags.sh | 34 ++++++++++++++++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)

--- a/scripts/tags.sh
+++ b/scripts/tags.sh
@@ -152,7 +152,24 @@ dogtags()

exuberant()
{
- all_target_sources | xargs $1 -a \
+ NR_CPUS=1
+ if [ -e /proc/cpuinfo ]; then
+ NR_CPUS=$(grep -e '^processor : ' /proc/cpuinfo | wc -l)
+ fi
+
+ rm -f .make-tags.src.* .make-tags.*
+
+ all_target_sources >.make-tags.src
+ # seems like Useless Use of cat(1) but not really
+ NR_LINES=$(cat .make-tags.src | wc -l)
+ NR_LINES=$((($NR_LINES + $NR_CPUS - 1) / $NR_CPUS))
+
+ split -a 6 -d -l $NR_LINES .make-tags.src .make-tags.src.
+
+ for i in .make-tags.src.*; do
+ N=$(echo $i | sed -e 's/.*\.//')
+ # -u: don't sort now, sort later
+ cat $i | xargs $1 -a -f .make-tags.$N -u \
-I __initdata,__exitdata,__initconst, \
-I __cpuinitdata,__initdata_memblock \
-I __refdata,__attribute,__maybe_unused,__always_unused \
@@ -211,7 +228,20 @@ exuberant()
--regex-c='/DEFINE_PCI_DEVICE_TABLE\((\w*)/\1/v/' \
--regex-c='/(^\s)OFFSET\((\w*)/\2/v/' \
--regex-c='/(^\s)DEFINE\((\w*)/\2/v/' \
- --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/'
+ --regex-c='/DEFINE_HASHTABLE\((\w*)/\1/v/' \
+ &
+ done
+ wait
+ rm -f .make-tags.src .make-tags.src.*
+
+ # write header
+ $1 -f tags /dev/null
+ # remove header
+ for i in .make-tags.*; do
+ sed -i -e '/^!/d' $i
+ done
+ sort .make-tags.* >>tags
+ rm -f .make-tags.*

all_kconfigs | xargs $1 -a \
--langdef=kconfig --language-force=kconfig \

\
 
 \ /
  Last update: 2015-04-14 19:41    [W:0.063 / U:0.188 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site