Messages in this thread | | | Date | Wed, 25 May 2011 15:15:50 -0700 | From | Yinghai Lu <> | Subject | Re: [tip:core/rcu] Revert "rcu: Decrease memory-barrier usage based on semi-formal proof" |
| |
On 05/24/2011 09:52 PM, Paul E. McKenney wrote: > On Tue, May 24, 2011 at 05:10:11PM -0700, Yinghai Lu wrote: >> On 05/24/2011 02:23 PM, Yinghai Lu wrote: >>> On 05/23/2011 06:35 PM, Paul E. McKenney wrote: >>>> On Mon, May 23, 2011 at 06:26:23PM -0700, Yinghai Lu wrote: >>>>> On 05/23/2011 06:18 PM, Paul E. McKenney wrote: >>>>> >>>>>> OK, so it looks like I need to get this out of the way in order to track >>>>>> down the delays. Or does reverting PeterZ's patch get you a stable >>>>>> system, but with the longish delays in memory_dev_init()? If the latter, >>>>>> it might be more productive to handle the two problems separately. >>>>>> >>>>>> For whatever it is worth, I do see about 5% increase in grace-period >>>>>> duration when switching to kthreads. This is acceptable -- your >>>>>> 30x increase clearly is completely unacceptable and must be fixed. >>>>>> Other than that, the main thing that affects grace period duration is >>>>>> the setting of CONFIG_HZ -- the smaller the HZ value, the longer the >>>>>> grace-period duration. >>>>> >>>>> for my 1024g system when memory hotadd is enabled in kernel config: >>>>> 1. current linus tree + tip tree: memory_dev_init will take about 100s. >>>>> 2. current linus tree + tip tree + your tree - Peterz patch: >>>>> a. on fedora 14 gcc: will cost about 4s: like old times >>>>> b. on opensuse 11.3 gcc: will cost about 10s. >>>> >>>> So some patch in my tree that is not yet in tip makes things better? >>>> >>>> If so, could you please see which one? Maybe that would give me a hint >>>> that could make things better on opensuse 11.3 as well. >>> >>> today's tip: >>> >>> [ 31.795597] cpu_dev_init done >>> [ 40.930202] memory_dev_init done >>> >> >> another boot from tip got: >> >> [ 35.211927] cpu_dev_init done >> [ 136.053698] memory_dev_init done >> >> wonder if you can have clean revert for >> >> commit a26ac2455ffcf3be5c6ef92bc6df7182700f2114 >>> Author: Paul E. McKenney <paul.mckenney@linaro.org> >>> Date: Wed Jan 12 14:10:23 2011 -0800 >>> >>> rcu: move TREE_RCU from softirq to kthread >>> >>> If RCU priority boosting is to be meaningful, callback invocation must >>> be boosted in addition to preempted RCU readers. Otherwise, in presence >>> of CPU real-time threads, the grace period ends, but the callbacks don't >>> get invoked. If the callbacks don't get invoked, the associated memory >>> doesn't get freed, so the system is still subject to OOM. >>> >>> But it is not reasonable to priority-boost RCU_SOFTIRQ, so this commit >>> moves the callback invocations to a kthread, which can be boosted easily. >>> >>> Also add comments and properly synchronized all accesses to >>> rcu_cpu_kthread_task, as suggested by Lai Jiangshan. >>> >>> Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org> >>> Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> >>> Reviewed-by: Josh Triplett <josh@joshtriplett.org> > > There is a new branch yinghai.2011.05.24a on: > > git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-2.6-rcu.git > > Or will be as soon as kernel.org updates its mirrors. > > I am not sure I could call this "clean", but it does revert that commit > and 11 of the subsequent commits that depend on it. It does build, > and I will test it once my currently running tests complete.
yes, with those revert, there is no delay in 10 times booting.
Thanks
Yinghai
| |