Messages in this thread | | | Date | Thu, 15 Feb 2024 18:59:25 -0800 | From | "Paul E. McKenney" <> | Subject | Re: [PATCH 00/30] PREEMPT_AUTO: support lazy rescheduling |
| |
On Thu, Feb 15, 2024 at 04:45:17PM -0800, Ankur Arora wrote: > > Paul E. McKenney <paulmck@kernel.org> writes: > > > On Thu, Feb 15, 2024 at 01:24:59PM -0800, Ankur Arora wrote: > >> > >> Paul E. McKenney <paulmck@kernel.org> writes: > >> > >> > On Wed, Feb 14, 2024 at 07:45:18PM -0800, Paul E. McKenney wrote: > >> >> On Wed, Feb 14, 2024 at 06:03:28PM -0800, Ankur Arora wrote: > >> >> > > >> >> > Paul E. McKenney <paulmck@kernel.org> writes: > >> >> > > >> >> > > On Mon, Feb 12, 2024 at 09:55:24PM -0800, Ankur Arora wrote: > >> >> > >> Hi, > >> >> > >> > >> >> > >> This series adds a new scheduling model PREEMPT_AUTO, which like > >> >> > >> PREEMPT_DYNAMIC allows dynamic switching between a none/voluntary/full > >> >> > >> preemption model. However, unlike PREEMPT_DYNAMIC, it doesn't depend > >> >> > >> on explicit preemption points for the voluntary models. > >> >> > >> > >> >> > >> The series is based on Thomas' original proposal which he outlined > >> >> > >> in [1], [2] and in his PoC [3]. > >> >> > >> > >> >> > >> An earlier RFC version is at [4]. > >> >> > > > >> >> > > This uncovered a couple of latent bugs in RCU due to its having been > >> >> > > a good long time since anyone built a !SMP preemptible kernel with > >> >> > > non-preemptible RCU. I have a couple of fixes queued on -rcu [1], most > >> >> > > likely for the merge window after next, but let me know if you need > >> >> > > them sooner. > >> >> > > >> >> > Thanks. As you can probably tell, I skipped out on !SMP in my testing. > >> >> > But, the attached diff should tide me over until the fixes are in. > >> >> > >> >> That was indeed my guess. ;-) > >> >> > >> >> > > I am also seeing OOM conditions during rcutorture testing of callback > >> >> > > flooding, but I am still looking into this. > >> >> > > >> >> > That's on the PREEMPT_AUTO && PREEMPT_VOLUNTARY configuration? > >> >> > >> >> On two of the PREEMPT_AUTO && PREEMPT_NONE configurations, but only on > >> >> two of them thus far. I am running a longer test to see if this might > >> >> be just luck. If not, I look to see what rcutorture scenarios TREE10 > >> >> and TRACE01 have in common. > >> > > >> > And still TRACE01 and TREE10 are hitting OOMs, still not seeing what > >> > sets them apart. I also hit a grace-period hang in TREE04, which does > >> > CONFIG_PREEMPT_VOLUNTARY=y along with CONFIG_PREEMPT_AUTO=y. Something > >> > to dig into more. > >> > >> So, the only PREEMPT_VOLUNTARY=y configuration is TREE04. I wonder > >> if you would continue to hit the TREE04 hang with CONFIG_PREEMTP_NONE=y > >> as well? > >> (Just in the interest of minimizing configurations.) > > > > I would be happy to, but in the spirit of full disclosure... > > > > First, I have seen that failure only once, which is not enough to > > conclude that it has much to do with TREE04. It might simply be low > > probability, so that TREE04 simply was unlucky enough to hit it first. > > In contrast, I have sufficient data to be reasonably confident that the > > callback-flooding OOMs really do have something to do with the TRACE01 and > > TREE10 scenarios, even though I am not yet seeing what these two scenarios > > have in common that they don't also have in common with other scenarios. > > But what is life without a bit of mystery? ;-) > > :). > > > Second, please see the attached tarball, which contains .csv files showing > > Kconfig options and kernel boot parameters for the various torture tests. > > The portions of the filenames preceding the "config.csv" correspond to > > the directories in tools/testing/selftests/rcutorture/configs. > > So, at least some of the HZ_FULL=y tests don't run into problems. > > > Third, there are additional scenarios hand-crafted by the script at > > tools/testing/selftests/rcutorture/bin/torture.sh. Thus far, none of > > them have triggered, other than via the newly increased difficulty > > of configurating a tracing-free kernel with which to test, but they > > can still be useful in ruling out particular Kconfig options or kernel > > boot parameters being related to a given issue. > > > > But please do take a look at the .csv files and let me know what > > adjustments would be appropriate given the failure information. > > Nothing stands out just yet. Let me start a run here and see if > that gives me some ideas.
Sounds good, thank you!
> I'm guessing the splats don't give any useful information or > you would have attached them ;).
My plan is to extract what can be extracted from the overnight run that I just started. Just in case the fixes have any effect on things, unlikely though that might be given those fixes and the runs that failed.
> Thanks for testing, btw.
The sooner we find them, the sooner they get fixed. ;-)
Thanx, Paul
| |