Messages in this thread |  | | Date | Tue, 15 Apr 2025 12:07:05 +0200 | | From | Peter Zijlstra <> | | Subject | Re: [PATCH] sched/fair: Add null pointer check to pick_next_entity() |
| |
On Mon, Apr 14, 2025 at 11:38:15AM -0400, Chris Mason wrote: > > > On 4/14/25 5:08 AM, Peter Zijlstra wrote: > > [ math and such ] > > > > The zero_vruntime patch I gave earlier should avoid this particular > > issue. > > Here's a crash with the zero runtime patch.
And indeed it doesn't have these massive (negative) avg_vruntime values.
> I'm trying to reproduce > this outside of prod so we can crank up the iteration speed a bit.
Thanks.
Could you add which pick went boom for the next dump?
I am however, slightly confused by this output format.
It looks like it dumps the cfs_rq the first time it encounters it, either through curr or through the tree.
So if I read this correct the root is something like:
> nr_running = 2 > zero_vruntime = 19194347104893960 > avg_vruntime = 6044054790 > avg_load = 2 > curr = { > cgroup urgent > vruntime = 24498183812106172 > weight = 3561684 => 3478 > } > tasks_timeline = [ > { > cgroup optional > vruntime = 19194350126921355 > weight = 1168 => 2 > }, > ]
group 19194347104893960 curr 24498183812106172 3561684 entity 19194350126921355 1168
But if I run those numbers, I get avg_load == 1, seeing how 1168/1024 = 1. But the thing says it should be 2.
Similarly, my avg_vruntime is exactly half of what it says.
avg_vruntime: 3022027395 avg_load: 1
(seeing how 19194350126921355-19194347104893960 = 3022027395)
Anyway, with curr being significantly to the right of that, the 0-lag point is well right of where optional sits. So this pick should be fine, and result in 'optional' getting selected (curr is no longer eligible).
All the urgent/* groups have nr_running == 0, so are not interesting, we'll not pick there.
NOTE: I'm inferring curr is on_rq, because nr_running == 2 and the tree only has 1 entity in it.
NOTE: if we ignore curr, then optional sits at exactly the 0-lag point, with either sets of numbers and so should be eligible.
This then leaves us the optional/* groups.
> cgroup optional > rq = { > nr_running = 2 > zero_vruntime = 440280059357029 > avg_vruntime = 476 > avg_load = 688 > tasks_timeline = [ > { > cgroup optional/-610613050111295488 > vruntime = 440280059333960 > weight = 291271 => 284 > }, > { > cgroup optional/-610609318858457088 > vruntime = 440280059373247 > weight = 413911 => 404 > },
group 440280059357029 entity 440280059333960 291271 entity 440280059373247 413911
Which gives:
avg_vruntime: 476 avg_load: 688
And that matches.
Next we have:
> cgroup optional/-610613050111295488 > rq = { > nr_running = 5 > zero_vruntime = 65179829005 > avg_vruntime = 0 > avg_load = 75 > tasks_timeline = [ > { > task = 261672 (fc0) > vruntime = 65189926507 > weight = 15360 => 15 > }, > { > task = 261332 (fc0) > vruntime = 65189480962 > weight = 15360 => 15 > }, > { > task = 261329 (enc1:0:vp9_fbv) > vruntime = 65165843516 > weight = 15360 => 15 > }, > { > task = 261334 (dec0:0:hevc_fbv) > vruntime = 65174065035 > weight = 15360 => 15 > }, > { > task = 261868 (fc0) > vruntime = 65179829005 > weight = 15360 => 15 > }, > ] > }
avg_vruntime: 0 avg_load: 75
This again matches, leaving the bottom 3 tasks eligible.
And finally:
> cgroup optional/-610609318858457088 > rq = { > nr_running = 1 > zero_vruntime = 22819875784 > avg_vruntime = 0 > avg_load = 15 > tasks_timeline = [ > { > task = 273291 (fc0) > vruntime = 22819875784 > weight = 15360 => 15 > }, > ] > }
Rather boring indeed, but the numbers appear correct.
So I'm not immediately seeing where it would go boom, but seeing how the root group is the one with dodgy numbers, I would suspect that -- but I'm not immediately seeing how... :-(
|  |