Messages in this thread | | | Date | Sat, 12 Nov 2022 10:51:11 +0800 | Subject | Re: [PATCH] sched/fair: Introduce priority load balance for CFS | From | Song Zhang <> |
| |
Hi, Vincent
On 2022/11/3 17:22, Vincent Guittot wrote: > On Thu, 3 Nov 2022 at 10:20, Song Zhang <zhangsong34@huawei.com> wrote: >> >> >> >> On 2022/11/3 16:33, Vincent Guittot wrote: >>> On Thu, 3 Nov 2022 at 04:01, Song Zhang <zhangsong34@huawei.com> wrote: >>>> >>>> Thanks for your reply! >>>> >>>> On 2022/11/3 2:01, Vincent Guittot wrote: >>>>> On Wed, 2 Nov 2022 at 04:54, Song Zhang <zhangsong34@huawei.com> wrote: >>>>>> >>>>> >>>>> This really looks like a v3 of >>>>> https://lore.kernel.org/all/20220810015636.3865248-1-zhangsong34@huawei.com/ >>>>> >>>>> Please keep versioning. >>>>> >>>>>> Add a new sysctl interface: >>>>>> /proc/sys/kernel/sched_prio_load_balance_enabled >>>>> >>>>> We don't want to add more sysctl knobs for the scheduler, we even >>>>> removed some. Knob usually means that you want to fix your use case >>>>> but the solution doesn't make sense for all cases. >>>>> >>>> >>>> OK, I will remove this knobs later. >>>> >>>>>> >>>>>> 0: default behavior >>>>>> 1: enable priority load balance for CFS >>>>>> >>>>>> For co-location with idle and non-idle tasks, when CFS do load balance, >>>>>> it is reasonable to prefer migrating non-idle tasks and migrating idle >>>>>> tasks lastly. This will reduce the interference by SCHED_IDLE tasks >>>>>> as much as possible. >>>>> >>>>> I don't agree that it's always the best choice to migrate a non-idle task 1st. >>>>> >>>>> CPU0 has 1 non idle task and CPU1 has 1 non idle task and hundreds of >>>>> idle task and there is an imbalance between the 2 CPUS: migrating the >>>>> non idle task from CPU1 to CPU0 is not the best choice >>>>> >>>> >>>> If the non idle task on CPU1 is running or cache hot, it cannot be >>>> migrated and idle tasks can also be migrated from CPU1 to CPU0. So I >>>> think it does not matter. >>> >>> What I mean is that migrating non idle tasks first is not a universal >>> win and not always what we want. >>> >> >> But migrating online tasks first is mostly a trade-off that >> non-idle(Latency Sensitive) tasks can obtain more CPU time and minimize >> the interference caused by IDLE tasks. I think this makes sense in most >> cases, or you can point out what else I need to think about it ? >> >> Best regards. >> >>>> >>>>>> >>>>>> Testcase: >>>>>> - Spawn large number of idle(SCHED_IDLE) tasks occupy CPUs >>>>> >>>>> What do you mean by a large number ? >>>>> >>>>>> - Let non-idle tasks compete with idle tasks for CPU time. >>>>>> >>>>>> Using schbench to test non-idle tasks latency: >>>>>> $ ./schbench -m 1 -t 10 -r 30 -R 200 >>>>> >>>>> How many CPUs do you have ? >>>>> >>>> >>>> OK, some details may not be mentioned. >>>> My virtual machine has 8 CPUs running with a schbench process and 5000 >>>> idle tasks. The idle task is a while dead loop process below: >>> >>> How can you care about latency when you start 10 workers on 8 vCPUs >>> with 5000 non idle threads ? >>> >> >> No no no... spawn 5000 idle(SCHED_IDLE) processes not 5000 non-idle >> threads, and with 10 non-idle schbench workers on 8 vCPUs. > > yes spawn 5000 idle tasks but my point remains the same >
I am so sorry that I have not received your reply for a long time, and I am still waiting for it anxiously. In fact, migrating non-idle tasks 1st works well in most scenarios, so it maybe possible to add a sched_feat(LB_PRIO) to enable or disable that. Finally, I really hope you can give me some better advice.
Best regards.
Song Zhang
| |