Messages in this thread Patch in this message | | | Date | Wed, 17 May 2023 18:34:13 -0700 | Subject | [PATCH 1/2] sched: don't account throttle time for empty groups | From | Josh Don <> |
| |
It is easy for a cfs_rq to become throttled even when it has no enqueued entities (for example, if we have just put_prev()'d the last runnable task of the cfs_rq, and the cfs_rq is out of quota).
Avoid accounting this time towards total throttle time, since it otherwise falsely inflates the stats.
Signed-off-by: Josh Don <joshdon@google.com> --- kernel/sched/fair.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index f4b8b906d30a..85c2c0c3cab6 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -4768,6 +4768,7 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) { bool renorm = !(flags & ENQUEUE_WAKEUP) || (flags & ENQUEUE_MIGRATED); bool curr = cfs_rq->curr == se; + struct rq *rq = rq_of(cfs_rq); /* * If we're the current task, we must renormalise before calling @@ -4816,8 +4817,12 @@ enqueue_entity(struct cfs_rq *cfs_rq, struct sched_entity *se, int flags) if (cfs_rq->nr_running == 1) { check_enqueue_throttle(cfs_rq); - if (!throttled_hierarchy(cfs_rq)) + if (!throttled_hierarchy(cfs_rq)) { list_add_leaf_cfs_rq(cfs_rq); + } else { + if (!cfs_rq->throttled_clock) + cfs_rq->throttled_clock = rq_clock(rq); + } } } @@ -5423,7 +5428,9 @@ static bool throttle_cfs_rq(struct cfs_rq *cfs_rq) * throttled-list. rq->lock protects completion. */ cfs_rq->throttled = 1; - cfs_rq->throttled_clock = rq_clock(rq); + SCHED_WARN_ON(cfs_rq->throttled_clock); + if (cfs_rq->nr_running) + cfs_rq->throttled_clock = rq_clock(rq); return true; } @@ -5441,7 +5448,10 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq) update_rq_clock(rq); raw_spin_lock(&cfs_b->lock); - cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; + if (cfs_rq->throttled_clock) { + cfs_b->throttled_time += rq_clock(rq) - cfs_rq->throttled_clock; + cfs_rq->throttled_clock = 0; + } list_del_rcu(&cfs_rq->throttled_list); raw_spin_unlock(&cfs_b->lock); -- 2.40.1.606.ga4b1b128d6-goog
| |