[lkml]   [2006]   [Jun]   [2]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
SubjectRe: [ckrm-tech] [RFC 3/5] sched: Add CPU rate hard caps
Kirill Korotaev wrote:

>>>>- disk I/O bandwidth:
>>>>we started to use CFQv2, but it is quite poor in this regard. First, it
>>>>doesn't prioritizes writes and async disk operations :( And even for
>>>>sync reads we found some problems we work on now...
>>CKRM (on e-series) had an implementation based on a modified CFQ
>>scheduler. Shailabh is currently working on porting that controller to
>can you explain what was changed by CKRM there? Did you made it to
>control ASYNC read/writes? I don't think so...
In e-series, CFQ was modified to
- maintain request queues per ckrm-class (now resource group) rather
than per-tgid
- explicitly maintain I/O bandwidth of each request queue (in terms of
I/O issued by the I/O scheduler)
- select the "next request queue to service" based on its I/O
bandwidth...if a queue exceeds its allocation (as calculated
from the CKRM guarantee values), the queue gets skipped.

So this did not use the CFQ priority scheme as such and only implemented
the "limit" part.

The current plan is to exploit the CFQ prio levels and rely on CFQ doing
a good enough job in maintaining an adequate
bandwidth differential between those prio levels.
Again, each queue would maintain a count of its consumed bandwidth as
well as target bandwidth. While picking the next request
from the queue, if its observed that the queue is above its "guarantee",
its priority will get reduced (it'll still supply a request) while
a queue that is below its share will get bumped up....Control will be
much more gradual but the basic idea is to leverage CFQ's priority
handling than supplant it (since we get anticipation in the form of
time-slicing for free).

One concern is whether the time-slicing of CFQ plays well with queues
that aren't organized by tgid...I'm still looking into that.

>Do you have any plots on what is concurrent bandwidth is depending on
>weights? Because, our measurements show that CFQ is not ideal and
>behaves poorly when prio 0,5,6,7 are used :/ Only 1,2,3,4 are really
Interesting. Whats the time-scale over which you expect I/O bandwidth
rates to get enforced ?

Perhaps the iosched discussion should use a different thread....


To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to
More majordomo info at
Please read the FAQ at

 \ /
  Last update: 2006-06-02 23:27    [W:0.171 / U:8.244 seconds]
©2003-2018 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site