lkml.org 
[lkml]   [2011]   [Dec]   [19]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [RFC PATCH 0/4] Gang scheduling in CFS

    * Nikunj A. Dadhania <nikunj@linux.vnet.ibm.com> wrote:

    > The following patches implements gang scheduling. These
    > patches are *highly* experimental in nature and are not
    > proposed for inclusion at this time.
    >
    > Gang scheduling is an approach where we make an effort to
    > run related tasks (the gang) at the same time on a number
    > of CPUs.

    The thing is, the (non-)scalability consequences are awful, gang
    scheduling is a true scalability nightmare. Things like this in
    gang_sched():

    + for_each_domain(cpu_of(rq), sd) {
    + count = 0;
    + for_each_cpu(i, sched_domain_span(sd))
    + count++;

    makes me shudder.

    So could we please approach this from the benchmarked workload
    angle first? The highest improvement is in ebizzy:

    > ebizzy 2vm (improved 15 times, i.e. 1520%)
    > +------------+--------------------+--------------------+----------+
    > | Ebizzy |
    > +------------+--------------------+--------------------+----------+
    > | Parameter | Basline | gang:V2 | % imprv |
    > +------------+--------------------+--------------------+----------+
    > | EbzyRecords| 1709.50 | 27701.00 | 1520 |
    > | EbzyUser| 20.48 | 376.64 | 1739 |
    > | EbzySys| 1384.65 | 1071.40 | 22 |
    > | EbzyReal| 300.00 | 300.00 | 0 |
    > | BwUsage| 2456114173416.00 | 2483447784640.00 | 1 |
    > | HostIdle| 34.00 | 35.00 | -2 |
    > | UsrTime| 6.00 | 14.00 | 133 |
    > | SysTime| 30.00 | 24.00 | 20 |
    > | IOWait| 10.00 | 9.00 | 10 |
    > | IdleTime| 51.00 | 51.00 | 0 |
    > | TPS| 25.00 | 24.00 | -4 |
    > | CacheMisses| 766543805.00 | 8113721819.00 | -958 |
    > | CacheRefs| 9420204706.00 | 136290854100.00 | 1346 |
    > |BranchMisses| 1191336154.00 | 11336436452.00 | -851 |
    > | Branches| 618882621656.00 | 459161727370.00 | -25 |
    > |Instructions| 2517045997661.00 | 2325227247092.00 | 7 |
    > | Cycles| 7642374654922.00 | 7657626973214.00 | 0 |
    > | PageFlt| 23779.00 | 22195.00 | 6 |
    > | ContextSW| 1517241.00 | 1786319.00 | -17 |
    > | CPUMigrat| 537.00 | 241.00 | 55 |
    > +-----------------------------------------------------------------+

    What's behind this huge speedup? Does ebizzy use user-space
    spinlocks perhaps? Could we do something on the user-space side
    to get a similar speedup?

    Thanks,

    Ingo


    \
     
     \ /
      Last update: 2011-12-19 12:27    [W:0.034 / U:29.868 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site