lkml.org 
[lkml]   [2005]   [Jun]   [11]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectPREEMPT_RT vs ADEOS: the numbers, part 1
    For the past few weeks, we've been conducting comparison tests between
    PREEMPT_RT and the Adeos nanokernel. As was clear from previous discussion,
    we've been open to be proven wrong regarding endorsement of either method.
    Hence, this comparison was done in order to better understand the impact
    of each method vis-a-vis vanilla Linux.

    At this time, we are publishing the summary results of the various test
    runs we've conducted. There is, of course, a lot of background information
    that needs to be provided (.configs, scripts, drivers, etc.) We will be
    making those available sometime early next week. In the mean time, the
    following is meant as food-for-thought.

    In our tests, we've used a set up with 3 machines. The two main systems
    were Dell PowerEdge SC420 machines with a P4 2.8 (UP not SMP configured) with
    FC3. One had 256 MB RAM and was the guinea pig (i.e. the machine controlling
    the mechanical saw a.k.a. TARGET) The other, having 512 MB RAM, was used to
    collect data regarding the guinea pig's responsiveness (a.k.a LOGGER.) The
    third machine, an Apple PowerBook 5,6 G4 / 1GB, was used for a dual purpose.
    First, it controlled both the target and the logger via ssh, and was also
    used to ping flood the target. This 3rd system is known as the HOST.

    Data was generated on all three systems:
    TARGET: LMbench data
    LOGGER: Interrupt response time
    HOST: LMbench total running time

    Both the host and the logger had a constant kernel configuration. The logger
    was running an adeos-enabled kernel in order to trigger and deterministically
    measure the responsiveness of the target. The host was running a plain
    gentoo-based kernel. The target and the logger were rigged via their parallel
    ports so that an output from the logger would trigger an interrupt on the
    target who's response would itself trigger an interrupt on the logger.

    In the various test runs, we've attempted to collect two sets of data. One
    regarding LMbench's total running time for a given set up and the other
    regarding the system's interrupt response time. Where appropriate, both tests
    were conducted simultaneously. Otherwise, they were conducted in isolation.
    The following tables should be self-explanatory.

    For LMbench test runs, 3 passes were conducted and an average running time
    was collected. Certainly, 3 passes is not as much as we'd like, but for the
    immediate purposes, it provides a sufficiently corroborated data set for
    analysis (as can be seen in the following tables.)

    For the interrupt response time measurement, the logger generated between
    500,000 to 650,000 interrupts and measured the target's response time. The
    logger was not subject to any load whatsoever, except that imposed by the
    logging driver (running in a prioritary Adeos domain, and hence being truly
    hard-rt, a.k.a. "ruby" hard) and that of the user-space daemon committing the
    data to storage. It could be argued that the use of Adeos imposes a penalty
    to the measured response time. However, this penalty is imposed on all data
    sets, and verification of its impact can be inferred by analyzing the adeos-to-
    adeos set up provided below.

    With no further ado, here are the results we've obtained. As we said above, we
    will be making all related scripts, patches, and drivers available, so that
    others may conduct their own tests.

    Note that in the tests we've conducted, we've tried, in as much possible, to
    use similar kernels. However, we were unable to find a recent Adeos and a
    recent PREEMPT_RT patch which would both cleanly apply to a same recent kernel.
    Hence, we've compared vanilla 2.6.12-rc2 with an adeos-patched one and a
    2.6.12-rc4 with a PREEMPT_RT-patched one. As can be seen below, the runs of
    the vanilla rc2 and rc4 yield very similar numbers, and can therefore
    reasonably be considered equivalent.

    LMbench running times:
    +--------------------+--------+----------+------------+------------+----------+
    | Kernel | plain | IRQ test | ping flood | IRQ & ping | IRQ & hd |
    +====================+========+==========+============+============+==========+
    | Vanilla-2.6.12-rc2 | 174 s | 175 s | 189 s | 193 s | 217 s |
    +--------------------+--------+----------+------------+------------+----------+
    | with Adeos-r10c3 | 180 s | 180 s | 185 s | 183 s | 211 s |
    +--------------------+--------+----------+------------+------------+----------+
    | % | +3.4 | +2.9 | -2.1 | -5.2 | -2.8 |
    +====================+========+==========+============+============+==========+
    | Vanilla-2.6.12-rc4 | 176 s | 177 s | 189 s | 191 s | 218 s |
    +--------------------+--------+----------+------------+------------+----------+
    | with RT-V0.7.47-08 | 184 s | 187 s | 206 s | 201 s | 225 s |
    +--------------------+--------+----------+------------+------------+----------+
    | % | +4.5 | +5.6 | +9.0 | +5.2 | +3.2 |
    +--------------------+--------+----------+------------+------------+----------+

    Legend:
    plain = Nothing special
    IRQ test = on logger: triggering target every 1ms
    ping flood = on host: "sudo ping -f $TARGET_IP_ADDR"
    IRQ & ping = combination of the previous two
    IRQ & hd = IRQ test with the following being done on the target:
    "while [ true ]
    do dd if=/dev/zero of=/tmp/dummy count=512
    done"

    In the following, interrupts are triggered by the logger at every 1ms. It would
    be interesting to redo such tests with shorter trigger times. However, we
    wanted to keep the logger as "off-the-shelf" as possible.

    Interrupt response times (all in micro-seconds):
    +--------------------+------------+------+-------+------+--------+
    | Kernel | sys load | Aver | Max | Min | StdDev |
    +====================+============+======+=======+======+========+
    | | None | 15.5 | 64.8 | 15.2 | 1.0 |
    | | Ping | 15.7 | 63.4 | 15.2 | 1.2 |
    | Vanilla-2.6.12-rc2 | lm. + ping | 16.0 | 72.2 | 15.2 | 1.4 |
    | | lmbench | 15.8 | 65.6 | 15.2 | 1.1 |
    | | lm. + hd | 15.8 | 179.9 | 15.2 | 1.3 |
    +--------------------+------------+------+-------+------+--------+
    | | None | 13.4 | 53.3 | 13.2 | 0.2 |
    | | Ping | 13.8 | 53.3 | 13.3 | 0.6 |
    | with Adeos-r10c3 | lm.+ ping | 13.9 | 21.8 | 13.2 | 0.7 |
    | | lmbench | 13.9 | 21.3 | 13.3 | 0.6 |
    | | lm. + hd | 13.9 | 53.2 | 13.2 | 0.5 |
    +====================+============+======+=======+======+========+
    | | None | 15.2 | 64.2 | 15.2 | 0.5 |
    | | Ping | 15.6 | 63.0 | 15.2 | 0.9 |
    | Vanilla-2.6.12-rc4 | lm. + ping | 16.0 | 170.5 | 16.0 | 1.4 |
    | | lmbench | 15.8 | 184.1 | 15.2 | 1.2 |
    | | lm. + hd | 15.8 | 67.1 | 15.0 | 1.1 |
    +--------------------+------------+------+-------+------+--------+
    | | None | 15.5 | 73.8 | 15.2 | 1.2 |
    | | Ping | 17.1 | 79.8 | 15.2 | 2.3 |
    | with RT-V0.7.47-08 | lm. + ping | 17.7 | 77.2 | 15.2 | 3.1 |
    | | lmbench | 17.1 | 80.0 | 15.3 | 2.3 |
    | | lm. + hd | 17.0 | 80.0 | 15.3 | 1.8 |
    +--------------------+------------+------+-------+------+--------+

    Legend:
    None = nothing special
    ping = on host: "sudo ping -f $TARGET_IP_ADDR"
    lm. + ping = previous test and "make rerun" in lmbench-2.0.4/src/ on target
    lmbench = "make rerun" in lmbench-2.0.4/src/ on target
    lm. + hd = previous test with the following being done on the target:
    "while [ true ]
    do dd if=/dev/zero of=/tmp/dummy count=512
    done"

    Note: Adeos-r10c3 is a "combo" patch including both Adeos and PREEMPT_RT, though
    PREEMPT_RT is disabled.

    The above data has been provided as-is without any analysis for now. We will
    provide such analysis when publishing the complete data sets and related
    software. In the mean time, we hope such results will help further reflection.

    Best regards and have fun !!!

    Kristian Benoit
    Karim Yaghmour
    --
    Pushing Embedded and Real-Time Linux Systems Beyond the Limits
    http://www.opersys.com || 1-866-677-4546


    -
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2005-06-11 06:43    [W:0.090 / U:0.000 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site