lkml.org 
[lkml]   [2012]   [Mar]   [27]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [RFC][PATCH 00/32] Nohz cpusets v2 (adaptive tickless kernel)
    From
    On Wed, Mar 21, 2012 at 3:58 PM, Frederic Weisbecker <fweisbec@gmail.com> wrote:
    > Hi all,
    >
    > A summary of what this is about can be found here:
    >  https://lkml.org/lkml/2011/8/15/245
    >
    > There are still a lot of things to handle. Especially about
    > what is done by scheduler_tick() but we also need to:
    >
    > - completely handle cputime accounting (need to find every "reader"
    > of cputime and flush cputimes for all of them).
    > -handle  perf
    > - handle irqtime finegrained accounting
    > - handle ilb load balancing
    > - etc...
    >

    I gave the new version a spin (x86 8 way VM) and it looks cool.

    I did get the following warning once, but couldn't recreate it:

    [ 31.812741] ------------[ cut here ]------------
    [ 31.812741] WARNING: at
    /home/giladb/Workspace/linux/kernel/time/tick-sched.c:706
    tick_nohz_account_ticks+0x7c/0x90()
    [ 31.812741] Hardware name: Bochs
    [ 31.812741] Modules linked in:
    [ 31.812741] Pid: 1006, comm: sh Not tainted 3.3.0-rc7+ #167
    [ 31.812741] Call Trace:
    [ 31.812741] [<c102a3ad>] warn_slowpath_common+0x6d/0xa0
    [ 31.812741] [<c106be0c>] ? tick_nohz_account_ticks+0x7c/0x90
    [ 31.812741] [<c106be0c>] ? tick_nohz_account_ticks+0x7c/0x90
    [ 31.812741] [<c102a3fd>] warn_slowpath_null+0x1d/0x20
    [ 31.812741] [<c106be0c>] tick_nohz_account_ticks+0x7c/0x90
    [ 31.812741] [<c106be5f>] tick_nohz_flush_current_times+0x3f/0x80
    [ 31.812741] [<c106bf8d>] tick_nohz_restart_adaptive+0xd/0x30
    [ 31.812741] [<c106c02e>] tick_nohz_check_adaptive+0x3e/0x50
    [ 31.812741] [<c1018180>] smp_cpuset_update_nohz_interrupt+0x20/0x30
    [ 31.812741] [<c1639c6a>] cpuset_update_nohz_interrupt+0x2a/0x30
    [ 31.812741] [<c16395fd>] ? _raw_spin_unlock_irq+0xd/0x30
    [ 31.812741] [<c10575c6>] finish_task_switch+0x46/0xa0
    [ 31.812741] [<c1638558>] __schedule+0x398/0x910
    [ 31.812741] [<c10ef2f1>] ? deactivate_slab+0x611/0x730
    [ 31.812741] [<c1120777>] ? __find_get_block+0x97/0x1a0
    [ 31.812741] [<c1221214>] ? cpumask_next_and+0x24/0xa0
    [ 31.812741] [<c10558cb>] ? get_parent_ip+0xb/0x40
    [ 31.812741] [<c1638b50>] schedule+0x30/0x50
    [ 31.812741] [<c16379b5>] schedule_hrtimeout_range_clock+0xf5/0x110
    [ 31.812741] [<c10558cb>] ? get_parent_ip+0xb/0x40
    [ 31.812741] [<c10586db>] ? sub_preempt_count+0x7b/0xb0
    [ 31.812741] [<c1639633>] ? _raw_spin_unlock_irqrestore+0x13/0x40
    [ 31.812741] [<c1054140>] ? __wake_up+0x40/0x50
    [ 31.812741] [<c1294d1f>] ? put_ldisc+0x3f/0xa0
    [ 31.812741] [<c16379e2>] schedule_hrtimeout_range+0x12/0x20
    [ 31.812741] [<c1107969>] poll_schedule_timeout+0x39/0x60
    [ 31.812741] [<c1108020>] do_sys_poll+0x400/0x490
    [ 31.812741] [<c1054d15>] ? cpuacct_charge+0x65/0x70
    [ 31.812741] [<c1107a20>] ? poll_freewait+0x70/0x70
    [ 31.812741] [<c1107af0>] ? __pollwait+0xd0/0xd0
    [ 31.812741] [<c1107af0>] ? __pollwait+0xd0/0xd0
    [ 31.812741] [<c10094a3>] ? native_sched_clock+0x33/0xe0
    [ 31.812741] [<c105a0e2>] ? sched_clock_local+0xb2/0x190
    [ 31.812741] [<c1054d15>] ? cpuacct_charge+0x65/0x70
    [ 31.812741] [<c105b376>] ? update_curr+0x1a6/0x2a0
    [ 31.812741] [<c105a2f9>] ? sched_clock_cpu+0x139/0x190
    [ 31.812741] [<c105a0e2>] ? sched_clock_local+0xb2/0x190
    [ 31.812741] [<c104dd43>] ? hrtimer_forward+0x163/0x1b0
    [ 31.812741] [<c10644e2>] ? ktime_get+0x62/0x100
    [ 31.812741] [<c1018b56>] ? lapic_next_event+0x16/0x20
    [ 31.812741] [<c1069df2>] ? clockevents_program_event+0xc2/0x170
    [ 31.812741] [<c106b514>] ? tick_program_event+0x24/0x30
    [ 31.812741] [<c104cd1d>] ? hrtimer_interrupt+0x1ad/0x2e0
    [ 31.812741] [<c1095128>] ? rcu_pending+0x58/0x70
    [ 31.812741] [<c1030a3d>] ? irq_exit+0x6d/0x80
    [ 31.812741] [<c1019363>] ? smp_apic_timer_interrupt+0x53/0x90
    [ 31.812741] [<c11e0128>] ? avc_has_perm_noaudit+0xc8/0x360
    [ 31.812741] [<c163a3b6>] ? apic_timer_interrupt+0x2a/0x30
    [ 31.812741] [<c128f31e>] ? tty_ioctl+0x47e/0xa30
    [ 31.812741] [<c11e0d66>] ? inode_has_perm+0x36/0x50
    [ 31.812741] [<c11e13e8>] ? file_has_perm+0xa8/0xb0
    [ 31.812741] [<c128eea0>] ? tty_check_change+0xe0/0xe0
    [ 31.812741] [<c1106763>] ? do_vfs_ioctl+0x83/0x570
    [ 31.812741] [<c11e4e46>] ? selinux_file_ioctl+0x56/0x110
    [ 31.812741] [<c1108224>] sys_poll+0x54/0xb0
    [ 31.812741] [<c1639b29>] syscall_call+0x7/0xb
    [ 31.812741] ---[ end trace 1d7d659b4aead681 ]---

    With the two patches I'll attach to the next replies to this message,
    I've been able to get a task running
    on an isolated CPU with 0 timer interrupts.

    In my case, I also had to disable the clocksource watchdog, but only
    because TSC is not stable on my VM.
    This is really not a nohz/cpuset problem.

    There is one source of interference to cpu isolation this causes,
    which is the cputime flush IPI. Every time you
    run a command in the shell you get 3 - 4 IPIs sent to the nohz cpuset
    to flush the cputimes so that thread group
    times get computed correctly. That's not very nice :-)

    I've tried disabling the IPI send, just to see how it goes and as far
    as I've been able to tell you get bare metal like
    environment for a 100% cpu bound code with no interrupts. Of course.
    ps/top then show 0% cpu utilization for
    that task since without the IPI the times it spends on the CPU is not
    registered... that is a small price to pay
    in my eyes for bare metal performance on Linux, but what do I know? :-)

    Overall, way cool. Please keep it up !

    GIlad

    --
    Gilad Ben-Yossef
    Chief Coffee Drinker
    gilad@benyossef.com
    Israel Cell: +972-52-8260388
    US Cell: +1-973-8260388
    http://benyossef.com

    "If you take a class in large-scale robotics, can you end up in a
    situation where the homework eats your dog?"
     -- Jean-Baptiste Queru
    --
    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to majordomo@vger.kernel.org
    More majordomo info at http://vger.kernel.org/majordomo-info.html
    Please read the FAQ at http://www.tux.org/lkml/

    \
     
     \ /
      Last update: 2012-03-27 17:05    [W:0.045 / U:60.304 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site