lkml.org 
[lkml]   [2008]   [Nov]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [Bug #11308] tbench regression on each kernel release from 2.6.22 -> 2.6.28

    * Ingo Molnar <mingo@elte.hu> wrote:

    4> The place for the sock_rfree() hit looks a bit weird, and i'll
    > investigate it now a bit more to place the real overhead point
    > properly. (i already mapped the test-bit overhead: that comes from
    > napi_disable_pending())

    ok, here's a new set of profiles. (again for tbench 64-thread on a
    16-way box, with v2.6.28-rc5-19-ge14c8bf and with the kernel config i
    posted before.)

    Here are the per major subsystem percentages:

    NET overhead ( 5786945/10096751): 57.31%
    security overhead ( 925933/10096751): 9.17%
    usercopy overhead ( 837887/10096751): 8.30%
    sched overhead ( 753662/10096751): 7.46%
    syscall overhead ( 268809/10096751): 2.66%
    IRQ overhead ( 266500/10096751): 2.64%
    slab overhead ( 180258/10096751): 1.79%
    timer overhead ( 92986/10096751): 0.92%
    pagealloc overhead ( 87381/10096751): 0.87%
    VFS overhead ( 53295/10096751): 0.53%
    PID overhead ( 44469/10096751): 0.44%
    pagecache overhead ( 33452/10096751): 0.33%
    gtod overhead ( 11064/10096751): 0.11%
    IDLE overhead ( 0/10096751): 0.00%
    ---------------------------------------------------------
    left ( 753878/10096751): 7.47%

    The breakdown is very similar to what i sent before, within noise.

    [ 'left' is random overhead from all around the place - i categorized
    the 500 most expensive functions in the profile per subsystem.
    I stopped short of doing it for all 1300+ functions: it's rather
    laborous manual work even with hefty use of regex patterns.
    It's also less meaningful in practice: the trend in the first 500
    functions is present in the remaining 800 functions as well. I
    watched the breakdown evolve as i increased the coverage - in
    practice it is the first 100 functions that matter - it just doesnt
    change after that. ]

    The readprofile output below seems structured in a more useful way now
    - i tweaked compiler options to have the profiler hits spread out in a
    more meaningful way. I collected 10 million NMI profiler hits, and
    normalized the readprofile output up to 100%.

    [ I'll post per function analysis as i complete them, as a reply to
    this mail. ]

    Ingo

    100.000000 total
    ................
    7.253355 copy_user_generic_string
    3.934833 avc_has_perm_noaudit
    3.356152 ip_queue_xmit
    3.038025 skb_release_data
    2.118525 skb_release_head_state
    1.997533 tcp_ack
    1.833688 tcp_recvmsg
    1.717771 eth_type_trans
    1.673249 __inet_lookup_established
    1.508888 system_call
    1.469183 tcp_current_mss
    1.431553 tcp_transmit_skb
    1.385125 tcp_sendmsg
    1.327643 tcp_v4_rcv
    1.292328 nf_hook_thresh
    1.203205 schedule
    1.059501 nf_hook_slow
    1.027373 constant_test_bit
    0.945183 sock_rfree
    0.922748 __switch_to
    0.911605 netif_rx
    0.876270 register_gifconf
    0.788200 ip_local_deliver_finish
    0.781467 dev_queue_xmit
    0.766530 constant_test_bit
    0.758208 _local_bh_enable_ip
    0.747184 load_cr3
    0.704341 memset_c
    0.671260 sysret_check
    0.651845 ip_finish_output2
    0.620204 audit_free_names
    0.617781 audit_syscall_exit
    0.615149 skb_copy_datagram_iovec
    0.613848 selinux_socket_sock_rcv_skb
    0.606995 constant_test_bit
    0.593936 __tcp_push_pending_frames
    0.592198 tcp_cleanup_rbuf
    0.574093 ip_rcv
    0.567886 netif_receive_skb
    0.563377 get_page_from_freelist
    0.557657 tcp_event_data_recv
    0.539274 ip_local_deliver
    0.534130 sys_recvfrom
    0.512321 __tcp_select_window
    0.498427 tcp_rcv_established
    0.494862 sys_sendto
    0.487473 audit_syscall_entry
    0.478495 sched_clock_cpu
    0.474861 kfree
    0.466310 tcp_established_options
    0.461384 net_rx_action
    0.447162 __mod_timer
    0.442078 ip_rcv_finish
    0.441631 find_pid_ns
    0.441124 sk_wait_data
    0.423943 __sock_recvmsg
    0.422126 selinux_parse_skb
    0.417975 __napi_schedule
    0.414082 __do_softirq
    0.403604 task_rq_lock
    0.380792 nf_iterate
    0.377614 select_task_rq_fair
    0.374973 sock_sendmsg
    0.374635 kmem_cache_alloc_node
    0.368775 avc_has_perm
    0.368706 local_bh_disable
    0.361834 release_sock
    0.346400 sock_common_recvmsg
    0.342825 skb_clone
    0.338704 __alloc_skb
    0.326488 do_softirq
    0.323410 lock_sock_nested
    0.322129 __copy_skb_header
    0.316835 put_page
    0.310966 selinux_ip_postroute
    0.306229 sel_netport_sid
    0.299863 try_to_wake_up
    0.296288 process_backlog
    0.294818 __inet_lookup
    0.294778 thread_return
    0.293219 cfs_rq_of
    0.292315 internal_add_timer
    0.292305 tcp_rcv_space_adjust
    0.281053 constant_test_bit
    0.278779 local_bh_enable
    0.272910 *unknown*
    0.269593 schedule_timeout
    0.261846 tcp_v4_md5_lookup
    0.260992 __ip_local_out
    0.255868 __enqueue_entity
    0.253931 avc_audit
    0.252004 finish_task_switch
    0.249263 audit_get_context
    0.248290 sockfd_lookup_light
    0.247416 virt_to_head_page
    0.244149 tcp_options_write
    0.243603 memcpy_toiovec
    0.243434 sock_recvmsg
    0.242599 call_softirq
    0.242391 __unlazy_fpu
    0.236412 fput_light
    0.235628 ret_from_sys_call
    0.234933 sk_reset_timer
    0.228358 math_state_restore
    0.227117 socket_has_perm
    0.223492 virt_to_cache
    0.219063 __cache_free
    0.216401 update_curr
    0.216232 tcp_v4_send_check
    0.213978 audit_free_aux
    0.213223 tcp_v4_do_rcv
    0.212975 __kfree_skb
    0.211137 dev_hard_start_xmit
    0.209052 tcp_rtt_estimator
    0.207999 netif_needs_gso
    0.207662 __update_sched_clock
    0.207284 rb_erase
    0.204861 enqueue_task_fair
    0.203490 skb_release_all
    0.203252 tcp_send_delayed_ack
    0.203232 inet_ehashfn
    0.199846 sel_netport_find
    0.195396 system_call_after_swapgs
    0.186756 lock_timer_base
    0.186687 pick_next_task_fair
    0.183986 mod_timer
    0.182982 loopback_xmit
    0.182605 native_read_tsc
    0.181195 skb_set_owner_r
    0.179248 switch_mm
    0.175584 set_next_entity
    0.173329 raw_local_deliver
    0.171641 sys_kill
    0.164510 dequeue_task_fair
    0.161938 clear_bit
    0.160528 sock_def_readable
    0.157628 __tcp_ack_snd_check
    0.156893 skb_can_coalesce
    0.156556 tcp_snd_wnd_test
    0.155662 ip_output
    0.150627 sk_stream_alloc_skb
    0.150219 cpu_sdc
    0.149425 sysret_careful
    0.148760 tcp_data_snd_check
    0.147816 auditsys
    0.147419 pskb_may_pull
    0.147151 fget_light
    0.143774 tcp_cwnd_test
    0.143029 rb_insert_color
    0.142265 __wake_up
    0.141808 tcp_bound_to_half_wnd
    0.138600 __sk_dst_check
    0.138431 free_hot_cold_page
    0.137954 unroll_tree_refs
    0.137080 __skb_unlink
    0.135124 __sock_sendmsg
    0.135064 get_pageblock_flags_group
    0.132701 kmem_cache_free
    0.128152 bictcp_cong_avoid
    0.127874 __napi_complete
    0.127527 ____cache_alloc
    0.127368 tcp_is_cwnd_limited
    0.127278 find_vpid
    0.126941 constant_test_bit
    0.126504 sk_mem_charge
    0.126255 __alloc_pages_internal
    0.125977 dst_release
    0.125521 hash_64
    0.124895 put_prev_task_fair
    0.123802 netlbl_enabled
    0.122829 sched_clock
    0.122640 skb_push
    0.122035 __phys_addr
    0.121161 dput
    0.120515 tcp_prequeue_process
    0.118916 __skb_dequeue
    0.117715 selinux_socket_sendmsg
    0.117536 __inc_zone_state
    0.115907 sk_wake_async
    0.113504 selinux_ipv4_output
    0.113017 sel_netif_sid
    0.112431 skb_reset_network_header
    0.111170 check_preempt_wakeup
    0.111061 bictcp_acked
    0.110882 sel_netnode_find
    0.109978 update_min_vruntime
    0.109889 resched_task
    0.109879 current_kernel_time
    0.109432 tcp_checksum_complete_user
    0.107476 ip_dont_fragment
    0.107386 sysret_audit
    0.106979 inet_csk_reset_xmit_timer
    0.106006 skb_entail
    0.105777 sysret_signal
    0.105420 avc_hash
    0.105251 __skb_clone
    0.105211 tcp_init_tso_segs
    0.103523 __dequeue_entity
    0.101715 PageLRU
    0.101378 tcp_parse_aligned_timestamp
    0.101219 __xchg
    0.100544 constant_test_bit
    0.097991 __kmalloc
    0.097584 test_tsk_thread_flag
    0.097475 autoremove_wake_function
    0.095747 selinux_task_kill
    0.094416 get_page
    0.093353 dequeue_task
    0.092728 __local_bh_disable
    0.091943 selinux_netlbl_sock_rcv_skb
    0.091655 path_put
    0.090970 skb_headroom
    0.090950 PageTail
    0.090642 dst_destroy
    0.090523 netpoll_rx
    0.089589 skb_header_pointer
    0.085935 security_socket_recvmsg
    0.084008 alloc_pages_current
    0.083184 compare_ether_addr
    0.082479 rb_next
    0.082439 sk_wmem_schedule
    0.081635 next_zones_zonelist
    0.080135 tcp_cwnd_validate
    0.079877 tcp_event_new_data_sent
    0.079817 fcheck_files
    0.079082 ip_skb_dst_mtu
    0.078804 ip_finish_output
    0.078278 wakeup_preempt_entity
    0.077026 sel_netif_find
    0.076788 __skb_queue_tail
    0.076570 sock_flag
    0.076520 tcp_win_from_space
    0.076510 zone_watermark_ok
    0.076282 sel_netnode_sid
    0.076162 policy_zonelist
    0.074732 __wake_up_common
    0.074613 compound_head
    0.074593 task_has_perm
    0.073243 __find_general_cachep
    0.073064 tcp_push
    0.072925 skb_cloned
    0.072309 pskb_may_pull
    0.071852 TCP_ECN_check_ce
    0.071495 cap_task_to_inode
    0.070770 default_wake_function
    0.069429 xfrm4_policy_check
    0.069091 tcp_parse_md5sig_option
    0.068287 tcp_v4_md5_do_lookup
    0.068059 tcp_v4_tw_remember_stamp
    0.067344 tcp_ca_event
    0.067125 tcp_ca_event
    0.065457 place_entity
    0.065318 write_seqlock
    0.065089 device_not_available
    0.065069 test_ti_thread_flag
    0.063878 tcp_set_skb_tso_segs
    0.063550 selinux_netlbl_inode_permission
    0.063391 sock_wfree
    0.063311 prepare_to_wait
    0.058872 pid_vnr
    0.058803 __cycles_2_ns
    0.057631 ip_local_out
    0.057333 tcp_ack_saw_tstamp
    0.056896 copy_to_user
    0.056628 set_bit
    0.055913 free_pages_check
    0.054969 tcp_rcv_rtt_measure_ts
    0.053797 init_rootdomain
    0.053708 selinux_socket_recvmsg
    0.053698 pid_nr_ns
    0.053629 sk_eat_skb
    0.052814 _local_bh_enable
    0.052645 nf_hook_thresh
    0.052516 sched_info_queued
    0.052457 enqueue_task
    0.052228 sk_filter
    0.052159 __cpu_clear
    0.051980 local_bh_enable_ip
    0.050292 update_rq_clock
    0.048981 task_tgid_vnr
    0.048881 copy_from_user
    0.048782 tcp_parse_options
    0.048484 lock_sock
    0.047779 net_timestamp
    0.047044 open_softirq
    0.046955 tcp_win_from_space
    0.045981 __skb_dequeue
    0.043846 getboottime
    0.043777 account_group_exec_runtime
    0.043519 can_checksum_protocol
    0.043469 set_user_nice
    0.042784 skb_fill_page_desc
    0.042247 security_socket_sendmsg
    0.041989 read_profile
    0.041930 tcp_validate_incoming
    0.041612 check_preempt_curr
    0.041413 skb_pull
    0.041026 generic_smp_call_function_interrupt
    0.041016 calc_delta_fair
    0.040936 clear_buddies
    0.040768 tcp_data_queue
    0.040698 page_count
    0.039695 lock_sock
    0.039099 skb_headroom
    0.038851 system_call_fastpath
    0.038622 zone_statistics
    0.037500 tcp_sack_extend
    0.037381 __kmalloc_node
    0.036587 first_zones_zonelist
    0.036497 mntput
    0.036179 pick_next_task
    0.035991 kmap
    0.035911 sock_put
    0.035613 deactivate_task
    0.035027 __nr_to_section
    0.033985 page_zone
    0.033190 native_load_tls
    0.032882 netif_tx_queue_stopped
    0.032713 __skb_insert
    0.032187 sock_flag
    0.031988 check_kill_permission
    0.031790 policy_nodemask
    0.031621 detach_timer
    0.030558 inet_csk_clear_xmit_timer
    0.030469 task_rq_unlock
    0.029883 tcp_nagle_test
    0.029744 tracesys
    0.028383 virt_to_slab
    0.028115 tcp_v4_check
    0.028046 __cpu_set
    0.027658 page_get_cache
    0.027063 tcp_store_ts_recent
    0.027053 __skb_pull
    0.026953 gfp_zone
    0.026586 sock_rcvlowat
    0.026576 csum_partial
    0.026397 init_waitqueue_head
    0.026109 finish_wait
    0.026040 kill_pid_info
    0.025404 tcp_full_space
    0.024888 __skb_queue_before
    0.024550 dst_confirm
    0.022603 inet_ehash_bucket
    0.021888 activate_task
    0.021650 tcp_rto_min
    0.021283 d_callback
    0.020965 signal_pending
    0.020925 avc_node_free
    0.020915 empty_bucket
    0.020746 group_send_sig_info
    0.020657 skb_reset_transport_header
    0.020061 sock_put
    0.019992 signal_pending_state
    0.019684 tcp_sync_mss
    0.019346 skb_network_offset
    0.019276 skb_split
    0.018988 tcp_adjust_fackets_out
    0.018204 tcp_fast_path_check
    0.017727 __skb_unlink
    0.017687 napi_disable_pending
    0.017678 sg_set_page
    0.017022 get_pageblock_bitmap
    0.016972 tcp_cong_avoid
    0.016962 pid_task
    0.016754 skb_set_tail_pointer
    0.016039 selinux_ipv4_postroute
    0.015930 idle_cpu
    0.015632 skb_reset_network_header
    0.015552 __count_vm_events
    0.015483 source_load
    0.014867 __skb_unlink
    0.014738 skb_reset_transport_header
    0.014599 set_bit
    0.014241 audit_zero_context
    0.014231 zone_page_state
    0.014152 clear_bit
    0.013874 PageSlab
    0.013546 __memset
    0.013238 get_pageblock_migratetype
    0.012623 __rb_rotate_right
    0.012543 kmem_find_general_cachep
    0.012414 __kprobes_text_start
    0.012344 security_sock_rcv_skb
    0.012344 node_zonelist
    0.012335 dnotify_parent
    0.012096 skb_headroom
    0.011778 tcp_push_one
    0.011540 mnt_want_write
    0.011143 kmalloc
    0.011073 retint_swapgs
    0.010954 __rb_rotate_left
    0.010805 check_pgd_range
    0.010785 tcp_mss_split_point
    0.010755 migrate_timer_list
    0.010338 __send_IPI_dest_field
    0.010229 reschedule_interrupt
    0.010179 sock_flag
    0.009882 smp_call_function_mask
    0.009673 test_tsk_need_resched
    0.009564 tcp_urg
    0.009504 generic_file_aio_read
    0.009176 PageReserved
    0.009147 net_invalid_timestamp
    0.009087 __node_set
    0.008749 do_tcp_setsockopt
    0.008730 set_tsk_thread_flag
    0.008720 tcp_enter_loss
    0.008422 sock_error
    0.008362 target_load
    0.008302 crypto_hash_update
    0.008104 PageReadahead
    0.008044 tcp_poll
    0.007915 tcp_checksum_complete
    0.007329 tcp_snd_test
    0.007309 selinux_file_permission
    0.007290 sel_netif_destroy
    0.007220 put_pages_list
    0.006992 dst_output
    0.006743 prepare_to_copy
    0.006694 tcp_init_cwnd
    0.006555 clear_bit
    0.006535 set_bit
    0.006425 normal_prio
    0.006366 msleep
    0.006346 error_sti
    0.006336 tcp_rcv_rtt_update
    0.006167 tcp_send_ack
    0.005989 tcp_init_nondata_skb
    0.005720 kfree_skb
    0.005502 call_function_interrupt
    0.005413 __count_vm_event
    0.005403 __skb_checksum_complete_head
    0.005363 page_cache_get_speculative
    0.005323 dev_kfree_skb_irq
    0.005174 skb_store_bits
    0.004956 cpu_avg_load_per_task
    0.004916 dev_cpu_callback
    0.004807 __kmem_cache_destroy
    0.004777 tcp_init_metrics
    0.004777 io_schedule
    0.004777 find_get_page
    0.004707 eth_header_parse
    0.004688 cap_task_kill
    0.004678 error_exit
    0.004668 rb_prev
    0.004658 tso_fragment
    0.004648 mmdrop
    0.004628 skb_reset_tail_pointer
    0.004598 apic_timer_interrupt
    0.004588 clear_bit
    0.004519 tcp_simple_retransmit
    0.004449 get_max_files
    0.004370 sk_stop_timer
    0.004340 tcp_reset
    0.004251 netlbl_cache_add
    0.004201 tcp_add_reno_sack
    0.004151 __pskb_trim_head
    0.004102 __profile_flip_buffers
    0.004092 sk_common_release
    0.004052 audit_copy_inode
    0.003953 eth_change_mtu
    0.003943 vfs_read
    0.003923 run_timer_softirq
    0.003843 mnt_drop_write
    0.003814 clear_page_c
    0.003804 do_sync_read
    0.003744 unset_migratetype_isolate
    0.003714 sk_stream_moderate_sndbuf
    0.003545 tcp_try_rmem_schedule
    0.003476 native_apic_mem_write
    0.003466 sys_read
    0.003446 skb_checksum
    0.003436 timer_set_base
    0.003426 security_task_kill
    0.003416 __flow_cache_shrink
    0.003406 __skb_checksum_complete
    0.003277 alloc_skb
    0.003267 physflat_send_IPI_mask
    0.003218 skb_gso_ok
    0.003178 constant_test_bit
    0.003168 find_next_bit
    0.003158 selinux_netlbl_skbuff_getsid
    0.003118 constant_test_bit
    0.003099 pull_task
    0.003079 hrtimer_run_queues
    0.003049 free_hot_page
    0.003009 scheduler_tick
    0.002900 set_32bit_tls
    0.002890 tcp_acceptable_seq
    0.002811 rw_verify_area
    0.002751 radix_tree_lookup_slot
    0.002731 zero_user_segment
    0.002731 sock_common_setsockopt
    0.002612 __load_balance_iterator
    0.002473 run_posix_cpu_timers
    0.002264 task_utime
    0.002254 switched_to_fair
    0.002185 fsnotify_access
    0.002145 __rmqueue_smallest
    0.002125 __schedule_bug
    0.002095 __task_rq_lock
    0.002086 tcp_may_update_window
    0.002076 restore_args
    0.002066 hrtimer_run_pending
    0.002056 generic_segment_checks
    0.002026 getnstimeofday
    0.002006 idle_task
    0.001976 touch_atime
    0.001956 __wake_up_locked
    0.001927 sk_mem_charge
    0.001877 smp_apic_timer_interrupt
    0.001827 native_smp_send_reschedule
    0.001798 __tcp_fast_path_on
    0.001788 file_read_actor
    0.001768 _cond_resched
    0.001738 avc_policy_seqno
    0.001718 tcp_ack_snd_check
    0.001629 ip_send_check
    0.001619 account_system_time
    0.001579 __xapic_wait_icr_idle
    0.001579 get_stats
    0.001539 tcp_set_state
    0.001539 bictcp_state
    0.001529 tcp_fast_path_on
    0.001519 file_accessed
    0.001480 get_seconds
    0.001450 kernel_math_error
    0.001410 ktime_set
    0.001331 kmap_atomic
    0.001281 printk_tick
    0.001281 __next_cpu_nr
    0.001271 account_group_system_time
    0.001261 __mod_zone_page_state
    0.001222 weighted_cpuload
    0.001192 security_file_permission
    0.001162 ack_APIC_irq
    0.001152 __free_one_page
    0.001142 rcu_pending
    0.001142 drain_array
    0.001122 sched_clock_tick
    0.001122 csum_fold
    0.001102 ret_from_intr
    0.001083 retint_careful
    0.001073 need_resched
    0.001073 calc_delta_mine
    0.001043 tcp_v4_md5_do_del
    0.001043 PageActive
    0.001033 mark_page_accessed
    0.001033 ktime_get_ts
    0.001023 tcp_insert_write_queue_after
    0.001013 tcp_delack_timer
    0.001013 task_tick_fair
    0.000973 delay_tsc
    0.000963 nv_nic_irq_optimized
    0.000904 tick_periodic
    0.000894 skb_reserve
    0.000884 cache_reap
    0.000874 timespec_trunc
    0.000864 skb_header_release
    0.000854 zone_page_state_add
    0.000844 update_process_times
    0.000834 sk_rmem_schedule
    0.000824 find_busiest_group
    0.000804 current_fs_time
    0.000785 tick_handle_periodic
    0.000785 __sk_mem_schedule
    0.000785 irq_enter
    0.000755 use_cpu_writer_for_mount
    0.000755 tcp_ratehalving_spur_to_response
    0.000745 update_wall_time
    0.000745 tcp_sendpage
    0.000745 __alloc_pages_nodemask
    0.000725 ktime_get
    0.000725 irq_exit
    0.000705 inotify_inode_queue_event
    0.000665 set_pageblock_flags_group
    0.000646 inotify_dentry_parent_queue_event
    0.000626 ack_APIC_irq
    0.000606 write_profile
    0.000566 set_normalized_timespec
    0.000566 raise_softirq
    0.000526 task_cputime_zero
    0.000516 smp_reschedule_interrupt
    0.000516 __skb_insert
    0.000497 page_fault
    0.000497 __copy_user_nocache
    0.000487 run_local_timers
    0.000487 read_tsc
    0.000487 nf_unregister_hook
    0.000477 __rcu_pending
    0.000477 jiffies_to_usecs
    0.000457 timespec_to_ktime
    0.000437 __skb_trim
    0.000427 __call_rcu
    0.000417 free_pages_bulk
    0.000407 smp_call_function_interrupt
    0.000397 set_irq_regs
    0.000397 radix_tree_deref_slot
    0.000397 expand
    0.000387 handle_mm_fault
    0.000387 handle_IRQ_event
    0.000387 fput_light
    0.000377 refresh_cpu_vm_stats
    0.000377 n_tty_write
    0.000367 get_page
    0.000358 run_rebalance_domains
    0.000358 get_cpu_mask
    0.000348 task_hot
    0.000348 __skb_queue_after
    0.000348 retint_check
    0.000348 do_select
    0.000338 PageUptodate
    0.000338 copy_page_c
    0.000328 cond_resched
    0.000318 unmap_vmas
    0.000318 sk_mem_reclaim
    0.000318 rmqueue_bulk
    0.000318 reciprocal_value
    0.000318 irq_return
    0.000308 rb_first
    0.000308 alloc_skb
    0.000308 account_process_tick
    0.000298 net_enable_timestamp
    0.000298 clocksource_read
    0.000298 account_system_time_scaled
    0.000288 sched_slice
    0.000278 ip_compute_csum
    0.000278 constant_test_bit
    0.000278 constant_test_bit
    0.000268 set_curr_task_fair
    0.000268 note_interrupt
    0.000268 exit_idle
    0.000258 native_apic_mem_write
    0.000258 exit_intr
    0.000248 PageReferenced
    0.000238 usb_hcd_irq
    0.000238 __mnt_is_readonly
    0.000238 constant_test_bit
    0.000218 IRQ0xba_interrupt
    0.000218 handle_fasteoi_irq
    0.000209 raise_softirq_irqoff
    0.000209 __find_get_block
    0.000199 tcp_current_ssthresh
    0.000199 n_tty_receive_buf
    0.000189 wake_up_page
    0.000189 vgacon_save_screen
    0.000189 free_block
    0.000189 constant_test_bit
    0.000179 pagefault_disable
    0.000169 clocksource_get_next
    0.000169 __bitmap_weight
    0.000159 tty_ldisc_deref
    0.000159 tcp_write_timer
    0.000159 kmem_cache_alloc
    0.000159 free_alien_cache
    0.000159 ext3_mark_iloc_dirty
    0.000159 constant_test_bit
    0.000159 __bitmap_equal
    0.000149 transfer_objects
    0.000149 __rcu_process_callbacks
    0.000149 page_waitqueue
    0.000149 constant_test_bit
    0.000139 __rmqueue
    0.000139 release_pages
    0.000139 constant_test_bit
    0.000129 __tcp_checksum_complete
    0.000129 run_workqueue
    0.000129 poll_freewait
    0.000129 n_tty_read
    0.000129 iommu_area_free
    0.000129 generic_file_llseek
    0.000129 __cpus_setall
    0.000129 cond_resched_softirq
    0.000129 avc_node_populate
    0.000129 add_to_page_cache_lru
    0.000129 account_user_time
    0.000119 wait_consider_task
    0.000119 sys_select
    0.000119 round_jiffies_common
    0.000119 nv_start_xmit_optimized
    0.000119 core_sys_select
    0.000109 tcp_tso_segment
    0.000109 sigprocmask
    0.000109 proc_reg_read
    0.000109 path_to_nameidata
    0.000109 PageBuddy
    0.000109 ohci_irq
    0.000109 nv_tx_done_optimized
    0.000109 nv_msi_workaround
    0.000109 IRQ0xc2_interrupt
    0.000109 __ext3_get_inode_loc
    0.000109 account_group_user_time
    0.000099 __wake_up_sync
    0.000099 __up_read
    0.000099 update_vsyscall
    0.000099 memmove
    0.000099 kmalloc
    0.000099 ext3_get_blocks_handle
    0.000099 do_device_not_available
    0.000099 constant_test_bit
    0.000089 tcp_incr_quickack
    0.000089 smp_send_reschedule
    0.000089 remove_from_page_cache
    0.000089 rcu_process_callbacks
    0.000089 prepare_to_wait_exclusive
    0.000089 pde_users_dec
    0.000089 find_first_bit
    0.000089 constant_test_bit
    0.000089 common_interrupt
    0.000089 add_wait_queue
    0.000079 task_gtime
    0.000079 sys_lseek
    0.000079 start_this_handle
    0.000079 schedule_hrtimeout_range
    0.000079 __sched_fork
    0.000079 journal_put_journal_head
    0.000079 find_first_zero_bit
    0.000079 do_syslog
    0.000079 do_sync_write
    0.000079 constant_test_bit
    0.000079 ack_apic_level
    0.000070 write_seqlock
    0.000070 slab_get_obj
    0.000070 remove_wait_queue
    0.000070 pty_chars_in_buffer
    0.000070 ____pagevec_lru_add
    0.000070 lock_hrtimer_base
    0.000070 kstat_incr_irqs_this_cpu
    0.000070 journal_dirty_data
    0.000070 journal_add_journal_head
    0.000070 find_lock_page
    0.000070 copy_from_read_buf
    0.000070 bit_waitqueue
    0.000070 alloc_page_vma
    0.000060 vfs_write
    0.000060 tty_write
    0.000060 __strnlen_user
    0.000060 sk_mem_uncharge
    0.000060 rt_worker_func
    0.000060 radix_tree_preload
    0.000060 poll_select_copy_remaining
    0.000060 pagefault_enable
    0.000060 __mark_inode_dirty
    0.000060 lru_add_drain_all
    0.000060 lock_page
    0.000060 list_replace_init
    0.000060 journal_stop
    0.000060 iowrite8
    0.000060 hrtimer_forward
    0.000060 gart_unmap_single
    0.000060 find_vma
    0.000060 __down_read_trylock
    0.000060 do_page_fault
    0.000060 do_IRQ
    0.000060 create_empty_buffers
    0.000060 constant_test_bit
    0.000060 constant_test_bit
    0.000060 alloc_iommu
    0.000060 add_to_page_cache_locked
    0.000050 zero_fd_set
    0.000050 vsnprintf
    0.000050 unlock_page
    0.000050 tty_read
    0.000050 tty_poll
    0.000050 sock_poll
    0.000050 sock_def_error_report
    0.000050 set_wq_data
    0.000050 rcu_check_callbacks
    0.000050 radix_tree_node_rcu_free
    0.000050 pipe_poll
    0.000050 opost
    0.000050 n_tty_chars_in_buffer
    0.000050 __next_cpu
    0.000050 mutex_trylock
    0.000050 msecs_to_jiffies
    0.000050 mempool_alloc_slab
    0.000050 load_elf_binary
    0.000050 __link_path_walk
    0.000050 __journal_remove_journal_head
    0.000050 journal_commit_transaction
    0.000050 journal_cancel_revoke
    0.000050 irq_complete_move
    0.000050 irq_cfg
    0.000050 fsnotify_modify
    0.000050 __first_cpu
    0.000050 file_update_time
    0.000050 filemap_fault
    0.000050 ext3_new_blocks
    0.000050 ext3_mark_inode_dirty
    0.000050 do_wp_page
    0.000050 __do_fault
    0.000050 buffer_dirty
    0.000050 anon_vma_prepare
    0.000040 yield
    0.000040 wq_per_cpu
    0.000040 walk_page_buffers
    0.000040 __wake_up_bit
    0.000040 vma_adjust
    0.000040 tty_put_char
    0.000040 tty_paranoia_check
    0.000040 tcp_current_ssthresh
    0.000040 sys_write
    0.000040 sys_rt_sigprocmask
    0.000040 sock_no_bind
    0.000040 show_stat
    0.000040 SetPageSwapBacked
    0.000040 set_irq_regs
    0.000040 set_buffer_write_io_error
    0.000040 recalc_sigpending
    0.000040 radix_tree_delete
    0.000040 queue_delayed_work_on
    0.000040 pty_write
    0.000040 __pollwait
    0.000040 physflat_send_IPI_allbutself
    0.000040 page_zone
    0.000040 page_remove_rmap
    0.000040 page_is_file_cache
    0.000040 page_evictable
    0.000040 nv_get_empty_tx_slots
    0.000040 n_tty_poll
    0.000040 next_zone
    0.000040 next_online_pgdat
    0.000040 need_resched
    0.000040 mutex_unlock
    0.000040 mpol_needs_cond_ref
    0.000040 __lookup
    0.000040 journal_invalidatepage
    0.000040 journal_dirty_metadata
    0.000040 ioread8
    0.000040 input_available_p
    0.000040 inet_csk_reset_xmit_timer
    0.000040 get_fd_set
    0.000040 generic_write_checks
    0.000040 free_poll_entry
    0.000040 fput
    0.000040 __ext3_journal_stop
    0.000040 ext3_get_group_desc
    0.000040 ext3_get_block
    0.000040 do_mpage_readpage
    0.000040 __d_lookup
    0.000040 del_page_from_lru
    0.000040 __dec_zone_state
    0.000040 copy_user_generic
    0.000040 __bitmap_and
    0.000040 add_page_to_lru_list
    0.000040 account_user_time_scaled
    0.000040 account_steal_time
    0.000030 worker_thread
    0.000030 wake_up_bit
    0.000030 vmstat_update
    0.000030 vm_normal_page
    0.000030 tty_write_unlock
    0.000030 tty_write_lock
    0.000030 tty_wakeup
    0.000030 tty_ldisc_try
    0.000030 tty_ioctl
    0.000030 tag_get
    0.000030 sys_pread64
    0.000030 submit_bh
    0.000030 stop_this_cpu
    0.000030 sock_aio_write
    0.000030 sk_mem_reclaim
    0.000030 sk_backlog_rcv
    0.000030 show_interrupts
    0.000030 sg_next
    0.000030 seq_printf
    0.000030 send_remote_softirq
    0.000030 remove_vma
    0.000030 reg_delay
    0.000030 radix_tree_lookup
    0.000030 radix_tree_insert
    0.000030 proc_lookup_de
    0.000030 pipe_write
    0.000030 __percpu_counter_add
    0.000030 pci_map_single
    0.000030 nv_napi_poll
    0.000030 __next_node
    0.000030 native_send_call_func_ipi
    0.000030 mpage_readpages
    0.000030 mix_pool_bytes_extract
    0.000030 mii_rw
    0.000030 mempool_alloc
    0.000030 __make_request
    0.000030 jbd_lock_bh_state
    0.000030 iov_iter_copy_from_user_atomic
    0.000030 insert_work
    0.000030 hrtimer_try_to_cancel
    0.000030 get_dma_ops
    0.000030 __generic_file_aio_write_nolock
    0.000030 gart_map_sg
    0.000030 __fput
    0.000030 fixup_irqs
    0.000030 __find_get_block_slow
    0.000030 filp_close
    0.000030 ext3_get_branch
    0.000030 ext3_dirty_inode
    0.000030 ext3_block_to_path
    0.000030 do_get_write_access
    0.000030 delayed_work_timer_fn
    0.000030 csum_block_add
    0.000030 copy_process
    0.000030 copy_page_range
    0.000030 constant_test_bit
    0.000030 constant_test_bit
    0.000030 check_irqs_on
    0.000030 call_rcu
    0.000030 __brelse
    0.000030 _atomic_dec_and_lock
    0.000020 __xchg
    0.000020 vm_stat_account
    0.000020 vma_prio_tree_remove
    0.000020 tty_mode_ioctl
    0.000020 tty_audit_add_data
    0.000020 try_to_free_buffers
    0.000020 truncate_inode_pages_range
    0.000020 tcp_slow_start
    0.000020 task_curr
    0.000020 sys_setpgid
    0.000020 sys_rt_sigreturn
    0.000020 sys_getppid
    0.000020 strncpy_from_user
    0.000020 sock_put
    0.000020 smp_call_function
    0.000020 __sk_mem_reclaim
    0.000020 signal_wake_up
    0.000020 signal_pending
    0.000020 set_termios
    0.000020 SetPageUptodate
    0.000020 SetPageLRU
    0.000020 set_fd_set
    0.000020 set_bit
    0.000020 __send_IPI_shortcut
    0.000020 security_inode_need_killpriv
    0.000020 scsi_request_fn
    0.000020 sb_bread
    0.000020 restore_i387_xstate
    0.000020 __qdisc_run
    0.000020 pud_alloc
    0.000020 pmd_alloc
    0.000020 pfn_pte
    0.000020 pfifo_fast_enqueue
    0.000020 pfifo_fast_dequeue
    0.000020 pci_map_page
    0.000020 path_get
    0.000020 __pagevec_free
    0.000020 pagevec_add
    0.000020 PageUnevictable
    0.000020 page_mapping
    0.000020 nv_get_hw_stats
    0.000020 number
    0.000020 normalize_rt_tasks
    0.000020 __netif_tx_lock
    0.000020 mk_pid
    0.000020 memscan
    0.000020 memcpy_c
    0.000020 __lru_cache_add
    0.000020 __lookup_mnt
    0.000020 load_balance_rt
    0.000020 kthread_should_stop
    0.000020 journal_start
    0.000020 journal_remove_journal_head
    0.000020 __journal_file_buffer
    0.000020 jbd_unlock_bh_journal_head
    0.000020 itimer_get_remtime
    0.000020 irq_to_desc
    0.000020 iowrite32
    0.000020 inotify_remove_watch_locked
    0.000020 inode_permission
    0.000020 inode_has_perm
    0.000020 init_timer
    0.000020 goal_in_my_reservation
    0.000020 get_vma_policy
    0.000020 __get_free_pages
    0.000020 generic_sync_sb_inodes
    0.000020 gart_map_single
    0.000020 freezing
    0.000020 free_pgtables
    0.000020 free_pages_and_swap_cache
    0.000020 free_buffer_head
    0.000020 __follow_mount
    0.000020 flush_tlb_page
    0.000020 find_busiest_queue
    0.000020 file_has_perm
    0.000020 ext3_try_to_allocate
    0.000020 ext3_journal_start
    0.000020 __ext3_journal_dirty_metadata
    0.000020 ext3_file_write
    0.000020 enqueue_hrtimer
    0.000020 dup_mm
    0.000020 do_wait
    0.000020 do_vfs_ioctl
    0.000020 do_path_lookup
    0.000020 do_munmap
    0.000020 do_machine_check
    0.000020 do_lookup
    0.000020 do_follow_link
    0.000020 dma_unmap_single
    0.000020 __dec_zone_page_state
    0.000020 count_vm_event
    0.000020 constant_test_bit
    0.000020 constant_test_bit
    0.000020 compound_head
    0.000020 clear_buffer_jbddirty
    0.000020 clear_buffer_delay
    0.000020 claim_block
    0.000020 cascade
    0.000020 cancel_dirty_page
    0.000020 cache_grow
    0.000020 brelse
    0.000020 __block_prepare_write
    0.000020 __blocking_notifier_call_chain
    0.000020 blk_rq_map_sg
    0.000020 __bitmap_empty
    0.000020 __bitmap_andnot
    0.000020 anon_vma_unlink
    0.000010 zone_page_state
    0.000010 zero_user_segments
    0.000010 __xchg
    0.000010 __vma_link_rb
    0.000010 vma_link
    0.000010 vfs_llseek
    0.000010 __up_write
    0.000010 update_xtime_cache
    0.000010 unmap_underlying_metadata
    0.000010 unmap_region
    0.000010 unix_poll
    0.000010 tty_write_room
    0.000010 tty_unthrottle
    0.000010 tty_ldisc_ref_wait
    0.000010 tty_ldisc_ref
    0.000010 tty_fasync
    0.000010 tty_check_change
    0.000010 tty_chars_in_buffer
    0.000010 tty_audit_fork
    0.000010 truncate_complete_page
    0.000010 test_tsk_thread_flag
    0.000010 taskstats_exit
    0.000010 sys_writev
    0.000010 sys_readahead
    0.000010 sys_poll
    0.000010 sys_newstat
    0.000010 sys_nanosleep
    0.000010 sys_ioctl
    0.000010 syscall_trace_leave
    0.000010 sync_supers
    0.000010 stub_execve
    0.000010 split_page
    0.000010 sock_kfree_s
    0.000010 __sleep_on_page_lock
    0.000010 skip_atoi
    0.000010 signal_pending
    0.000010 signal_pending
    0.000010 sg_init_table
    0.000010 set_task_cpu
    0.000010 __set_page_dirty
    0.000010 SetPageActive
    0.000010 set_bit
    0.000010 seq_puts
    0.000010 selinux_task_setpgid
    0.000010 selinux_secctx_to_secid
    0.000010 selinux_sb_show_options
    0.000010 selinux_inode_permission
    0.000010 selinux_inode_need_killpriv
    0.000010 selinux_inode_free_security
    0.000010 selinux_inode_alloc_security
    0.000010 selinux_d_instantiate
    0.000010 security_vm_enough_memory
    0.000010 second_overflow
    0.000010 scsi_run_queue
    0.000010 __scsi_put_command
    0.000010 scsi_init_sgtable
    0.000010 scsi_end_request
    0.000010 schedule_tail
    0.000010 schedule_delayed_work
    0.000010 sb_any_quota_enabled
    0.000010 rt_hash
    0.000010 round_jiffies_relative
    0.000010 remove_hrtimer
    0.000010 __remove_hrtimer
    0.000010 __remove_from_page_cache
    0.000010 rcu_bh_qsctr_inc
    0.000010 radix_tree_tag_clear
    0.000010 radix_tree_gang_lookup_tag_slot
    0.000010 radix_tree_gang_lookup_slot
    0.000010 queue_delayed_work
    0.000010 qdisc_run
    0.000010 put_tty_queue_nolock
    0.000010 put_io_context
    0.000010 pty_write_room
    0.000010 pty_open
    0.000010 ptep_set_access_flags
    0.000010 profile_munmap
    0.000010 proc_pident_lookup
    0.000010 proc_get_inode
    0.000010 prio_tree_replace
    0.000010 prio_tree_remove
    0.000010 prio_tree_insert
    0.000010 pmd_none_or_clear_bad
    0.000010 pipe_release
    0.000010 pipe_read
    0.000010 pid_revalidate
    0.000010 pgd_alloc
    0.000010 pci_unmap_single
    0.000010 pci_read_config_dword
    0.000010 pci_conf1_write
    0.000010 pci_bus_read_config_dword
    0.000010 path_walk
    0.000010 page_zone
    0.000010 PageSwapCache
    0.000010 PageSwapCache
    0.000010 PageSwapCache
    0.000010 __page_set_anon_rmap
    0.000010 PagePrivate
    0.000010 PagePrivate
    0.000010 PagePrivate
    0.000010 page_add_file_rmap
    0.000010 on_each_cpu
    0.000010 nv_do_interrupt
    0.000010 net_tx_action
    0.000010 netif_start_queue
    0.000010 netif_carrier_ok
    0.000010 need_resched
    0.000010 need_iommu
    0.000010 native_pte_clear
    0.000010 native_io_delay
    0.000010 mutex_lock
    0.000010 mprotect_fixup
    0.000010 mod_zone_page_state
    0.000010 mntput_no_expire
    0.000010 mm_init
    0.000010 mmap_region
    0.000010 mempool_free
    0.000010 memcmp
    0.000010 mcheck_check_cpu
    0.000010 may_open
    0.000010 __lookup_tag
    0.000010 locks_remove_posix
    0.000010 locks_remove_flock
    0.000010 lock_buffer
    0.000010 load_elf_binary
    0.000010 load_balance_fair
    0.000010 ll_back_merge_fn
    0.000010 kzalloc
    0.000010 ktime_add_safe
    0.000010 kill_fasync
    0.000010 __journal_temp_unlink_buffer
    0.000010 journal_switch_revoke_table
    0.000010 __journal_remove_checkpoint
    0.000010 journal_get_write_access
    0.000010 journal_get_undo_access
    0.000010 journal_get_descriptor_buffer
    0.000010 journal_bmap
    0.000010 jbd_unlock_bh_state
    0.000010 jbd_unlock_bh_state
    0.000010 IRQ0xd2_interrupt
    0.000010 ip_append_data
    0.000010 iov_iter_advance
    0.000010 iov_fault_in_pages_read
    0.000010 iommu_area_alloc
    0.000010 inode_sub_bytes
    0.000010 inode_doinit_with_dentry
    0.000010 inode_add_bytes
    0.000010 __inc_zone_page_state
    0.000010 inc_zone_page_state
    0.000010 hweight_long
    0.000010 hweight64
    0.000010 hrtimer_wakeup
    0.000010 hrtimer_init
    0.000010 hash_64
    0.000010 half_md4_transform
    0.000010 __grab_cache_page
    0.000010 get_user_pages
    0.000010 get_signal_to_deliver
    0.000010 get_random_int
    0.000010 getname
    0.000010 get_empty_filp
    0.000010 __getblk
    0.000010 generic_permission
    0.000010 generic_make_request
    0.000010 generic_fillattr
    0.000010 generic_file_open
    0.000010 generic_file_llseek_unlocked
    0.000010 generic_file_buffered_write
    0.000010 generic_file_aio_write
    0.000010 generic_cont_expand_simple
    0.000010 generic_block_bmap
    0.000010 freezing
    0.000010 free_swap_cache
    0.000010 free_pid
    0.000010 free_pgd_range
    0.000010 free_pages
    0.000010 flush_old_exec
    0.000010 first_online_pgdat
    0.000010 find_vma_prepare
    0.000010 find_task_by_pid_type_ns
    0.000010 find_next_zero_bit
    0.000010 find_inode_fast
    0.000010 file_remove_suid
    0.000010 file_mask_to_av
    0.000010 file_free_rcu
    0.000010 __FD_CLR
    0.000010 ext3_write_begin
    0.000010 ext3_try_to_allocate_with_rsv
    0.000010 ext3_ordered_write_end
    0.000010 ext3_journalled_set_page_dirty
    0.000010 ext3_invalidatepage
    0.000010 ext3_iget_acl
    0.000010 ext3_get_inode_flags
    0.000010 ext3_free_data
    0.000010 ext3_discard_reservation
    0.000010 exit_thread
    0.000010 exit_task_namespaces
    0.000010 exit_sem
    0.000010 end_that_request_last
    0.000010 end_buffer_write_sync
    0.000010 end_buffer_async_write
    0.000010 elv_rb_del
    0.000010 elv_queue_empty
    0.000010 elv_merged_request
    0.000010 elv_completed_request
    0.000010 elf_map
    0.000010 echo_char
    0.000010 e1000_watchdog
    0.000010 e1000_read_phy_reg
    0.000010 __drain_alien_cache
    0.000010 __d_path
    0.000010 __down_write_nested
    0.000010 __down_write
    0.000010 double_rq_lock
    0.000010 do_timer
    0.000010 do_sys_open
    0.000010 do_sigaltstack
    0.000010 do_sigaction
    0.000010 do_setitimer
    0.000010 do_pipe_flags
    0.000010 __do_page_cache_readahead
    0.000010 do_notify_parent
    0.000010 do_filp_open
    0.000010 do_exit
    0.000010 dnotify_flush
    0.000010 d_kill
    0.000010 destroy_inode
    0.000010 dequeue_signal
    0.000010 de_put
    0.000010 delayacct_end
    0.000010 create_write_pipe
    0.000010 create_workqueue_thread
    0.000010 __cpus_equal
    0.000010 cpu_quiet
    0.000010 __cpu_clear
    0.000010 __cpu_clear
    0.000010 count
    0.000010 copy_thread
    0.000010 copy_namespaces
    0.000010 constant_test_bit
    0.000010 constant_test_bit
    0.000010 constant_test_bit
    0.000010 constant_test_bit
    0.000010 constant_test_bit
    0.000010 __cond_resched
    0.000010 clocksource_forward_now
    0.000010 __clear_user
    0.000010 clear_inode
    0.000010 clear_buffer_new
    0.000010 clear_bit
    0.000010 clear_bit
    0.000010 check_for_bios_corruption
    0.000010 __cfq_slice_expired
    0.000010 cfq_set_request
    0.000010 cfq_dispatch_requests
    0.000010 cfq_completed_request
    0.000010 cap_set_effective
    0.000010 can_share_swap_page
    0.000010 bvec_alloc_bs
    0.000010 buffer_uptodate
    0.000010 buffer_mapped
    0.000010 buffer_locked
    0.000010 buffer_jbd
    0.000010 buffer_jbd
    0.000010 brelse
    0.000010 __bread
    0.000010 blk_invoke_request_fn
    0.000010 __blk_complete_request
    0.000010 blk_add_trace_generic
    0.000010 blk_add_trace_bio
    0.000010 bit_spin_lock
    0.000010 bio_put
    0.000010 bio_alloc_bioset
    0.000010 bdi_read_congested
    0.000010 balance_runtime
    0.000010 balance_dirty_pages_ratelimited_nr
    0.000010 audit_log_task_context
    0.000010 ata_sff_qc_prep
    0.000010 ata_scsi_queuecmd
    0.000010 ata_link_max_devices
    0.000010 ata_get_xlat_func
    0.000010 arp_process
    0.000010 arch_pick_mmap_layout
    0.000010 arch_irq_stat_cpu
    0.000010 arch_dup_task_struct
    0.000010 alloc_pid
    0.000010 alloc_fdtable
    0.000010 alloc_fd
    0.000010 add_mm_rss
    0.000010 acct_collect


    \
     
     \ /
      Last update: 2008-11-17 19:53    [W:0.085 / U:31.556 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site