lkml.org 
[lkml]   [2019]   [Aug]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    SubjectRe: [PATCH] x86/mm/pti: in pti_clone_pgtable() don't increase addr by PUD_SIZE
    Date


    > On Aug 20, 2019, at 9:05 AM, Song Liu <songliubraving@fb.com> wrote:
    >
    >
    >
    >> On Aug 20, 2019, at 7:18 AM, Dave Hansen <dave.hansen@intel.com> wrote:
    >>
    >> On 8/20/19 7:14 AM, Song Liu wrote:
    >>>> *But*, that shouldn't get hit on a Skylake CPU since those have PCIDs
    >>>> and shouldn't have a global kernel image. Could you confirm whether
    >>>> PCIDs are supported on this CPU?
    >>> Yes, pcid is listed in /proc/cpuinfo.
    >>
    >> So what's going on? Could you confirm exactly which pti_clone_pgtable()
    >> is causing you problems? Do you have a theory as to why this manifests
    >> as a performance problem rather than a functional one?
    >>
    >> A diff of these:
    >>
    >> /sys/kernel/debug/page_tables/current_user
    >> /sys/kernel/debug/page_tables/current_kernel
    >>
    >> before and after your patch might be helpful.
    >
    > I believe the difference is from the following entries (7 PMDs)
    >
    > Before the patch:
    >
    > current_kernel: 0xffffffff81000000-0xffffffff81e04000 14352K ro GLB x pte
    > efi: 0xffffffff81000000-0xffffffff81e04000 14352K ro GLB x pte
    > kernel: 0xffffffff81000000-0xffffffff81e04000 14352K ro GLB x pte
    >
    >
    > After the patch:
    >
    > current_kernel: 0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
    > efi: 0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
    > kernel: 0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd
    >
    > current_kernel and kernel show same data though.

    A little more details on how I got here.

    We use huge page for hot text and thus reduces iTLB misses. As we
    benchmark 5.2 based kernel (vs. 4.16 based), we found ~2.5x more
    iTLB misses.

    To figure out the issue, I use a debug patch that dumps page table for
    a pid. The following are information from the workload pid.


    For the 4.16 based kernel:

    host-4.16 # grep "x pmd" /sys/kernel/debug/page_tables/dump_pid
    0x0000000000600000-0x0000000000e00000 8M USR ro PSE x pmd
    0xffffffff81a00000-0xffffffff81c00000 2M ro PSE x pmd


    For the 5.2 based kernel before this patch:

    host-5.2-before # grep "x pmd" /sys/kernel/debug/page_tables/dump_pid
    0x0000000000600000-0x0000000000e00000 8M USR ro PSE x pmd


    The 8MB text in pmd is from user space. 4.16 kernel has 1 pmd for the
    irq entry table; while 4.16 kernel doesn't have it.


    For the 5.2 based kernel after this patch:

    host-5.2-after # grep "x pmd" /sys/kernel/debug/page_tables/dump_pid
    0x0000000000600000-0x0000000000e00000 8M USR ro PSE x pmd
    0xffffffff81000000-0xffffffff81e00000 14M ro PSE GLB x pmd


    So after this patch, the 5.2 based kernel has 7 PMDs instead of 1 PMD
    in 4.16 kernel. This further reduces iTLB miss rate

    Thanks,
    Song
    \
     
     \ /
      Last update: 2019-08-20 18:38    [W:3.759 / U:0.080 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site