lkml.org 
[lkml]   [2023]   [Jun]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    Subjectkvm/powerpc: memcached benchmark
    From
    TLDR
    ====
    Memcached achieved 10% more operations per second (in ~4 hours) after this patchset [1].

    Hardware
    ========
    HOST $ lscpu
    Architecture: ppc64le
    Byte Order: Little Endian
    CPU(s): 184
    On-line CPU(s) list: 0-183
    Model name: POWER9 (raw), altivec supported
    Model: 2.2 (pvr 004e 1202)
    Thread(s) per core: 4
    Core(s) per socket: 23
    Socket(s): 2
    CPU max MHz: 3000.0000
    CPU min MHz: 2300.0000
    Caches (sum of all):
    L1d: 1.4 MiB (46 instances)
    L1i: 1.4 MiB (46 instances)
    L2: 12 MiB (24 instances)
    L3: 240 MiB (24 instances)
    NUMA:
    NUMA node(s): 2
    NUMA node0 CPU(s): 0-91
    NUMA node1 CPU(s): 92-183
    Vulnerabilities:
    Itlb multihit: Not affected
    L1tf: Mitigation; RFI Flush, L1D private per thread
    Mds: Not affected
    Meltdown: Mitigation; RFI Flush, L1D private per thread
    Mmio stale data: Not affected
    Retbleed: Not affected
    Spec store bypass: Mitigation; Kernel entry/exit barrier (eieio)
    Spectre v1: Mitigation; __user pointer sanitization, ori31 speculation barrier enabled
    Spectre v2: Mitigation; Indirect branch serialisation (kernel only), Indirect branch cache disabled, Software link stack flush
    Srbds: Not affected
    Tsx async abort: Not affected

    HOST $ numactl -H
    available: 2 nodes (0-1)
    node 0 cpus: 0-91
    node 0 size: 261659 MB
    node 0 free: 259152 MB
    node 1 cpus: 92-183
    node 1 size: 261713 MB
    node 1 free: 261076 MB
    node distances:
    node 0 1
    0: 10 40
    1: 40 10

    HOST $ cat /sys/class/nvme/nvme0/model
    INTEL SSDPF21Q800GB

    HOST $ cat /sys/class/nvme/nvme0/numa_node
    0

    Software
    ========
    HOST $ cat /etc/lsb-release
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=22.04
    DISTRIB_CODENAME=jammy
    DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"

    HOST $ uname -a
    Linux ppc 6.3.0 #1 SMP Sun Jun 4 18:26:37 UTC 2023 ppc64le ppc64le ppc64le GNU/Linux

    HOST $ cat /proc/swaps
    Filename Type Size Used Priority
    /dev/nvme0n1p2 partition 466838272 0 -2

    HOST $ cat /sys/kernel/mm/lru_gen/enabled
    0x0009

    HOST $ cat /sys/kernel/mm/transparent_hugepage/enabled
    always madvise [never]

    HOST $ cat /sys/kernel/mm/transparent_hugepage/defrag
    always defer defer+madvise madvise [never]

    HOST $ qemu-system-ppc64 --version
    QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
    Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

    GUEST $ cat /etc/lsb-release
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=22.04
    DISTRIB_CODENAME=jammy
    DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

    GUEST $ cat /etc/memcached.conf
    ...
    -t 92
    -m 262144
    -B binary
    -s /var/run/memcached/memcached.sock
    -a 0766

    GUEST $ memtier_benchmark -v
    memtier_benchmark 1.4.0
    Copyright (C) 2011-2022 Redis Ltd.
    This is free software. You may redistribute copies of it under the terms of
    the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
    There is NO WARRANTY, to the extent permitted by law.

    Procedure
    =========
    HOST $ sudo numactl -N 0 -m 0 qemu-system-ppc64 \
    -M pseries,accel=kvm,kvm-type=HV -cpu host -smp 92 -m 270g
    -nographic -nic user \
    -drive if=virtio,format=raw,file=/dev/nvme0n1p1

    GUEST $ memtier_benchmark -S /var/run/memcached/memcached.sock \
    -P memcache_binary -c 1 -t 92 --pipeline 1 --ratio 1:0 \
    --key-minimum=1 --key-maximum=120000000 --key-pattern=P:P \
    -n allkeys -d 2000

    GUEST $ memtier_benchmark -S /var/run/memcached/memcached.sock \
    -P memcache_binary -c 1 -t 92 --pipeline 1 --ratio 0:1 \
    --key-minimum=1 --key-maximum=120000000 --key-pattern=R:R \
    -n allkeys --randomize --distinct-client-seed

    Results
    =======
    Before [1] After Change
    -------------------------------------------------
    Ops/sec 721586.10 800210.12 +10%
    Avg. Latency 0.12546 0.11260 -10%
    p50 Latency 0.08700 0.08700 N/C
    p99 Latency 0.28700 0.24700 -13%

    Notes
    =====
    [1] "mm: rmap: Don't flush TLB after checking PTE young for page
    reference" was included so that the comparison is apples to
    Apples.
    https://lore.kernel.org/r/20220706112041.3831-1-21cnbao@gmail.com/

    \
     
     \ /
      Last update: 2023-06-09 03:00    [W:2.470 / U:0.024 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site