Messages in this thread | | | Date | Tue, 21 Feb 2023 22:40:07 -0800 | From | Tavis Ormandy <> | Subject | x86: AMD Zen2 ymm registers rolling back |
| |
Hello, I'm experiencing a serious bug on AMD Zen2 that is causing registers to "roll back" to previous values after a context switch.
I know that sounds unbelievable - but I have a reliable reproducer!
$ grep -m1 'model name' /proc/cpuinfo model name : AMD Ryzen Threadripper PRO 3945WX 12-Cores
The bug occurs when there is a context switch after a ucomiss instruction. The YMM registers are "rolled back" to some previous state.
It's not clear to me how or why this is happening, or if it can happen across a process boundary.
To reproduce:
$ nasm -felf64 -O0 zenymmasm.asm $ ld -o zenymmasm zenymmasm.o
If you run it it should just print some nuls:
$ ./zenymmasm $
That is the expected, correct result.
However, if you run the attached program hammer.c (it just runs sched_yield() in a loop), and then pin this testcase to the same core as that:
$ taskset -c 1 ./zenymmasm SECRETSECRET
The previous register values are restored.
I think this should be impossible.
The code does this:
vmovdqu ymm0, [rel secret] <--- put SECRET into ymm0 mov rax, SYS_sched_yield syscall vpxor ymm0, ymm0, ymm0 <--- Here the value of ymm0 should be lost
It's not related to to VPXOR, you can use VZEROALL or whatever else.
ucomiss xmm0, dword [rel space] mov rax, SYS_sched_yield syscall
It's the UCOMISS r128,m32 instruction that triggers the bug, the value of the m32 must be < 0x80000. As far as I know, this should only ever change condition flags, but if we dump the value of ymm0:
mov rax, SYS_write mov rdi, 1 lea rsi, [rel regstate] mov rdx, 32 syscall
It will have reverted back to the pre-vpxor SECRET value !?!?
In the original C program, we were seeing register values randomly restored from significantly earlier in the program execution (like, from ld.so), sometimes values we don't recognize and can't explain.
We've reproduced on multiple Zen2 machines.
Obviously, not great if your registers are randomly time travelling :)
Thanks, Tavis.
-- _o) $ lynx lock.cmpxchg8b.com /\\ _o) _o) $ finger taviso@sdf.org _\_V _( ) _( ) @taviso #define _GNU_SOURCE #include <stdint.h> #include <stdlib.h> #include <string.h> #include <stdio.h> #include <unistd.h> #include <stdbool.h> #include <sched.h> #include <syscall.h> #include <x86intrin.h> #include <sys/random.h>
#include <err.h>
int main(int argc, char **argv) { cpu_set_t set;
CPU_ZERO(&set); CPU_SET(1, &set);
if (sched_setaffinity(0, sizeof(set), &set) != 0) { err(EXIT_FAILURE, "failed to set cpu affinity"); }
while (true) { asm volatile ("syscall" :: "a"(SYS_sched_yield) : "rcx", "r11"); }
return 0; } BITS 64
global _start
%define SYS_sched_yield 0x18 %define SYS_write 0x01 %define SYS_exit 0x3c
section .data align 32 secret: times 4 dq 'SECRET' align 32 regstate: dq 0,0,0,0 align 32 space: dd 1 section .text _start: vmovdqu ymm0, [rel secret] mov rax, SYS_sched_yield syscall vpxor ymm0, ymm0, ymm0 mov rax, SYS_sched_yield syscall ucomiss xmm0, dword [rel space] mov rax, SYS_sched_yield syscall vmovdqu [rel regstate], ymm0 mov rax, SYS_write mov rdi, 1 lea rsi, [rel regstate] mov rdx, 32 syscall mov rax, SYS_exit mov rdi, 0 syscall int3
| |