lkml.org 
[lkml]   [2023]   [Feb]   [22]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
Subjectx86: AMD Zen2 ymm registers rolling back
Hello, I'm experiencing a serious bug on AMD Zen2 that is causing
registers to "roll back" to previous values after a context switch.

I know that sounds unbelievable - but I have a reliable reproducer!

$ grep -m1 'model name' /proc/cpuinfo
model name : AMD Ryzen Threadripper PRO 3945WX 12-Cores

The bug occurs when there is a context switch after a ucomiss
instruction. The YMM registers are "rolled back" to some previous state.

It's not clear to me how or why this is happening, or if it can happen
across a process boundary.

To reproduce:

$ nasm -felf64 -O0 zenymmasm.asm
$ ld -o zenymmasm zenymmasm.o

If you run it it should just print some nuls:

$ ./zenymmasm
$

That is the expected, correct result.

However, if you run the attached program hammer.c (it just runs
sched_yield() in a loop), and then pin this testcase to the same core as
that:

$ taskset -c 1 ./zenymmasm
SECRETSECRET

The previous register values are restored.

I think this should be impossible.

The code does this:

vmovdqu ymm0, [rel secret] <--- put SECRET into ymm0
mov rax, SYS_sched_yield
syscall
vpxor ymm0, ymm0, ymm0 <--- Here the value of ymm0 should be lost

It's not related to to VPXOR, you can use VZEROALL or whatever else.

ucomiss xmm0, dword [rel space]
mov rax, SYS_sched_yield
syscall

It's the UCOMISS r128,m32 instruction that triggers the bug, the value
of the m32 must be < 0x80000. As far as I know, this should only ever
change condition flags, but if we dump the value of ymm0:

mov rax, SYS_write
mov rdi, 1
lea rsi, [rel regstate]
mov rdx, 32
syscall

It will have reverted back to the pre-vpxor SECRET value !?!?

In the original C program, we were seeing register values randomly
restored from significantly earlier in the program execution (like, from
ld.so), sometimes values we don't recognize and can't explain.

We've reproduced on multiple Zen2 machines.

Obviously, not great if your registers are randomly time travelling :)

Thanks, Tavis.

--
_o) $ lynx lock.cmpxchg8b.com
/\\ _o) _o) $ finger taviso@sdf.org
_\_V _( ) _( ) @taviso
#define _GNU_SOURCE
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <unistd.h>
#include <stdbool.h>
#include <sched.h>
#include <syscall.h>
#include <x86intrin.h>
#include <sys/random.h>

#include <err.h>

int main(int argc, char **argv)
{
cpu_set_t set;

CPU_ZERO(&set);
CPU_SET(1, &set);

if (sched_setaffinity(0, sizeof(set), &set) != 0) {
err(EXIT_FAILURE, "failed to set cpu affinity");
}

while (true) {
asm volatile ("syscall" :: "a"(SYS_sched_yield) : "rcx", "r11");
}

return 0;
}
BITS 64

global _start

%define SYS_sched_yield 0x18
%define SYS_write 0x01
%define SYS_exit 0x3c

section .data
align 32
secret: times 4 dq 'SECRET'
align 32
regstate: dq 0,0,0,0
align 32
space: dd 1
section .text
_start:
vmovdqu ymm0, [rel secret]
mov rax, SYS_sched_yield
syscall
vpxor ymm0, ymm0, ymm0
mov rax, SYS_sched_yield
syscall
ucomiss xmm0, dword [rel space]
mov rax, SYS_sched_yield
syscall
vmovdqu [rel regstate], ymm0
mov rax, SYS_write
mov rdi, 1
lea rsi, [rel regstate]
mov rdx, 32
syscall
mov rax, SYS_exit
mov rdi, 0
syscall
int3
\
 
 \ /
  Last update: 2023-03-27 00:32    [W:0.336 / U:0.204 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site