Messages in this thread | | | From | Jim Mattson <> | Date | Thu, 20 Aug 2020 11:39:23 -0700 | Subject | Re: FSGSBASE causing panic on 5.9-rc1 |
| |
On Thu, Aug 20, 2020 at 11:38 AM Jim Mattson <jmattson@google.com> wrote: > > On Thu, Aug 20, 2020 at 11:34 AM Tom Lendacky <thomas.lendacky@amd.com> wrote: > > > > On 8/20/20 11:30 AM, Tom Lendacky wrote: > > > On 8/20/20 11:17 AM, Tom Lendacky wrote: > > >> On 8/20/20 10:55 AM, Andy Lutomirski wrote: > > >>> On Thu, Aug 20, 2020 at 8:21 AM Tom Lendacky <thomas.lendacky@amd.com> > > >>> wrote: > > >>>> > > >>>> On 8/20/20 10:10 AM, Sean Christopherson wrote: > > >>>>> On Wed, Aug 19, 2020 at 05:21:33PM -0700, Andy Lutomirski wrote: > > >>>>>> On Wed, Aug 19, 2020 at 2:25 PM Andy Lutomirski <luto@kernel.org> > > >>>>>> wrote: > > >>>>>>> > > >>>>>>> On Wed, Aug 19, 2020 at 11:19 AM Tom Lendacky > > >>>>>>> <thomas.lendacky@amd.com> wrote: > > >>>>>>>> > > >>>>>>>> On 8/19/20 1:07 PM, Tom Lendacky wrote: > > >>>>>>>>> It looks like the FSGSBASE support is crashing my second > > >>>>>>>>> generation EPYC > > >>>>>>>>> system. I was able to bisect it to: > > >>>>>>>>> > > >>>>>>>>> b745cfba44c1 ("x86/cpu: Enable FSGSBASE on 64bit by default and > > >>>>>>>>> add a chicken bit") > > >>>>>>>>> > > >>>>>>>>> The panic only happens when using KVM. Doing kernel builds or stress > > >>>>>>>>> on bare-metal appears fine. But if I fire up, in this case, a > > >>>>>>>>> 64-vCPU > > >>>>>>>>> guest and do a kernel build within the guest, I get the following: > > >>>>>>>> > > >>>>>>>> I should clarify that this panic is on the bare-metal system, not > > >>>>>>>> in the > > >>>>>>>> guest. And that specifying nofsgsbase on the bare-metal command > > >>>>>>>> line fixes > > >>>>>>>> the issue. > > >>>>>>> > > >>>>>>> I certainly see some oddities: > > >>>>>>> > > >>>>>>> We have this code: > > >>>>>>> > > >>>>>>> static void svm_vcpu_put(struct kvm_vcpu *vcpu) > > >>>>>>> { > > >>>>>>> struct vcpu_svm *svm = to_svm(vcpu); > > >>>>>>> int i; > > >>>>>>> > > >>>>>>> avic_vcpu_put(vcpu); > > >>>>>>> > > >>>>>>> ++vcpu->stat.host_state_reload; > > >>>>>>> kvm_load_ldt(svm->host.ldt); > > >>>>>>> #ifdef CONFIG_X86_64 > > >>>>>>> loadsegment(fs, svm->host.fs); > > >>>>>>> wrmsrl(MSR_KERNEL_GS_BASE, current->thread.gsbase); > > >>>>> > > >>>>> Pretty sure current->thread.gsbase can be stale, i.e. this needs: > > >>>>> > > >>>>> current_save_fsgs(); > > >>>> > > >>>> I did try adding current_save_fsgs() in svm_vcpu_load(), saving the > > >>>> current->thread.gsbase value to a new variable in the svm struct. I then > > >>>> used that variable in the wrmsrl below, but it still crashed. > > >>> > > >>> Can you try bisecting all the way back to: > > >>> > > >>> commit dd649bd0b3aa012740059b1ba31ecad28a408f7f > > >>> Author: Andy Lutomirski <luto@kernel.org> > > >>> Date: Thu May 28 16:13:48 2020 -0400 > > >>> > > >>> x86/cpu: Add 'unsafe_fsgsbase' to enable CR4.FSGSBASE > > >>> > > >>> and adding the unsafe_fsgsbase command line option while you bisect. > > >> > > >> I'll give that a try. > > > > Bisecting with unsafe_fsgsbase identified: > > > > c82965f9e530 ("x86/entry/64: Handle FSGSBASE enabled paranoid entry/exit") > > > > But I'm thinking that could be because it starts using GET_PERCPU_BASE, > > which on Rome would use RDPID. So is SVM restoring TSC_AUX_MSR too late? > > That would explain why I don't see the issue on Naples, which doesn't > > support RDPID. > > It looks to me like SVM loads the guest TSC_AUX from vcpu_load to > vcpu_put, with this comment: > > /* This assumes that the kernel never uses MSR_TSC_AUX */ > if (static_cpu_has(X86_FEATURE_RDTSCP)) > wrmsrl(MSR_TSC_AUX, svm->tsc_aux);
Correction: It never restores TSC_AUX, AFAICT.
| |