lkml.org 
[lkml]   [2018]   [Mar]   [28]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: [PATCH 4.9 00/67] 4.9.91-stable review
    On Wed, Mar 28, 2018 at 06:21:04PM +0200, Greg Kroah-Hartman wrote:
    > On Wed, Mar 28, 2018 at 10:50:35AM -0500, Dan Rue wrote:
    > > On Wed, Mar 28, 2018 at 10:21:45AM +0200, Greg Kroah-Hartman wrote:
    > > > On Tue, Mar 27, 2018 at 08:35:01PM -0500, Dan Rue wrote:
    > > > > qemu_x86_64
    > > > > * boot - pass: 21
    > > > > * kselftest - skip: 28, pass: 52
    > > >
    > > > Do you have a list of what you are skipping anywhere? There was some
    > > > x86 changes that I had to backport that I was worried about getting
    > > > right here, are you running the x86 kselftests?
    > >
    > > Yes we run the x86 selftests. Here's the full list of what ran and what was
    > > skipped. Remember (I know you know, but for anyone else observing), we
    > > run kselftest from 4.15.
    >
    > That's good, as kselftest from 4.9 was a bit broken for x86, which is
    > why I backported a bunch of patches for it for this release :)

    I've noticed these backports. Should we expect to see selftests
    supported in LTS kernels going forward? What is the current policy on
    taking selftest backports?

    >
    > > breakpoint_test — SKIP
    > > efivarfs.sh — SKIP
    > > fsgsbase_64 — SKIP
    > > ftracetest — SKIP
    > > fw_fallback.sh — SKIP
    > > fw_filesystem.sh — SKIP
    > > gpio-mockup.sh — SKIP
    > > ldt_gdt_64 — SKIP
    > > main.sh — SKIP
    > > mem-on-off-test.sh — SKIP
    > > pstore_tests — SKIP
    > > reuseport_bpf — SKIP
    > > run.sh — SKIP
    > > run_vmtests — SKIP
    > > seccomp_bpf — SKIP
    > > sigreturn_64 — SKIP
    > > step_after_suspend_test — SKIP
    > > sync_test — SKIP
    > > sysctl.sh — SKIP
    > > test_align — SKIP
    > > test_dev_cgroup — SKIP
    > > test_kmod.sh — SKIP
    > > test_lpm_map — SKIP
    > > test_lru_map — SKIP
    > > test_maps — SKIP
    > > test_progs — SKIP
    > > test_tag — SKIP
    > > test_verifier — SKIP
    >
    > Figuring out of any of these are from the tools/testing/selftests/x86/
    > directory is a pain, that is the tests I was referring to here...

    Agree. It would be nice if we preserved the directory name - I had the
    same issue when trying to answer your question.

    >
    > That being said, why are you skipping so many? ldt_gdt_64 should be
    > run, what fails with it? Same for sigreturn_64, which I think means
    > that only 2 tests from the x86 directory are being skipped, right?
    >
    > Have you gone back and looked at any of those other tests to see why
    > they are being skipped?

    I'll try to keep this short - the tl;dr is that it's historical, and
    something we're fixing now.

    Originally, we just skipped everything that failed so that we could
    focus on regressions. As new things failed, we have investigated them
    and reported them upstream or fixed with patches. Over time, many of
    these original failures have been resolved but the skipfiles may not
    have been updated because...

    Our skipfiles are per-branch and per-board, and we used to have one file
    for each combination, resulting in dozens of files that ended up with
    inconsistencies between them over time. So, we converted it to a
    structured data format that is then parsed at runtime to generate a
    skipfile for a given board/branch combination.

    That work was just completed a couple weeks ago. Now that our kselftest
    skipfile is a single structured data file[1], we need to go through it
    and resolve all of the inconsistencies, remove the skips that have long
    since been resolved, etc. This is work we plan to do in the next couple
    weeks.

    If you look through that file, you'll see quite a few things that are
    wrong. When we're done auditing it, each skip should have a coherent
    "reason" for skipping and an accurate branch/board list.

    With regard to the specific tests you asked about:
    - ldt_gdt_64: As of 4.9.89 this is still failing. I'll test again with
    the new RC.
    - sigreturn_64: Passing on x86_64 (real hardware) and getting skipped on
    qemu x86_64. For some reason it fails intermittently on qemu in our
    environment.

    Hope that helps,
    Dan

    [1] https://git.linaro.org/qa/test-definitions.git/tree/automated/linux/kselftest/skipfile-lkft.yaml

    \
     
     \ /
      Last update: 2018-03-28 20:28    [W:2.562 / U:0.008 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site