[lkml]   [2000]   [Aug]   [8]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
Messages in this thread
    SubjectRe: [Announce] Linux Test Project

    Larry, thank you for your post. This is exactly the post I was looking
    for to seed conversation on this subject. Thank you for your testimony
    into the value of regression testing. Regression testing isn't a
    panacea, but it can have great value if leveraged correctly.

    In regard to your framework, this is very similar to some ideas we had
    for our project. We (SGI) have a regression testing framework in use
    internally that is very similar in basis to your framework. The major
    difference is that it has been in use and development for years now.
    Its quite grandiose and complete -- I think we might open source it if
    LTP came to need it. For now, your framework is more along the line of
    what we had in mind for LTP.

    The exit status method is probably the simplest way of reporting test
    results. It makes it relatively easy for anyone to write tests (take
    note you lkml lurkers). Basically, writing a simple test goes something
    like this:
    Setup - get the proc environment ready to perform the test operation
    Test - perform the test operation
    Analysis - determine the success/failure of the operation
    Cleanup - get ready to go away, cleanup scratch space, make sure system
    is in a clean state
    Exit - report results via exit status 0-PASS,!0-FAIL

    Larry, I may hit you up for your test harness. It may be a great place
    for us to start. We already have 90+ tests, and managing them is
    already a problem.

    Again, anyone interested in helping us with our project is encouraged to


    Larry McVoy wrote:
    > I'm a huge fan of this idea. We have a very simple, but effective, test
    > harness in BitKeeper. You are welcome to it if you want it. All we do is
    > have directory full of "t.files" which are shell scripts. There is a
    > setup program called "doit" which clears out /tmp/.regression-$USER,
    > sets up the environment, goes to that directory, and then runs the
    > script. If the script exits 0, it does the next one, else it stops.
    > We've got 12,000 lines of shell scripts we use to test BitKeeper and it
    > has been priceless. In theory, we don't fix a bug without adding a
    > regression, in practice that is true about 80% of the time. We're quite
    > happy that it is that high. The reason it is that high is that once the
    > harness is there, you can create a new test like so:
    > cat > t/t.small
    > echo $N Try wrapping and unwrapping some data .......................$NL
    > ls / > data
    > bk uuwrap < data > data.uu
    > bk unuuwrap < data.uu | cmp -s - data
    > if [ $? != 0 ]; then echo Data is different when unwrapped; exit 1; fi
    > echo OK
    > ^D
    > I strongly urge you to do the work to create a similar simple harness.
    > If it is trivial for people to create tests, they _might_ create them.
    > If it is anything other than trivial, it's just another good idea that
    > won't happen.
    > If I get to Linux World I'll look you up, otherwise you can ping me
    > for the test harness.
    > On Tue, Aug 08, 2000 at 03:06:28PM -0500, Nathan Straz wrote:
    > > SGI would like to announce the Linux Test Project. The goal of this
    > > project is to create a formalized test system for the Linux kernel.
    > >
    > > We have released a set of 96 tests on the project's website
    > > ( These tests exercise file systems
    > > and system calls and can be used for stress testing or sanity tests.
    > >
    > > We would like to discuss the following topics with the community.
    > >
    > > - The testing philosophy that is most important to the kernel
    > > developers. What approach best fits the development process?
    > > Regression? Functional? Stress? Performance?
    > >
    > > - What is needed immediately? Building a test suite for the kernel is
    > > going to take time. What tests or tools are most important?
    > >
    > > - We need to plan a development road map that works with the Linux
    > > kernel development road map.
    > >
    > > We are hoping to hold an unscheduled BOF at LinuxWorld on Wednesday.
    > > Aaron Laffin and Richard Logan will be there to discuss these issues.
    > > If you are interested in testing and are attending LinuxWorld, please
    > > keep an eye open for our BOF.
    > >
    > > Linux Test Project
    > > LTP Mailing list
    > >
    > > -
    > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    > > the body of a message to
    > > Please read the FAQ at
    > --
    > ---
    > Larry McVoy
    > -
    > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    > the body of a message to
    > Please read the FAQ at

    Aaron Laffin
    Silicon Graphics, Inc. OS Test Development
    Email: Voice: 651-683-5756

    To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
    the body of a message to
    Please read the FAQ at

     \ /
      Last update: 2005-03-22 13:58    [W:0.025 / U:1.472 seconds]
    ©2003-2016 Jasper Spaans. hosted at Digital OceanAdvertise on this site