Messages in this thread | | | Date | Wed, 18 Jan 2023 16:54:35 -0500 | From | Steven Rostedt <> | Subject | [for-linus][PATCH 0/3] ktest.pl: Fix ssh hanging and reseting of console |
| |
I've noticed that my ssh sessions would hang during test runs, which is really frustrating when you kick off a 13 hour test before going to bed, and the second test (1 hour into it) hangs, and you need to kick it off again in the morning (wasting all that time over night).
I finally figured out the cause. There is a disconnect between the run_command that executes the test, and the "wait_for_input" that monitors the test. The wait_for_input has a default of 2 minute timeout if it doesn't see any output it returns. The run_command takes the empty string from wait_for_input as the test is finished, and then stops monitoring it, and calls waitpid() waiting for the test to exit.
The problem is that if the test has a lot of output, it will continue writing into the pipe that was suppose to go to the monitor, which has now exited the loop. When the pipe fills up, it will not finish. When the test is over, it just hangs waiting for the pipe to flush (which never happens).
To fix this, change the run_command to by default have an infinite run (which can be overridden by the new RUN_TIMEOUT option), and make the wait_for_input also wait indefinitely in this case. It now Now the tests will have its content continuously read and will exit normally.
While debugging this, I also found out why you can lose stdout on the terminal sometimes. Especially if you hit Ctrl^C while the monitor is running. It was due to missing "end_monitor" which gives back the tty to the terminal. The first two patches fix that.
git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-ktest.git devel
Head SHA1: aa9aba9382884554fe6a303744884866d137422d
Steven Rostedt (3): ktest.pl: Fix missing "end_monitor" when machine check fails ktest.pl: Give back console on Ctrt^C on monitor ktest.pl: Add RUN_TIMEOUT option with default unlimited
---- tools/testing/ktest/ktest.pl | 26 +++++++++++++++++++++----- tools/testing/ktest/sample.conf | 5 +++++ 2 files changed, 26 insertions(+), 5 deletions(-)
| |