Messages in this thread | | | Date | Thu, 24 Sep 1998 19:21:11 +1000 | From | Richard Gooch <> | Subject | Re: Interesting scheduling times - NOT |
| |
Larry McVoy writes: > Richard Gooch <rgooch@atnf.csiro.au>: > : There is a *reason* the variance is there. The question is what is > : causing it. > > Yup. I've been asking you to explain that for many messages now. > I've also tried to say, over, and orver, and over that it is not > normal for there to be that kind of variance, that that has to be a > bug in your benchmark - every other context switch benchmark I've > ever seen doesn't behave like yours, they are very stable. For > every theory you've dreamed up, I've demonstrated that it is > incorrect. At what point do you just want to stop arguing and start > debugging your benchmark?
See my recent message where I quote numbers showing 30% variance in the lat_ctx numbers. It appears your code has more variance than you expected.
In case you haven't noticed, I *have* been "debugging" my benchmark. Why don't you stop arguing and be constructive instead?
> : Maybe I have a "complex" because your first (private) message to me > : started off with "What the fuck are you talking about?". And you've > : been pretty agressive about the whole thing and have used emotive > : words. > > If I've hurt your feelings, my apologies (as I said immediately in > private mail as well, nice of you to leave that out).
Yes, you did apologise, but a few messages later you were back to being impolite: "Your test is broken", "if you think about it for a millisecond". It's clear your first response is to come out all guns blazing with maximum agression. You exhibit a pattern of behaviour that is unproductive and offensive.
Every suggestion you've made as to why my test is broken (ranging from "look at top and see the low-priority processes run instead" to "read /proc/stat and count the unexpected context switches" (paraphrased)) has not identified the source of the variance in my tests.
Now I'm finding that you code also yields greater-than-expected variance. So are you still convinced that my test is broken (even though neither you, Linus or anyone else has shown *what* is broken)?
> : Even if the variances in my measurement turn out to be due to a broken > : test or sensitivity to some "uninteresting" effect, there is no > : denying that an increased run queue length slows down context switch > : times. With my test I get a cost of 0.2 us per process and with yours > : I get 0.19 us (PPro 180). > > I couldn't agree more. So maybe rather than trying to fix an > operating system that isn't broken, you should focus on getting rid > of all those extra processes. A lot of my frustration with this > argument is that I've been through this with customer after customer > at Sun and SGI. They all want you to change the operating system to > suit their application when what they should do is write their > application correctly in the first place. Too many processes are > going to screw you up no matter what queue they are on and no matter > what scheduler you use.
There are at least two separate discussions going on here. One is about the cost of extra processes on the run queue. I've demonstrated similar numbers for this with lmbench and my code. It see we have provisional agreement on this cost. The discussion now is whether the cost is significant.
The other discussion regards the variance I have measured. Here you persist in saying my test is flawed, and yet so far not a single one of your suggestions as to why it may be flawed has stood up to the test. Irrespective of whether my test turns out to be flawed, my point about run queue length costs remains.
Now, I agree in principle that you should fix broken applications rather than fix the OS. But it doesn't always work this way. There may be legitimate reasons for doing things a particular way. I have demonstrated a real-world case where there are up to 10 normal processes on the run queue. The "obvious" solution is to place the RT application on a separate machine and avoid the whole problem. This is not necessarily the right solution, as I have already indicated. One issue is the cost of transferring data between machines. The other issue is cost of hardware: buying a second machine is not always an option.
The change I'm proposing (two run queues) is very simple, it is robust and it in fact will *simplfy* some code. The current scheduler has to special-case RT processes, complicating the code and making it more prone to bugs. In fact, I found a bug this morning: RT processes which yield can end up giving control to lower priority processes. This is contrary to the POSIX.4 spec. and contrary to what I'd expect. I've fixed it and sent a patch to Linus. Not surprisingly, the patch complicates the code path, because we need yet another special case.
> This discussion reminds me of the C++ people that keep asking for > bigger hardware caches so their code will run fast. They are > constantly amazed when you show the less lines of C code, that > solves the problem they are trying to solve, using less cache and > running faster. You tell us - what's the right thing to do - fix > the hardware or fix the C++?
Don't get me started on C++. It's the most powerful language in use today. It is now an order of magnitude easier to write bloatware in 10% of the time. Fantastic.
However, sometimes we don't get to make the choices. We're stuck with the situation we're placed in and we have to make the best of it. If there is a cheap solution that allows the application to run faster, it should be considered seriously.
We should not allow general design principles to become mantra. That way lies inflexibility.
> : If you're not interested in being constructive (i.e. "I think your > : variance may be due to XYZ effect"), then why not get off my back and > : let me get on with tracking down the problem? > : If I propose a mechanism which I think may explain the variance, and > : you don't agree, all you need to do is say so and why. Repeating the > : mantra "your test is broken" is pointless. > > You keep asking me to debug it.
No, I'm not asking you to debug it. All I ask is that you don't feel like being constructive, at least don't be destructive. Constructive criticism is welcome. Finger pointing, abuse and agression are not.
> Please remember that I'm not at some cushy job, I'm spending my > savings working tools for Linux kernel developers. It's not like I > have tons of idle cycles to waste on your code. I've already put > out the effort, for free, to make an as portable and accurate > benchmark as I possibly can - a benchmark, by the way, that measures > a lot more than just toy context switches. It's been a tremendous > amount of work over quite a few years for which I've not received a > dime.
Ra, ra, ra. Do you want a medal? Do you have any idea of how much time I've contributed to the Linux community? Is it relevant? No.
You seem to have enough time argue about how my test is fundamentally flawed. You could have saved yourself time by just pointing out that you haven't measured this variance with lmbench and left it at that. Instead, you feel the need to constantly point out how my test is broken. An assertion you haven't been able to substantiate, BTW. Unexpected results don't automatically invalidate the results.
> It's bad enough making sure that my code is not full of bugs - > explain to me what possible reason I could have for debugging your > code. I think it's enough that I've proved that a properly written > test won't show that variance.
I don't demand you debug my code (again: it's not clear that my code actually has a bug).
> I think that the onus is on you to debug your own code. And until > you can show that you have it debugged, maybe it would be better if > you deferred to the code which has been show to work.
You carry on as though I'm not investigating the cause of the variance in my measurements. I'm not going to "defer" to code that (it now transpires) also shows variance. That will just hide the problem. I want to expose the problem and track it down.
BTW: your comment about "deferring" betrays an underlying arrogance.
So, if you're not willing to be constructive, or willing to admit that your code may be flawed because it's *insensitive* to the effect(s), just go away and let me get on with tracking down the problem.
Regards,
Richard....
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |