Messages in this thread | | | Date | Mon, 31 Jan 2000 08:40:44 +1100 | From | Richard Gooch <> | Subject | Re: Auto-Adaptive scheduler - Final chapter ( the numbers ) ... |
| |
Davide Libenzi writes: > Hi Richard, > > > Yeah, I'm reading it (barely, I've been deleting most of my email > > because I've been on holidays). The PMC (Performance Monitoring > > Counter) patch I wrote is available at: > > http://www.atnf.csiro.au/~rgooch/linux/ > > can You give me a hint on how to use int the better way Your PMC > code ?
The page with the kernel patches also has a link to the user-space API and command-line utilities. That should give you all you need.
> How is the noise figure induced onto the measure ?
What do you mean?
> What is Your impression of the patch ( if You've seen ) ?
I've not seen it. I've had to delete most of my email this month as I've been on holidays. I either had to delete or go mad. I chose sanity. For the same reason, this reply is late. Sorry.
> It has also the "side effect" of keeping RT task onto the higher > cluster for a better RT latency.
From what I've seen in the discussion, your patch enables your new scheduler after many processes have been on the run queue for a while, correct? That's not the same as the case that was interesting me when I was doing my RT run queue work.
That case that interests me is when you have a number of SCHED_OTHER processes on the run queue (usually due to some event waking up several processes), and then a new event wakes up a RT process (either an interrupt or another RT process).
With a large run queue, there is a significant slowdown in waking the RT process. What I did was to create a separate RT run queue so that RT context switch times were (more) isolated from SCHED_OTHER. The (real) application where this mattered was an embedded system that controlled hardware, plus had a serial interface to display status. The processes that controlled hardware (via device drivers) were RT whereas the display processes were SCHED_OTHER.
So it doesn't look like your patch does anything for the case that I care about, because there won't be a large number of processes on the run queue for long times. That doesn't mean your patch is bad, it just doesn't help *me*.
However, without having looked at your patch (but having glanced at the general response on the list), I'm skeptical about your patch, for these reasons:
- any change to the scheduler to optimise a particular usage pattern is likely to cost us in performance for the general case
- you've said your patch only adds cost when the system usage pattern is the one you want to improve. The onus is on you to clearly demonstrate this (even if you're right, you need to shout it over and over to be heard over the screams of panic and fear)
- even if your patch costs the general case a mere 0.01 percent, you have to justify that this (negligible) performance loss is justified
- and thus you need to demonstrate that your patch fixes a real problem, and doesn't just provide a crutch for silly applications.
And it's the final point where I have my concerns. While I'm willing to sit back and acknowledge that you can keeps costs to a level which is sufficiently low that I won't care, I don't feel comfortable with providing crutches for silly applications.
And I really feel that the pipeline processing model, where you pass a big chunk of data between various processes, is fundamentally flawed. It's fine for things done at the command line, and for prototyping. But having a production application that is constantly shuffling big data between many processes is just plain silly.
I know why people do it, of course: modularity. And modularity is good, no question of it. But it can cost you (not always, a good design can give you modularity with no cost or negligible cost, whereas a dumb (or just "obvious") design can give high cost). The programmer has to decide whether the costs outweigh the benefits.
But if you opt for modularity, then don't expect me to be happy about losing even 0.01% of my performance for your design. Especially when I know that the cache polluting effects of shuffling data around will swamp whatever performance gains you hope to achieve.
[I can see I'm going into rant mode here] Maybe it's just me, but whenever I see people going down the pipeline processing path, I just shake my head in sadness, especially when I see them using it in production systems. Yeah, sure, it's easy to code up, and boy does it look cute when display with a nice GUI and you can see the data trickling between processing nodes. Spiffy!
But the operative word is "trickling". If you're lucky, each node will process long enough that cache refill times are lost in the noise. In which case, scheduler overheads are lost in the noise of cache refills.
But I still think it's a bad design. It looks neat on paper (and really pretty on that GUI), but it sucks in real life and reflects laziness on the part of the programmer.
I'm convinced you can modularise without the hideous cost of pipeline processing. Therefore any change that gives the illusion that pipeline processing is not that inefficient should be strongly discouraged.
[More off-topic ranting: sorry people] For the same reason, I'm still doing my real work in C and not C++. While C++ can make some (good) things much easier, it's just *too* seductive. Too much of a good thing is bad for you. Man, if feels good at the time, but it's stealing your soul. A trick here, a feature there, and suddenly your object code no longer fits on a CD-ROM. But hey! It was quicker to write. And it's object-oriented! Wow!
You can lead a horse to water, but you can't force it to drink.
And if people say that Java (or another flavour-of-the-month language) forces large run queues, and that application programmers can't help it, I'll just say Java is broken. I don't care how big it is, who wrote it, or who says it's so wonderful. If it spawns processes like the proverbial rabbit, it's flawed. Fix Java. BTW: don't flame me if Java doesn't do this. I'm just going from what I've seen on the list. I know bugger-all about Java, and that's just fine by me.
[Well, that was a good rant. That felt rather good, actually]
Regards,
Richard.... Old: rgooch@atnf.csiro.au Current: rgooch@ras.ucalgary.ca
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.rutgers.edu Please read the FAQ at http://www.tux.org/lkml/
| |