lkml.org 
[lkml]   [2000]   [Jan]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: Auto-Adaptive scheduler - Final chapter ( the numbers ) ...
Davide Libenzi writes:
> Hi Richard,
>
> > Yeah, I'm reading it (barely, I've been deleting most of my email
> > because I've been on holidays). The PMC (Performance Monitoring
> > Counter) patch I wrote is available at:
> > http://www.atnf.csiro.au/~rgooch/linux/
>
> can You give me a hint on how to use int the better way Your PMC
> code ?

The page with the kernel patches also has a link to the user-space
API and command-line utilities. That should give you all you need.

> How is the noise figure induced onto the measure ?

What do you mean?

> What is Your impression of the patch ( if You've seen ) ?

I've not seen it. I've had to delete most of my email this month as
I've been on holidays. I either had to delete or go mad. I chose
sanity. For the same reason, this reply is late. Sorry.

> It has also the "side effect" of keeping RT task onto the higher
> cluster for a better RT latency.

From what I've seen in the discussion, your patch enables your new
scheduler after many processes have been on the run queue for a while,
correct? That's not the same as the case that was interesting me when
I was doing my RT run queue work.

That case that interests me is when you have a number of SCHED_OTHER
processes on the run queue (usually due to some event waking up
several processes), and then a new event wakes up a RT process (either
an interrupt or another RT process).

With a large run queue, there is a significant slowdown in waking the
RT process. What I did was to create a separate RT run queue so that
RT context switch times were (more) isolated from SCHED_OTHER.
The (real) application where this mattered was an embedded system that
controlled hardware, plus had a serial interface to display status. The
processes that controlled hardware (via device drivers) were RT
whereas the display processes were SCHED_OTHER.

So it doesn't look like your patch does anything for the case that I
care about, because there won't be a large number of processes on the
run queue for long times. That doesn't mean your patch is bad, it just
doesn't help *me*.

However, without having looked at your patch (but having glanced at
the general response on the list), I'm skeptical about your patch, for
these reasons:

- any change to the scheduler to optimise a particular usage pattern
is likely to cost us in performance for the general case

- you've said your patch only adds cost when the system usage pattern
is the one you want to improve. The onus is on you to clearly
demonstrate this (even if you're right, you need to shout it over
and over to be heard over the screams of panic and fear)

- even if your patch costs the general case a mere 0.01 percent, you
have to justify that this (negligible) performance loss is justified

- and thus you need to demonstrate that your patch fixes a real
problem, and doesn't just provide a crutch for silly applications.

And it's the final point where I have my concerns. While I'm willing
to sit back and acknowledge that you can keeps costs to a level which
is sufficiently low that I won't care, I don't feel comfortable with
providing crutches for silly applications.

And I really feel that the pipeline processing model, where you pass a
big chunk of data between various processes, is fundamentally flawed.
It's fine for things done at the command line, and for prototyping.
But having a production application that is constantly shuffling big
data between many processes is just plain silly.

I know why people do it, of course: modularity. And modularity is
good, no question of it. But it can cost you (not always, a good
design can give you modularity with no cost or negligible cost,
whereas a dumb (or just "obvious") design can give high cost). The
programmer has to decide whether the costs outweigh the benefits.

But if you opt for modularity, then don't expect me to be happy about
losing even 0.01% of my performance for your design. Especially when I
know that the cache polluting effects of shuffling data around will
swamp whatever performance gains you hope to achieve.

[I can see I'm going into rant mode here]
Maybe it's just me, but whenever I see people going down the pipeline
processing path, I just shake my head in sadness, especially when I
see them using it in production systems. Yeah, sure, it's easy to code
up, and boy does it look cute when display with a nice GUI and you can
see the data trickling between processing nodes. Spiffy!

But the operative word is "trickling". If you're lucky, each node will
process long enough that cache refill times are lost in the noise. In
which case, scheduler overheads are lost in the noise of cache
refills.

But I still think it's a bad design. It looks neat on paper (and
really pretty on that GUI), but it sucks in real life and reflects
laziness on the part of the programmer.

I'm convinced you can modularise without the hideous cost of pipeline
processing. Therefore any change that gives the illusion that pipeline
processing is not that inefficient should be strongly discouraged.

[More off-topic ranting: sorry people]
For the same reason, I'm still doing my real work in C and not C++.
While C++ can make some (good) things much easier, it's just *too*
seductive. Too much of a good thing is bad for you. Man, if feels good
at the time, but it's stealing your soul. A trick here, a feature
there, and suddenly your object code no longer fits on a CD-ROM. But
hey! It was quicker to write. And it's object-oriented! Wow!

You can lead a horse to water, but you can't force it to drink.

And if people say that Java (or another flavour-of-the-month language)
forces large run queues, and that application programmers can't help
it, I'll just say Java is broken. I don't care how big it is, who
wrote it, or who says it's so wonderful. If it spawns processes like
the proverbial rabbit, it's flawed. Fix Java.
BTW: don't flame me if Java doesn't do this. I'm just going from what
I've seen on the list. I know bugger-all about Java, and that's just
fine by me.

[Well, that was a good rant. That felt rather good, actually]

Regards,

Richard....
Old: rgooch@atnf.csiro.au
Current: rgooch@ras.ucalgary.ca

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

\
 
 \ /
  Last update: 2005-03-22 13:56    [W:0.411 / U:0.500 seconds]
©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site