Messages in this thread | | | From | (Larry McVoy) | Subject | Re: Process Migration on Linux - Impossible? | Date | Tue, 30 Sep 1997 18:59:52 -0700 |
| |
: I think that the SMP comparison is irrelevent to the generic process : migration case. Clearly, in an SMP box, if you have the resources : needed to keep a job in memory and run it, running it on the same CPU : is probably a win.
You neatly missed the point. If you can't make it work on an SMP, where the costs of dragging your context behind you is quite low, how do you make it work on a cluster?
The answer is: you don't move the processes very often. The point is that if you move them at startup time, it's trivial to have the OS to do it and you can get any needed further load balancing out of small SMP boxes.
If you can't see that, then answer this: if I load balance statically at startup time and dynamically at runtime (within an SMP), what added performance will I get from moving already running processes? Answer: extremely little.
Now consider the OS cost of creating a process remotely versus moving an existing process (and associated FS cache). It's the difference between rfork() and rexec(). It's obvious that one is trivial and the other is a bitch, right?
If you are in the least work for the most bang mode, then you don't do process migration.
: In a cluster of workstations, the decision to migrate a process is : done only when you (that is the migration algorithm) determines that : the process migration costs are outweighed by the improvement gained : by running on a relatively lightly loaded box. Migration allows you : to dynamically adjust to changing load and take advantage of that.
As I've repeatedly said, the dynamic changes are well handled by small SMPs. As has been repeatedly proven, moving an already started process is a lose almost 100% of the time.
: If you would like to read about actual measurements done on a real : system, try: : : A. Barak, R. Wheeler, MOSIX: An Integrated Multiprocessor UNIX, : Proceedings of 1989 Winter USENIX Technical Conference, San Diego, : February, 1989.
I've read it. And the references.
: We present several examples, doing a parallel "make" on a MOSIX : system was a joy.
I'm doing a parallel make on a cluster of boxes right now. You get 100% of the speedup from remote execution. I challenge you to show me a make that gets better throughput by migrating processes at any time other than exec time. Parallel make /does not/ require process migration of running processes. In fact, that is guarenteed to slow it down.
| |