Messages in this thread | | | Date | Tue, 12 Oct 2004 07:50:29 +0200 | From | Ingo Molnar <> | Subject | Re: [ANNOUNCE] Linux 2.6 Real Time Kernel |
| |
* Sven Dietrich <sdietrich@mvista.com> wrote:
> > But, there is a 'correctness' _minimum_ set of spinlocks that _must_ be > > raw spinlocks - this i tried to map in the -T4 patch. The patch does run > > on SMP systems for example. (it was developed as an SMP kernel - in fact > > i never compiled it as UP :-|.) If code has per-CPU or preemption > > assumptions then there is no choice but to make it a raw spinlock, until > > those assumptions are fixed.
> The grunt work is in identifying those problem areas and coming up > with elegant, low-impact solutions. RCU locks is one example as > mentioned before. We had a fix to serialize RCU access, but weren't > happy with that. We were hoping to get some input on this, but these > problems seem to show up more readily on slow systems (we are also > testing with a bunch of old P1, P2 and K6 boxes all far sub 1 GHz)
identifying problem areas is near 100% automatic if you look at -T5: all illegal sleeps and illegal smp_processor_id() assumptions are reported when they happen. That's how i identified & fixed the core 90 locks in the first wave, in just a couple of hours. The only minor annoyance when doing a conversion is the inflexibility of SPIN_LOCK_UNLOCKED and RW_LOCK_UNLOCKED initializer. If it werent for the initializers then a 'conversion' would be a matter of a 2-line change, the change of the prototype and the change of the definition. Now it's a 3-line change most of the time - still very fast and painless.
regarding RCU serialization - i think that is the way to go - i dont think there is any sensible way to extend RCU to a fully preempted model, RCU is all about per-CPU-ness and per-CPU-ness is quite limited in a fully preemptible model.
could you send those RCU patches (no matter how incomplete/broken)? It's the main issue that causes the dcache_lock to be raw still. (and a number of dependent locks: fs-writeback.c, inode.c, etc.) We can make those RCU changes not impact the normal !PREEMPT_REALTIME locking so it might have a chance for upstream merging as well.
> I was making this observation: One can't look at an arbitrary piece of > code and tell if it will be a spinlock or a mutex. One has to go look > elsewhere. In the spin_undefs case one can look the top of the file > and check for it, in the LOCK_OPS case, you have to call up the data > structure declaration.
ok, i now understand what you mean. The way i drove it wasnt really via code review but via: 'compile kernel, look at the bootlogs, fix the first lock reported, repeat' iterations. This was much easier and much more reliable than trying to figure out lock dependencies from the source. The turnaround for a single lock was 2-3 minutes in the typical case, allowing the conversion of 90 locks in a couple of hours.
> > > There are a whole lot of caveats and race conditions that have not yet > > > been unearthed by the brief LKML testing. [...] > > > > actually, have you tried your patchset on an SMP box? As far as i can > > see the locking in it ignores SMP issues _completely_, which makes the > > choice of locks much less useful. > > We stated that its been tested minimally on SMP. That means we have > had it up and running and found it to be unstable. I fully agree that > SMP is the superset to get it working on, and that PMutex is not > perfect at this point.
it's not just the problem of PMutex - i believe it's mainly the plain inadequacy of the 30 raw locks you have identified - and identifying the locks is the bigger work, not the semaphore implementation. I'm now at 90 locks (20% of all locking in this .config) and that's just to quiet the DEBUG_PREEMPT violations on my testboxes.
and no matter how well UP works, to fix SMP one has to 'cover' all the necessary locks first before fixing it, which (drastic) increase in raw locks invalidates most of the UP efforts of getting rid of raw locks. That's why i decided to go for SMP primarily - didnt see much point in going for UP.
> We will take a look at the T5 patch and see what we can do about PI > for the system semaphore, but I am not sure how portable it would be > without also touching the assembly. FWIW PMutex is already based in > part on the system semaphore, so we might get similar problems when > porting elsewhere.
there are in-C variants of Linux mutexes and rw-semaphores in the kernel source, so worst-case we could just make use of them in the PREEMPT_REALTIME case. I'm not a big fan of assembly optimizations (or having to touch assembly optimizations) at an early stage like this.
Ingo - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
| |