Messages in this thread |  | | Date | Mon, 20 Nov 2006 08:59:23 -0800 | From | "Paul E. McKenney" <> | Subject | Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync |
| |
On Mon, Nov 20, 2006 at 08:15:14AM +0100, Jens Axboe wrote: > On Sun, Nov 19 2006, Paul E. McKenney wrote: > > On Sat, Nov 18, 2006 at 09:46:24PM +0300, Oleg Nesterov wrote: > > > On 11/17, Paul E. McKenney wrote: > > > > > > > > Oleg, any thoughts about Jens's optimization? He would code something > > > > like: > > > > > > > > if (srcu_readers_active(&my_srcu)) > > > > synchronize_srcu(); > > > > else > > > > smp_mb(); > > > > > > Well, this is clearly racy, no? I am not sure, but may be we can do > > > > > > smp_mb(); > > > if (srcu_readers_active(&my_srcu)) > > > synchronize_srcu(); > > > > > > in this case we also need to add 'smp_mb()' into srcu_read_lock() after > > > 'atomic_inc(&sp->hardluckref)'. > > > > > > > However, he is doing ordered I/O requests rather than protecting data > > > > structures. > > > > > > Probably this makes a difference, but I don't understand this. > > > > OK, one hypothesis here... > > > > The I/Os must be somehow explicitly ordered to qualify > > for I/O-barrier separation. If two independent processes > > issue I/Os concurrently with a third process doing an > > I/O barrier, the I/O barrier is free to separate the > > two concurrent I/Os or not, on its whim. > > > > Jens, is the above correct? If so, what would the two processes > > That's completely correct, hence my somewhat relaxed approach with SRCU.
OK, less scary in that case. ;-)
> > need to do in order to ensure that their I/O was considered to be > > ordered with respect to the I/O barrier? Here are some possibilities: > > If we consider the barrier a barrier in a certain stream of requests, > it is the responsibility of the issuer of that barrier to ensure that > the queueing is ordered. So if two "unrelated" streams of requests with > barriers hit __make_request() at the same time, we don't go to great > lengths to ensure who gets there firt.
So the "preceding" requests have to have completed their I/O system calls? If this is the case, does this include normal (non-direct/raw) writes and asynchronous reads? My guess is that it would include asynchronous I/O, but not buffered writes.
> > 1. I/O barriers apply only to preceding and following I/Os from > > the process issuing the I/O barrier. > > > > 2. As for #1 above, but restricted to task rather than process. > > > > 3. I/O system calls that have completed are ordered by the > > barrier to precede I/O system calls that have not yet > > started, but I/O system calls still in flight could legally > > land on either side of the concurrently executing I/O > > barrier. > > > > 4. Something else entirely? > > > > Given some restriction like one of the above, it is entirely possible > > that we don't even need the memory barrier... > > 3 is the closest. The request queue doesn't really know the scope of the > barrier, it has to rely on the issuer getting it right. If you have two > competing processes issuing io and process A relies on process B issuing > a barrier, they have to synchronize that between them. Normally that is > not a problem, since that's how the file systems always did io before > barriers on items that need to be on disk (it was a serialization point > anyway, it's just a stronger one now).
So something like a user-level mutex or atomic instructions must be used by the tasks doing the pre-barrier I/Os to announce that these I/Os have been started in the kernel.
> That said, I think the > > smp_mb(); > if (srcu_readers_active(sp)) > synchronize_srcu(); > > makes the most sense.
If the user-level tasks/threads/processes must explicitly synchronize, and if the pre-barrier I/O-initation syscalls have to have completed, then I am not sure that the smp_mb() is needed. Seems like the queuing mechanisms in the syscall and the user-level synchronization would have supplied the needed memory barriers. Or are you using some extremely lightweight user-level synchronization?
Thanx, Paul - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
|  |