Messages in this thread Patch in this message | | | Date | Sun, 5 Jun 2011 08:56:33 +0200 | From | Jens Axboe <> | Subject | Re: Mysterious CFQ crash and RCU |
| |
On 2011-06-05 00:48, Paul Bolle wrote: > I think I finally found it! > > The culprit seems to be io_context.ioc_data (not the most clear of > names!). It seems to be a single entry "last-hit cache" of an hlist > called cic_list. (There are three, subtly different, cic_lists in the > CFQ code!) It is not entirely clear, but that last-hit cache can get out > of sync with the hlist it is supposed to cache. My guess it that every > now and then a member of the hlist gets deleted while it's still in that > (single entry) cache. If it then gets retrieved from that cache it > already points to poisoned memory. For some strange reason this only > results in an Oops if one or more debugging options are set (as are set > in the Fedora Rawhide non-stable kernels that I ran into this). I have > no clue whatsoever, why that is ... > > Anyhow, after ripping out ioc_data this bug seems to have disappeared! > Jens, Vivek, could you please have a look at this? In the mean time I > hope to pinpoint this issue and draft a small patch to really solve it > (ie, not by simply ripping out ioc_data).
Does this fix it? It will introduce a hierarchy that is queue -> ioc lock, but as far as I can remember (and tell from a quick look), we don't have any dependencies on that order of locking at this moment. So should be OK.
diff --git a/block/cfq-iosched.c b/block/cfq-iosched.c index 3c7b537..fa7ef54 100644 --- a/block/cfq-iosched.c +++ b/block/cfq-iosched.c @@ -2772,8 +2772,11 @@ static void __cfq_exit_single_io_context(struct cfq_data *cfqd, smp_wmb(); cic->key = cfqd_dead_key(cfqd); - if (ioc->ioc_data == cic) + if (ioc->ioc_data == cic) { + spin_lock(&ioc->lock); rcu_assign_pointer(ioc->ioc_data, NULL); + spin_unlock(&ioc->lock); + } if (cic->cfqq[BLK_RW_ASYNC]) { cfq_exit_cfqq(cfqd, cic->cfqq[BLK_RW_ASYNC]); -- Jens Axboe
| |