Messages in this thread | | | Subject | Re: [PATCH v5 5/6] nbd: convert to use blk_mq_find_and_get_req() | From | "yukuai (C)" <> | Date | Tue, 14 Sep 2021 17:12:24 +0800 |
| |
On 2021/09/14 17:08, yukuai (C) wrote: > On 2021/09/14 15:46, Ming Lei wrote: >> On Tue, Sep 14, 2021 at 03:13:38PM +0800, yukuai (C) wrote: >>> On 2021/09/14 14:44, Ming Lei wrote: >>>> On Tue, Sep 14, 2021 at 11:11:06AM +0800, yukuai (C) wrote: >>>>> On 2021/09/14 9:11, Ming Lei wrote: >>>>>> On Thu, Sep 09, 2021 at 10:12:55PM +0800, Yu Kuai wrote: >>>>>>> blk_mq_tag_to_rq() can only ensure to return valid request in >>>>>>> following situation: >>>>>>> >>>>>>> 1) client send request message to server first >>>>>>> submit_bio >>>>>>> ... >>>>>>> blk_mq_get_tag >>>>>>> ... >>>>>>> blk_mq_get_driver_tag >>>>>>> ... >>>>>>> nbd_queue_rq >>>>>>> nbd_handle_cmd >>>>>>> nbd_send_cmd >>>>>>> >>>>>>> 2) client receive respond message from server >>>>>>> recv_work >>>>>>> nbd_read_stat >>>>>>> blk_mq_tag_to_rq >>>>>>> >>>>>>> If step 1) is missing, blk_mq_tag_to_rq() will return a stale >>>>>>> request, which might be freed. Thus convert to use >>>>>>> blk_mq_find_and_get_req() to make sure the returned request is not >>>>>>> freed. >>>>>> >>>>>> But NBD_CMD_INFLIGHT has been added for checking if the reply is >>>>>> expected, do we still need blk_mq_find_and_get_req() for covering >>>>>> this issue? BTW, request and its payload is pre-allocated, so there >>>>>> isn't real use-after-free. >>>>> >>>>> Hi, Ming >>>>> >>>>> Checking NBD_CMD_INFLIGHT relied on the request founded by tag is >>>>> valid, >>>>> not the other way round. >>>>> >>>>> nbd_read_stat >>>>> req = blk_mq_tag_to_rq() >>>>> cmd = blk_mq_rq_to_pdu(req) >>>>> mutex_lock(cmd->lock) >>>>> checking NBD_CMD_INFLIGHT >>>> >>>> Request and its payload is pre-allocated, and either req->ref or >>>> cmd->lock can >>>> serve the same purpose here. Once cmd->lock is held, you can check >>>> if the cmd is >>>> inflight or not. If it isn't inflight, just return -ENOENT. Is there >>>> any >>>> problem to handle in this way? >>> >>> Hi, Ming >>> >>> in nbd_read_stat: >>> >>> 1) get a request by tag first >>> 2) get nbd_cmd by the request >>> 3) hold cmd->lock and check if cmd is inflight >>> >>> If we want to check if the cmd is inflight in step 3), we have to do >>> setp 1) and 2) first. As I explained in patch 0, blk_mq_tag_to_rq() >>> can't make sure the returned request is not freed: >>> >>> nbd_read_stat >>> blk_mq_sched_free_requests >>> blk_mq_free_rqs >>> blk_mq_tag_to_rq >>> -> get rq before clear mapping >>> blk_mq_clear_rq_mapping >>> __free_pages -> rq is freed >>> blk_mq_request_started -> UAF >> >> If the above can happen, blk_mq_find_and_get_req() may not fix it too, >> just > > Hi, Ming > > Why can't blk_mq_find_and_get_req() fix it? I can't think of any > scenario that might have problem currently. > >> wondering why not take the following simpler way for avoiding the UAF? >> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c >> index 5170a630778d..dfa5cce71f66 100644 >> --- a/drivers/block/nbd.c >> +++ b/drivers/block/nbd.c >> @@ -795,9 +795,13 @@ static void recv_work(struct work_struct *work) >> work); >> struct nbd_device *nbd = args->nbd; >> struct nbd_config *config = nbd->config; >> + struct request_queue *q = nbd->disk->queue; >> struct nbd_cmd *cmd; >> struct request *rq; >> + if (!percpu_ref_tryget(&q->q_usage_counter)) >> + return; >> + > > We can't make sure freeze_queue is called before this, thus this approch > can't fix the problem, right? > nbd_read_stat > blk_mq_tag_to_rq > elevator_switch > blk_mq_freeze_queue(q); > elevator_switch_mq > elevator_exit > blk_mq_sched_free_requests > blk_mq_request_started -> UAF
Hi, Ming
I forgot that if percpu_ref_tryget succeed here, blk_mq_free_queue() will block untill blk_queue_exit() in nbd_read_stat().
Thanks, Kuai > > Thanks, > Kuai > >> while (1) { >> cmd = nbd_read_stat(nbd, args->index); >> if (IS_ERR(cmd)) { >> @@ -813,6 +817,7 @@ static void recv_work(struct work_struct *work) >> if (likely(!blk_should_fake_timeout(rq->q))) >> blk_mq_complete_request(rq); >> } >> + blk_queue_exit(q); >> nbd_config_put(nbd); >> atomic_dec(&config->recv_threads); >> wake_up(&config->recv_wq); >> >> Thanks, >> Ming >> >> . >>
| |