lkml.org 
[lkml]   [2017]   [Jul]   [17]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
/
Date
From
SubjectRe: [PATCH] nvme: Acknowledge completion queue on each iteration
On 2017-07-17 18:56, Keith Busch wrote:
> On Mon, Jul 17, 2017 at 06:46:11PM -0400, Sinan Kaya wrote:
>> Hi Keith,
>>
>> On 7/17/2017 6:45 PM, Keith Busch wrote:
>> > On Mon, Jul 17, 2017 at 06:36:23PM -0400, Sinan Kaya wrote:
>> >> Code is moving the completion queue doorbell after processing all completed
>> >> events and sending callbacks to the block layer on each iteration.
>> >>
>> >> This is causing a performance drop when a lot of jobs are queued towards
>> >> the HW. Move the completion queue doorbell on each loop instead and allow new
>> >> jobs to be queued by the HW.
>> >
>> > That doesn't make sense. Aggregating doorbell writes should be much more
>> > efficient for high depth workloads.
>> >
>>
>> Problem is that code is throttling the HW as HW cannot queue more
>> completions until
>> SW get a chance to clear it.
>>
>> As an example:
>>
>> for each in N
>> (
>> blk_layer()
>> )
>> ring door bell
>>
>> HW cannot queue new job until N x blk_layer operations are processed
>> and queue
>> element ownership is passed to the HW after the loop. HW is just
>> sitting idle
>> there if no queue entries are available.
>
> If no completion queue entries are available, then there can't possibly
> be any submission queue entries for the HW to work on either.

Maybe, I need to understand the design better. I was curious why
completion and submission queues were protected by a single lock causing
lock contention.

I was treating each queue independently. I have seen slightly better
performance by an early doorbell. That was my explanation.

\
 
 \ /
  Last update: 2017-07-18 01:07    [W:0.057 / U:4.384 seconds]
©2003-2017 Jasper Spaans. hosted at Digital OceanAdvertise on this site