Messages in this thread | | | Subject | Re: [PATCH] latency improvement in __smp_call_single_queue | From | George Prekas <> | Date | Thu, 24 Sep 2020 10:04:43 -0500 |
| |
On 9/24/2020 3:42 AM, peterz@infradead.org wrote: > On Wed, Sep 23, 2020 at 10:00:41AM -0500, George Prekas wrote: >> If an interrupt arrives between llist_add and >> send_call_function_single_ipi in the following code snippet, then the >> remote CPU will not receive the IPI in a timely manner and subsequent >> SMP calls even from other CPUs for other functions will be delayed: >> >> if (llist_add(node, &per_cpu(call_single_queue, cpu))) >> send_call_function_single_ipi(cpu); >> >> Note: llist_add returns 1 if it was empty before the operation. >> >> CPU 0 | CPU 1 | CPU 2 >> __smp_call_single_q(2,f1) | __smp_call_single_q(2,f2) | >> llist_add returns 1 | | >> interrupted | llist_add returns 0 | >> ... | branch not taken | >> ... | | >> resumed | | >> send_call_function_single_ipi | | >> | | f1 >> | | f2 >> >> The call from CPU 1 for function f2 will be delayed because CPU 0 was >> interrupted. > > Do you happen to have any actual numbers and a use-case where this was > relevant?
Hi Peter,
I encountered this problem while developing a device driver that used smp_call_function_single to communicate with other cores. I observed latency spikes and after investigation I figured out the problem described above.
I have written a simple device driver to validate the above fix. It does smp_call_function_single and measures the latency. I can post it here if it is appropriate. The latency impact is equal to the duration of the CPU 0's interruption.
-- George
| |