sched/membarrier: reduce the ability to hammer on sys_membarrier
commit 944d5fe50f3f03daacfea16300e656a1691c4a23 upstream.
On some systems, sys_membarrier can be very expensive, causing overall
slowdowns for everything. So put a lock on the path in order to
serialize the accesses to prevent the ability for this to be called at
too high of a frequency and saturate the machine.
Reviewed-and-tested-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Acked-by: Borislav Petkov <bp@alien8.de>
Fixes: 22e4ebb97582 ("membarrier: Provide expedited private command")
Fixes: c5f58bd58f43 ("membarrier: Provide GLOBAL_EXPEDITED command")
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
[ converted to explicit mutex_*() calls - cleanup.h is not in this stable
branch - gregkh ]
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 3cd139875e9a7688b3fc715264032620812a5fa3)
[vegard: fixed conflict due to missing commit
c5f58bd58f432be5d92df33c5458e0bcbee3aadf ("membarrier: Provide
GLOBAL_EXPEDITED command")]
Signed-off-by: Vegard Nossum <vegard.nossum@oracle.com>
diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c
index 9bcbacb..d936a13 100644
--- a/kernel/sched/membarrier.c
+++ b/kernel/sched/membarrier.c
@@ -30,6 +30,8 @@
(MEMBARRIER_CMD_SHARED | MEMBARRIER_CMD_PRIVATE_EXPEDITED \
| MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED)
+static DEFINE_MUTEX(membarrier_ipi_mutex);
+
static void ipi_mb(void *info)
{
smp_mb(); /* IPIs should be serializing but paranoid. */
@@ -64,6 +66,7 @@ static int membarrier_private_expedited(void)
fallback = true;
}
+ mutex_lock(&membarrier_ipi_mutex);
cpus_read_lock();
for_each_online_cpu(cpu) {
struct task_struct *p;
@@ -102,6 +105,7 @@ static int membarrier_private_expedited(void)
* rq->curr modification in scheduler.
*/
smp_mb(); /* exit from system call is not a mb */
+ mutex_unlock(&membarrier_ipi_mutex);
return 0;
}