Merge tag 'irqchip-5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms into irq/core

Pull irqchip/genirq updates from Marc Zyngier:

 * Core code update:

  - Non-SMP IRQ affinity fixes, allowing UP kernel to behave similarly
    to SMP ones for the purpose of interrupt affinity

  - Let irq_set_chip_handler_name_locked() take a const struct irq_chip *

  - Tidy-up the NOMAP irqdomain API variant

  - Teach action_show() to use for_each_action_of_desc()

  - Make irq_chip_request_resources_parent() allow the parent callback
    to be optional

  - Remove dynamic allocations from populate_parent_alloc_arg()

 * New drivers:

  - Merge the long awaited IRQ support for the LoongArch architecture,
    with the provisional ACPICA update (to be reverted once the official
    support lands)

  - New Renesas RZ/G2L IRQC driver, equipped with its companion GPIO
    driver

 * Driver updates

  - Optimise the hot path operations for the SiFive PLIC, trading the
    locking for per-CPU priority masking masking operations which are
    apparently faster

  - Work around broken PLIC implementations that deal pretty badly with
    edge-triggered interrupts. Flag two implementations as affected.

  - Simplify the irq-stm32-exti driver, particularly the table that
    remaps the interrupts from exti to the GIC, reducing the memory usage

  - Convert the ocelot irq_chip to being immutable

  - Check ioremap() return value in the MIPS GIC driver

  - Move MMP driver init function declarations into the common .h

  - The obligatory typo fixes

Link: https://lore.kernel.org/all/20220727192356.1860546-1-maz@kernel.org

1 files changed, 33 insertions, 3 deletions

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index bfa7452ca92e..da0bf6fe9ecd 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -4798,25 +4798,55 @@ static void do_balance_callbacks(struct rq *rq, struct callback_head *head)
 
 static void balance_push(struct rq *rq);
 
+/*
+ * balance_push_callback is a right abuse of the callback interface and plays
+ * by significantly different rules.
+ *
+ * Where the normal balance_callback's purpose is to be ran in the same context
+ * that queued it (only later, when it's safe to drop rq->lock again),
+ * balance_push_callback is specifically targeted at __schedule().
+ *
+ * This abuse is tolerated because it places all the unlikely/odd cases behind
+ * a single test, namely: rq->balance_callback == NULL.
+ */
 struct callback_head balance_push_callback = {
 	.next = NULL,
 	.func = (void (*)(struct callback_head *))balance_push,
 };
 
-static inline struct callback_head *splice_balance_callbacks(struct rq *rq)
+static inline struct callback_head *
+__splice_balance_callbacks(struct rq *rq, bool split)
 {
 	struct callback_head *head = rq->balance_callback;
 
+	if (likely(!head))
+		return NULL;
+
 	lockdep_assert_rq_held(rq);
-	if (head)
+	/*
+	 * Must not take balance_push_callback off the list when
+	 * splice_balance_callbacks() and balance_callbacks() are not
+	 * in the same rq->lock section.
+	 *
+	 * In that case it would be possible for __schedule() to interleave
+	 * and observe the list empty.
+	 */
+	if (split && head == &balance_push_callback)
+		head = NULL;
+	else
 		rq->balance_callback = NULL;
 
 	return head;
 }
 
+static inline struct callback_head *splice_balance_callbacks(struct rq *rq)
+{
+	return __splice_balance_callbacks(rq, true);
+}
+
 static void __balance_callbacks(struct rq *rq)
 {
-	do_balance_callbacks(rq, splice_balance_callbacks(rq));
+	do_balance_callbacks(rq, __splice_balance_callbacks(rq, false));
 }
 
 static inline void balance_callbacks(struct rq *rq, struct callback_head *head)