diff options
| author | Xin Li (Intel) <xin@zytor.com> | 2024-08-22 00:39:04 -0700 |
|---|---|---|
| committer | Thomas Gleixner <tglx@linutronix.de> | 2024-08-25 19:23:00 +0200 |
| commit | 0dfac6f267fa091aa348c6a6742b463c9e7c98e3 (patch) | |
| tree | 753793a5adee543d9c67738db958347c71392cf1 /arch | |
| parent | 723edbd2ca5fb4c78ac4a5644511c63895fd1c57 (diff) | |
x86/entry: Test ti_work for zero before processing individual bits
In most cases, ti_work values passed to arch_exit_to_user_mode_prepare()
are zeros, e.g., 99% in kernel build tests. So an obvious optimization is
to test ti_work for zero before processing individual bits in it.
Omit the optimization when FPU debugging is enabled, otherwise the
FPU consistency check is never executed.
Intel 0day tests did not find a perfermance regression with this change.
Suggested-by: H. Peter Anvin (Intel) <hpa@zytor.com>
Signed-off-by: Xin Li (Intel) <xin@zytor.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/all/20240822073906.2176342-2-xin@zytor.com
Diffstat (limited to 'arch')
| -rw-r--r-- | arch/x86/include/asm/entry-common.h | 10 |
1 files changed, 8 insertions, 2 deletions
diff --git a/arch/x86/include/asm/entry-common.h b/arch/x86/include/asm/entry-common.h index fb2809b20b0a..db970828f385 100644 --- a/arch/x86/include/asm/entry-common.h +++ b/arch/x86/include/asm/entry-common.h @@ -44,8 +44,7 @@ static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs) } #define arch_enter_from_user_mode arch_enter_from_user_mode -static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, - unsigned long ti_work) +static inline void arch_exit_work(unsigned long ti_work) { if (ti_work & _TIF_USER_RETURN_NOTIFY) fire_user_return_notifiers(); @@ -56,6 +55,13 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, fpregs_assert_state_consistent(); if (unlikely(ti_work & _TIF_NEED_FPU_LOAD)) switch_fpu_return(); +} + +static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, + unsigned long ti_work) +{ + if (IS_ENABLED(CONFIG_X86_DEBUG_FPU) || unlikely(ti_work)) + arch_exit_work(ti_work); #ifdef CONFIG_COMPAT /* |