pm24.git, branch rust-6.9

rust: upgrade to Rust 1.76.0

2024-02-29T21:18:05Z

This is the next upgrade to the Rust toolchain, from 1.75.0 to 1.76.0 (i.e. the latest) [1]. See the upgrade policy [2] and the comments on the first upgrade in commit 3ed03f4da06e ("rust: upgrade to Rust 1.68.2"). # Unstable features No unstable features that we use were stabilized in Rust 1.76.0. The only unstable features allowed to be used outside the `kernel` crate are still `new_uninit,offset_of`, though other code to be upstreamed may increase the list. Please see [3] for details. # Required changes `rustc` (and others) now warns when it cannot connect to the Make jobserver, thus mark those invocations as recursive as needed. Please see the previous commit for details. # Other changes Rust 1.76.0 does not emit the `.debug_pub{names,types}` sections anymore for DWARFv4 [4][5]. For instance, in the uncompressed debug info case, this debug information took: samples/rust/rust_minimal.o ~64 KiB (~18% of total object size) rust/kernel.o ~92 KiB (~15%) rust/core.o ~114 KiB ( ~5%) In the compressed debug info (zlib) case: samples/rust/rust_minimal.o ~11 KiB (~6%) rust/kernel.o ~17 KiB (~5%) rust/core.o ~21 KiB (~1.5%) In addition, the `rustc_codegen_gcc` backend now does not emit the `.eh_frame` section when compiling under `-Cpanic=abort` [6], thus removing the need for the patch in the CI to compile the kernel [7]. Moreover, it also now emits the `.comment` section too [6]. # `alloc` upgrade and reviewing The vast majority of changes are due to our `alloc` fork being upgraded at once. There are two kinds of changes to be aware of: the ones coming from upstream, which we should follow as closely as possible, and the updates needed in our added fallible APIs to keep them matching the newer infallible APIs coming from upstream. Instead of taking a look at the diff of this patch, an alternative approach is reviewing a diff of the changes between upstream `alloc` and the kernel's. This allows to easily inspect the kernel additions only, especially to check if the fallible methods we already have still match the infallible ones in the new version coming from upstream. Another approach is reviewing the changes introduced in the additions in the kernel fork between the two versions. This is useful to spot potentially unintended changes to our additions. To apply these approaches, one may follow steps similar to the following to generate a pair of patches that show the differences between upstream Rust and the kernel (for the subset of `alloc` we use) before and after applying this patch: # Get the difference with respect to the old version. git -C rust checkout $(linux/scripts/min-tool-version.sh rustc) git -C linux ls-tree -r --name-only HEAD -- rust/alloc | cut -d/ -f3- | grep -Fv README.md | xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH git -C linux diff --patch-with-stat --summary -R > old.patch git -C linux restore rust/alloc # Apply this patch. git -C linux am rust-upgrade.patch # Get the difference with respect to the new version. git -C rust checkout $(linux/scripts/min-tool-version.sh rustc) git -C linux ls-tree -r --name-only HEAD -- rust/alloc | cut -d/ -f3- | grep -Fv README.md | xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH git -C linux diff --patch-with-stat --summary -R > new.patch git -C linux restore rust/alloc Now one may check the `new.patch` to take a look at the additions (first approach) or at the difference between those two patches (second approach). For the latter, a side-by-side tool is recommended. Link: https://github.com/rust-lang/rust/blob/stable/RELEASES.md#version-1760-2024-02-08 [1] Link: https://rust-for-linux.com/rust-version-policy [2] Link: https://github.com/Rust-for-Linux/linux/issues/2 [3] Link: https://github.com/rust-lang/compiler-team/issues/688 [4] Link: https://github.com/rust-lang/rust/pull/117962 [5] Link: https://github.com/rust-lang/rust/pull/118068 [6] Link: https://github.com/Rust-for-Linux/ci-rustc_codegen_gcc [7] Tested-by: Boqun Feng Reviewed-by: Alice Ryhl Link: https://lore.kernel.org/r/20240217002638.57373-2-ojeda@kernel.org Signed-off-by: Miguel Ojeda

kbuild: mark `rustc` (and others) invocations as recursive

2024-02-29T21:16:38Z

`rustc` (like Cargo) may take advantage of the jobserver at any time (e.g. for backend parallelism, or eventually frontend too). In the kernel, we call `rustc` with `-Ccodegen-units=1` (and `-Zthreads` is 1 so far), so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a warning is emitted by `rustc` [1] when it cannot connect to the jobserver it was passed (in many cases, but not all: compiling and `--print sysroot` do, but `--version` does not). And given GNU Make always passes the jobserver in the environment variable (even when a line is deemed non-recursive), `rustc` will end up complaining about it (in particular in Make 4.3 where there is only the simple pipe jobserver style). One solution is to remove the jobserver from `MAKEFLAGS`. However, we can mark the lines with calls to `rustc` (and Cargo) as recursive, which looks simpler. This is being documented as a recommendation in `rustc` [2] and allows us to be ready for the time we may use parallelism inside `rustc` (potentially now, if a user passes `-Zthreads`). Thus do so. Similarly, do the same for `rustdoc` and `cargo` calls. Finally, there is one case that the solution does not cover, which is the `$(shell ...)` call we have. Thus, for that one, set an empty `MAKEFLAGS` environment variable. Link: https://github.com/rust-lang/rust/issues/120515 [1] Acked-by: Masahiro Yamada Link: https://github.com/rust-lang/rust/pull/121564 [2] Link: https://lore.kernel.org/r/20240217002638.57373-1-ojeda@kernel.org [ Reworded to add link to PR documenting the recommendation. ] Signed-off-by: Miguel Ojeda

rust: add `container_of!` macro

2024-02-25T18:17:31Z

This macro is used to obtain a pointer to an entire struct when given a pointer to a field in that struct. Signed-off-by: Wedson Almeida Filho Reviewed-by: Alice Ryhl Tested-by: Alice Ryhl Signed-off-by: Matt Gilbride Link: https://lore.kernel.org/r/20240219-b4-rbtree-v2-1-0b113aab330d@google.com Signed-off-by: Miguel Ojeda

rust: str: implement `Display` and `Debug` for `BStr`

2024-02-25T18:17:31Z

Currently, `BStr` is just a type alias of `[u8]`, limiting its representation to a byte list rather than a character list, which is not ideal for printing and debugging. Implement `Display` and `Debug` traits for `BStr` to facilitate easier printing and debugging. Also, for this purpose, change `BStr` from a type alias of `[u8]` to a struct wrapper of `[u8]`. Co-developed-by: Virgile Andreani Signed-off-by: Virgile Andreani Signed-off-by: Yutaro Ohno Reviewed-by: Alice Ryhl Link: https://lore.kernel.org/r/ZcSlGMGP-e9HqybA@ohnotp [ Formatted code comment. ] Signed-off-by: Miguel Ojeda

rust: module: place generated init_module() function in .init.text

2024-02-25T18:17:31Z

Currently Rust kernel modules have their init code placed in the `.text` section of the .ko file. I don't think this causes any real problems for Rust modules as long as all code called during initialization lives in `.text`. However, if a Rust `init_module()` function (that lives in `.text`) calls a function marked with `__init` (in C) or `#[link_section = ".init.text"]` (in Rust), then a warning is generated by modpost because that function lives in `.init.text`. For example: WARNING: modpost: fs/bcachefs/bcachefs: section mismatch in reference: init_module+0x6 (section: .text) -> _RNvXCsj7d3tFpT5JS_15bcachefs_moduleNtB2_8BcachefsNtCsjDtqRIL3JAG_6kernel6Module4init (section: .init.text) I ran into this while experimenting with converting the bcachefs kernel module from C to Rust. The module's `init()`, written in Rust, calls C functions like `bch2_vfs_init()` which are placed in `.init.text`. This patch places the macro-generated `init_module()` Rust function in the `.init.text` section. It also marks `init_module()` as unsafe--now it may not be called after module initialization completes because it may be freed already. Note that this is not enough on its own to actually get all the module initialization code in that section. The module author must still add the `#[link_section = ".init.text"]` attribute to the Rust `init()` in the `impl kernel::Module` block in order to then call `__init` functions. However, this patch enables module authors do so, when previously it would not be possible (without warnings). Signed-off-by: Thomas Bertschinger Reviewed-by: Martin Rodriguez Reboredo Reviewed-by: Alice Ryhl Link: https://lore.kernel.org/r/20240206153806.567055-1-tahbertschinger@gmail.com [ Reworded title to add prefix. ] Signed-off-by: Miguel Ojeda

rust: types: add `try_from_foreign()` method

2024-02-25T18:17:31Z

Currently `ForeignOwnable::from_foreign()` only works for non-null pointers for the existing `impl`s (e.g. `Box`, `Arc`). In turn, this means callers may write code like: ```rust // `p` is a pointer that may be null. if p.is_null() { None } else { unsafe { Some(Self::from_foreign(ptr)) } } ``` Add a `try_from_foreign()` method to the trait with a default implementation that returns `None` if `ptr` is null, otherwise `Some(from_foreign(ptr))`, so that it can be used by callers instead. Link: https://github.com/Rust-for-Linux/linux/issues/1057 Signed-off-by: Obei Sideg Reviewed-by: Alice Ryhl Reviewed-by: Trevor Gross Link: https://lore.kernel.org/r/0100018d53f737f8-80c1fe97-0019-40d7-ab69-b1b192785cd7-000000@email.amazonses.com [ Fixed intra-doc links, improved `SAFETY` comment and reworded commit. ] Signed-off-by: Miguel Ojeda

docs: rust: Add description of Rust documentation test as KUnit ones

2024-02-18T20:22:27Z

Rust documentation tests are automatically converted into KUnit tests. The commit adding this feature commit a66d733da801 ("rust: support running Rust documentation tests as KUnit ones") from Miguel has a very nice commit message with a lot details for this. To not 'hide' that just in a commit message, pick the main parts of it and add it to the documentation. And add a short info how to enable this. While adding this, improve the structure of the sections. Reviewed-by: David Gow Reviewed-by: Alice Ryhl Co-developed-by: Miguel Ojeda Signed-off-by: Miguel Ojeda Signed-off-by: Dirk Behme Reviewed-by: Trevor Gross Link: https://lore.kernel.org/r/20240130075117.4137360-2-dirk.behme@de.bosch.com [ Fixed unordered list rendering, rewrapped text and made headers consistent with the other documents in `rust/`. ] Signed-off-by: Miguel Ojeda

docs: rust: Move testing to a separate page

2024-02-18T20:22:27Z

To be able to add more testing documentation move the testing section to it's own page. No change on the documentation itself. Suggested-by: Trevor Gross Suggested-by: Miguel Ojeda Reviewed-by: Trevor Gross Reviewed-by: David Gow Reviewed-by: Alice Ryhl Signed-off-by: Dirk Behme Link: https://lore.kernel.org/r/20240130075117.4137360-1-dirk.behme@de.bosch.com Signed-off-by: Miguel Ojeda

rust: kernel: stop using ptr_metadata feature

2024-02-18T20:22:27Z

The `byte_sub` method was stabilized in Rust 1.75.0. By using that method, we no longer need the unstable `ptr_metadata` feature for implementing `Arc::from_raw`. This brings us one step closer towards not using unstable compiler features. Reviewed-by: Benno Lossin Reviewed-by: Martin Rodriguez Reboredo Reviewed-by: Trevor Gross Signed-off-by: Alice Ryhl Link: https://lore.kernel.org/r/20240215104601.1267763-1-aliceryhl@google.com [ Reworded title. ] Signed-off-by: Miguel Ojeda

rust: kernel: add reexports for macros

2024-02-18T20:22:27Z

Currently, all macros are reexported with #[macro_export] only, which means that to access `new_work!` from the workqueue, you need to import it from the path `kernel::new_work` instead of importing it from the workqueue module like all other items in the workqueue. By adding reexports of the macros, it becomes possible to import the macros from the correct modules. It's still possible to import the macros from the root, but I don't think we can do anything about that. There is no functional change. This is merely a code cleanliness improvement. Signed-off-by: Alice Ryhl Reviewed-by: Trevor Gross Reviewed-by: Martin Rodriguez Reboredo Tested-by: Boqun Feng Link: https://lore.kernel.org/r/20240129145837.1419880-1-aliceryhl@google.com [ Removed new `use kernel::prelude::*`s, reworded title. ] Signed-off-by: Miguel Ojeda