Update MSRV to 1.85 and migrate to Rust 2024 #8227

mbrobbel · 2025-08-26T10:04:13Z

Which issue does this PR close?

Test Rust 2024 Edition #6827

Rationale for this change

Following https://github.com/apache/arrow-rs?tab=readme-ov-file#rust-version-compatibility-policy.

What changes are included in this PR?

Bump MSRV to 1.85
Migrate to Rust 2024
- Handle match ergnomics change (https://doc.rust-lang.org/edition-guide/rust-2024/match-ergonomics.html)
- Handle unsafe attributes (https://doc.rust-lang.org/edition-guide/rust-2024/unsafe-attributes.html)
- Handle unsafe_op_in_unsafe_fn errors (https://doc.rust-lang.org/edition-guide/rust-2024/unsafe-op-in-unsafe-fn.html)
- Handle gen keyword addition (https://doc.rust-lang.org/edition-guide/rust-2024/gen-keyword.html)
- Handle lifetime capture rules change (https://doc.rust-lang.org/edition-guide/rust-2024/rpit-lifetime-capture.html)
  - arrow::util::pretty::pretty_format_batches
  - arrow::util::pretty::pretty_format_batches_with_schema
  - arrow::util::pretty::pretty_format_columns
  - arrow::util::pretty::pretty_format_batches_with_options
  - arrow::util::pretty::pretty_format_columns_with_options
Fix clippy warnings
~~Use Rustfmt 2024 edition (https://doc.rust-lang.org/edition-guide/rust-2024/rustfmt-style-edition.html)~~

Are these changes tested?

CI

Are there any user-facing changes?

Yes:

MSRV is now 1.85
Signature change of arrow::util::pretty methods

alamb

Thanks @mbrobbel

I found this easier to review with whitespace blind diff on: https://github.com/apache/arrow-rs/pull/8227/files?w=1

I reviewed the changes up until the parquet crates I think they look great and make the code easier to read and maintain but I worry about my ability to review it in detail. However, I ran out of time.

The only real concern I have is to ensure that this change remain backwards compatible with existing users / rust versions? Given the MSRV checks I think it will, but we might want to double check (maybe we can try to update DataFusion to use this PR to see)

Would it be possible to split the PR so it is easier to review? Some ideas

just change rustfmt.toml file to get the formatting only changes? I tried it locally and it didn't seem to change anything
Change the edition for only a few crates at a time (arrow, arrow-schema in one PR, then parquet in the next, etc)

BTW I think this PR is likely to attract many merge conflicts, so it probably makes sense to get it merged asap

alamb · 2025-09-04T18:00:37Z

arrow-cast/src/pretty.rs

 /// +---+---+"#);
 /// ```
-pub fn pretty_format_batches(results: &[RecordBatch]) -> Result<impl Display, ArrowError> {
+pub fn pretty_format_batches(results: &[RecordBatch]) -> Result<impl Display + use<>, ArrowError> {


What is the implication of this change for users of this crate? I assume it is a no-op but I am not familiar with the use syntax

The relevant change is documented here https://doc.rust-lang.org/edition-guide/rust-2024/rpit-lifetime-capture.html

In Rust 2021 and earlier editions, when the use<..> bound is not present, generic lifetime parameters are only captured when they appear syntactically within a bound in RPIT opaque types in the signature of bare functions and associated functions and methods within inherent impls. However, starting in Rust 2024, these in-scope generic lifetime parameters are unconditionally captured.

The returned type does not capture the lifetime of the recordbatch slice, so use<> indicates this.

For users this means that the following still compiles in 2024 Rust:

let record_batches: &[RecordBatch] = ...; let pretty = pretty_format_batches(record_batches)?; drop(record_batches); dbg!(pretty);

Without use<> in 2024 Rust rustc would complain about record_batches still being borrowed when dropped.

alamb · 2025-09-04T18:02:38Z

arrow-ipc/src/gen/Schema.rs

    unsafe fn follow(buf: &'a [u8], loc: usize) -> Self::Inner {
-        let b = flatbuffers::read_scalar_at::<i16>(buf, loc);
-        Self(b)
+        unsafe {


since this file is automatically generated by flatbuffers compiler, can we avoid these changes (maybe by suppressing whatever compiler lint / or hard coding this file to use the old edition?)

I reverted the changes and updated regen.sh to handle the gen keyword, and after regenaration it seems flatc now adds the required unsafe.

mbrobbel · 2025-09-15T18:24:46Z

The only real concern I have is to ensure that this change remain backwards compatible with existing users / rust versions? Given the MSRV checks I think it will, but we might want to double check (maybe we can try to update DataFusion to use this PR to see)

The 2024 edition is supported by rustc >=1.85 so with the msrv bump it should be fine.

1. just change `rustfmt.toml` file to get the formatting only changes? I tried it locally and it didn't seem to change anything

I reverted the formatting changes. It's now using 2021 edition for rustfmt. We can update in a different PR.

2. Change the edition for only a few crates at a time (arrow, arrow-schema in one PR,  then `parquet` in the next, etc)

Let me know if you still want me to do this.

BTW I think this PR is likely to attract many merge conflicts, so it probably makes sense to get it merged asap

We have to wait until #7836.

alamb · 2025-09-23T18:19:32Z

BTW I am worried about this PR causing a lot of merge conflicts, especially for my push decoder PR:

Implement Push Parquet Decoder #7997
I wonder if there is some way to break it up into smaller chunks (like do we really need to adjust the rust edition as well as the msrv in one PR?)

mbrobbel · 2025-09-23T19:42:53Z

BTW I am worried about this PR causing a lot of merge conflicts, especially for my push decoder PR:

* [Implement Push Parquet Decoder #7997](https://github.com/apache/arrow-rs/pull/7997)
  I wonder if there is some way to break it up into smaller chunks (like do we really need to adjust the rust edition as well as the msrv in one PR?)

No we don't. I'll split this up:

Bump MSRV to 1.85 #8429

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-pyarrow-testing` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-ord` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-pyarrow` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-pyarrow-integration-testing` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-data` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-select` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-array` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-buffer` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-row` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-string` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `arrow-schema` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `parquet_derive_test` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `parquet-geospatial` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `parquet-variant-json` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `parquet_derive` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

# Which issue does this PR close? - Contribute to #6827 # Rationale for this change Splitting up #8227. # What changes are included in this PR? Migrate `parquet-variant-compute` to Rust 2024 # Are these changes tested? CI # Are there any user-facing changes? Yes

mbrobbel added 4 commits August 25, 2025 14:05

Bump MSRV to 1.85

afed13d

Migrate to Rust 2024

965b4c2

Fix clippy warnings

5fc65a0

Use 2024 edition for Rustfmt

f441022

github-actions bot added parquet Changes to the parquet crate arrow Changes to the arrow crate arrow-flight Changes to the arrow-flight crate parquet-derive parquet-variant parquet-variant* crates arrow-avro arrow-avro crate labels Aug 26, 2025

mbrobbel added 4 commits August 26, 2025 12:14

Fix parquet derive tests

eb553a5

Rustfmt for parquet

556bcea

Update more arrow::util::pretty methods to reflect lifetime captures

1f379f1

Merge branch 'main' into rust-1.85

6271b16

mbrobbel added the next-major-release the PR has API changes and it waiting on the next major version label Aug 26, 2025

mbrobbel marked this pull request as ready for review August 26, 2025 11:47

mbrobbel mentioned this pull request Aug 29, 2025

Update comfy-table requirement from =7.1.2 to =7.2.0 #8249

Closed

Merge branch 'main' into rust-1.85

4b5bcd1

alamb reviewed Sep 4, 2025

View reviewed changes

mbrobbel added 5 commits September 15, 2025 20:04

Revert style edition to 2021

2f508ef

Revert changes to generated sources in arrow-ipc

fec15d3

Handle gen keyword in generated arrow-ipc sources

456862e

Remove expect unsafe_op_in_unsafe_fn

78af55b

Merge branch 'main' into rust-1.85

14af724

Rustfmt parquet

aa2380f

mbrobbel added this to the 57.0.0 milestone Sep 15, 2025

Merge branch 'main' into rust-1.85

8fd570b

mbrobbel mentioned this pull request Oct 3, 2025

Migrate arrow-avro to Rust 2024 #8545

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update MSRV to 1.85 and migrate to Rust 2024 #8227

Update MSRV to 1.85 and migrate to Rust 2024 #8227

Uh oh!

mbrobbel commented Aug 26, 2025 •

edited

Loading

Uh oh!

alamb left a comment

Uh oh!

alamb Sep 4, 2025

Uh oh!

mbrobbel Sep 7, 2025

Uh oh!

alamb Sep 4, 2025

Uh oh!

mbrobbel Sep 15, 2025

Uh oh!

mbrobbel commented Sep 15, 2025

Uh oh!

alamb commented Sep 23, 2025

Uh oh!

mbrobbel commented Sep 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

Update MSRV to 1.85 and migrate to Rust 2024 #8227

Are you sure you want to change the base?

Update MSRV to 1.85 and migrate to Rust 2024 #8227

Uh oh!

Conversation

mbrobbel commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

mbrobbel Sep 7, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Sep 4, 2025

Choose a reason for hiding this comment

Uh oh!

mbrobbel Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

mbrobbel commented Sep 15, 2025

Uh oh!

alamb commented Sep 23, 2025

Uh oh!

mbrobbel commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

mbrobbel commented Aug 26, 2025 •

edited

Loading

mbrobbel commented Sep 23, 2025 •

edited

Loading