Native support for Rust-based Polars #10007
Description
Today Polars’ Rust engine can access lakeFS via the S3 gateway. This issue requests native lakeFS support so users can run Rust Polars without S3-gateway dependency.
Background
- Polars is moving away from Python fsspec toward Rust object_store.
- Rust-native I/O is significantly faster than s3fs/Python paths.
Details
Provide a lakeFS object_store::ObjectStore implementation (or equivalent adapter) that supports:
- Efficient read + write (including multipart/chunked upload where needed)
- lakeFS auth options, including temporary / STS-like tokens
Reference
delta-rs lakeFS integration: https://github.com/delta-io/delta-rs/tree/main/crates/lakefs