|
| 1 | +- Start Date: 2023-10-25 |
| 2 | +- RFC Type: decision |
| 3 | +- RFC PR: https://github.com/getsentry/rfcs/pull/119 |
| 4 | +- RFC Status: draft |
| 5 | + |
| 6 | +# Summary |
| 7 | + |
| 8 | +Make it easier to use Rust code from Sentry/Python. |
| 9 | + |
| 10 | +This is now live in the [`ophio`] repo. See the [Proposed Workflow](#proposed-workflow) section, |
| 11 | +or the README of the [`ophio`] repo to learn how to create / publish / use these bindings. |
| 12 | + |
| 13 | +# Motivation |
| 14 | + |
| 15 | +We want to improve the adoption of Rust within the Sentry/Python codebase, making it easier to do so, |
| 16 | +and removing some hurdles that make current Rust usage very annoying and brittle. |
| 17 | + |
| 18 | +# Background |
| 19 | + |
| 20 | +The current way we use Rust from Sentry/Python has a couple of downsides which we want to improve upon. |
| 21 | + |
| 22 | +These include: |
| 23 | + |
| 24 | +- Maintaining Python Bindings is itself a huge burden and cumbersome to do. |
| 25 | +- Specifically, the introduction of the bindings predates the stable Python ABI. |
| 26 | +- Publishing packages to PyPI means that we have to care about SemVer compatibility and public API. |
| 27 | +- The publishing process itself to PyPI has seen an extremely high failure rate as of late. |
| 28 | +- Getting a Rust dependency into Sentry/Python takes a long chain of publishes/updates and thus takes a long time. |
| 29 | + |
| 30 | +We currently have Rust bindings in the form of the `symbolic` and `sentry-relay`, |
| 31 | +where I believe both python package are built specifically for usage in Sentry, with very limited to no third-party usage. |
| 32 | +In particular, Python `symbolic` contains a subset of the functionality the Rust crate provides, |
| 33 | +and in other cases also exposes functionality that is not part of the Rust crate at all, |
| 34 | +for the sole purpose of having Python bindings for that functionality (example: `proguard`). |
| 35 | + |
| 36 | +# Options Considered |
| 37 | + |
| 38 | +We settled on creating a "dumping group" |
| 39 | + |
| 40 | +## Create a single Sentry-specific bindings package |
| 41 | + |
| 42 | +With this approach, we would a single project/package that acts a Rust binding layer. |
| 43 | +We would pull in only the Rust dependencies that we need, and only expose functionality to Python that we actually use. |
| 44 | +It would also make it possible to move functionality from Python to Rust that does not make sense as a standalone |
| 45 | +Rust crate. |
| 46 | + |
| 47 | +- **pro**: Only pulling in the dependencies that are actually needed. |
| 48 | +- **pro**: Only a single Python extension module to build / care about with fixed overhead. |
| 49 | +- **pro**: Ability to move functionality from Python to Rust more fine-grained. |
| 50 | +- **pro**: No need to ~maintain public PyPI packages and~ care about SemVer. |
| 51 | +- **pro**: Simpler release / update flow. ~Ideally things would only require a single commit to `sentry`.~ |
| 52 | +- **con**: Setting up and maintaining a workflow in the Monolith repo that suits every developer might be more complex. |
| 53 | + |
| 54 | +## Maintain crate-specific Python bindings |
| 55 | + |
| 56 | +This approach is listed for completeness, I do _not_ advocate for it. |
| 57 | +With this approach, each Rust crate we are interested in would have its own Python Bindings. |
| 58 | + |
| 59 | +- **pro**: We have smaller more targeted crates/packages to maintain and to publish. |
| 60 | +- **pro/con**: Both pro/con is that each crate/package is in charge of its own workflows. |
| 61 | +- **con**: This would still suffer from the problems around having to maintain a public SemVer API. |
| 62 | +- **con**: It would also rely on publishing to PyPI, and then updating that version within Sentry. |
| 63 | +- **con**: Having multiple smaller Python extension modules will increased the fixed per-module overhead. |
| 64 | + In particular, each module would have its own bundled copy of the Rust `std`. |
| 65 | + |
| 66 | +# Proposed Workflow |
| 67 | + |
| 68 | +- Have a separate repo ([`ophio`]) for the Rust bindings, using PyO3 / maturin. |
| 69 | +- Use the standard _release_/_publish_ workflow. |
| 70 | +- Manually trigger publishes and [auto-approve](https://github.com/getsentry/publish/blob/main/.github/workflows/auto-approve.yml). |
| 71 | +- The _publish_ workflow will publish to _public_ PyPI. |
| 72 | +- The package will be public, but we will not have any SemVer obligations whatsoever. |
| 73 | +- The internal PyPI mirror will automatically pick this up. |
| 74 | +- Trigger the [bump-version] workflow to update this in Sentry. |
| 75 | + |
| 76 | +Put even simpler: |
| 77 | + |
| 78 | +- Get PR in [`ophio`] approved and merged. |
| 79 | +- Trigger _release_/_publish_ workflow in [`ophio`]. |
| 80 | +- Either run [bump-version] workflow in [`sentry`], or open a PR that bumps the dependency version. |
| 81 | + |
| 82 | +# Unresolved questions |
| 83 | + |
| 84 | +- ~Can this live in the main `sentry` repo, or does it need to be a separate repo / project?~ |
| 85 | +- ~How would the developer workflow look like?~ |
| 86 | +- ~As in: Can Python-only developers just install pre-built binaries without the need to care about parts written in Rust at all?~ |
| 87 | + |
| 88 | +[bump-version]: https://github.com/getsentry/sentry/actions/workflows/bump-version.yml |
| 89 | +[`ophio`]: https://github.com/getsentry/ophio |
| 90 | +[`sentry`]: https://github.com/getsentry/sentry |
0 commit comments