pd: allow bundling the web frontend into the fullnode #3522

hdevalence · 2023-12-15T01:53:18Z

This is an experiment towards bundling the web frontend code into the fullnode, with the goal that users who run pd nodes can self-host a complete Penumbra frontend, on their custom domain with auto-HTTPS, with just one command: pd start.

To do this, we

add a new zipserve module in pd, which takes a &'static [u8] with archive data and produces an axum::Router;
bundle an assets/frontend.zip (tracked with git-lfs) into the pd crate;
use axum::Router::merge to combine the GRPC and frontend endpoints.

To make this work for real, we'd need to:

Fix up the auto-HTTPS support so that it can work directly with Axum, rather than Tonic;
Change the frontend code so that all URLs are relativizable to an arbitrary prefix (we want to have it at /app when served in pd, to allow a status page, but / when at app.testnet.penumbra.zone or other deployments);
Create a process to regenerate assets/frontend.zip from the web repo periodically and check the result into the repo (under LFS).

These seem pretty doable, and it'd be really cool to be able to have easily self-hosted frontends!

hdevalence · 2023-12-17T06:40:52Z

crates/bin/pd/src/main.rs

+            // TODO: will need a bit of finesse to make this work with auto-https,
+            // since auto-https support above works with the tonic `serve_with_incoming`
+            // method, but now we're dropping down to axum directly. just needs some realignment
+            // probably.


In the ~year since this code was written, there may be better options. https://docs.rs/tokio-rustls-acme/latest/tokio_rustls_acme/ seems promising

cratelyn · 2024-01-22T17:44:11Z

opened #3646 to track landing this feature.

``` /!\ this is a work in progress and will /!\ /!\ be force pushed until ready for review /!\ ``` fixes #3627. this reorganizes the logic in pd's startup code related to automatically managed https functionality. in particular, this is done in such a manner as to expose an axum `Router` that will permit later work (#3646, #3522) to serve other additional routes from our grpc endpoint. Refs: #3627 Refs: #3646 Refs: #3522

``` /!\ this is a work in progress and will /!\ /!\ be force pushed until ready for review /!\ ``` ## 👀 overview fixes #3627. this reorganizes the logic in pd's startup code related to automatically managed https functionality. ## 🎨 background & motivation this PR, besides cleaning up the `rustls-acme`-related auto-https logic, is also interested in *creating a state-of-affairs that will dovetail into pr #3522*. in particular, this expression to start the GRPC serve given a bound listener... ```rust tokio::task::Builder::new() .name("grpc_server") .spawn(grpc_server.serve_with_incoming(tls_incoming)) .expect("failed to spawn grpc server") ``` ...should be adjusted so as to accept an `axum::Router`. other tertiary requirements: - when no `grpc_auto_https` flag has been configured, serve without using an ACME resolver - ALPN permit http/1.1 for grpc-web backwards compatibility. ### ⚖️ `rustls-acme` and `tokio-rustls-acme` quoth the #3627 description, citing an earlier comment: > In the ~year since this code was written, there may be better options. > `tokio-rustls-acme` seems promising \- <#3522 (comment)> for reference, the repositories for each live here, and here: - <https://github.com/FlorianUekermann/rustls-acme> - <https://github.com/n0-computer/tokio-rustls-acme> after some comparison, i have come to the opinion that `rustls-acme` will still be adequate for our purposes. the latter is a fork of the former, but active development appears to have continued in the former, and i did not see any particular "_must-have_" features for us in the latter. ## 🚰 dropping down to `axum` (this part is lengthy...) as stated above, we want to switch to an `axum::Router`. this means that we won't be able to use the `AcmeConfig::incoming` function. the `rustls-acme` library provides some "low-level" examples we can check out: - <https://github.com/FlorianUekermann/rustls-acme/blob/main/examples/low_level_tokio.rs> - <https://github.com/FlorianUekermann/rustls-acme/blob/main/examples/low_level_axum.rs> `low_level_axum` sounds promising, given that #3522 itself drops down from tonic into axum. a catch is that the "axum" example is referring to `axum-server`, a library that wraps `axum`, rather than `axum` itself. more on that momentarily. we also use `tonic` 0.10.2 in pd, and elsewhere in the penumbra monorepo. tonic isn't using hyper 1.x yet. this was being worked on in hyperium/tonic#1583, continued on in hyperium/tonic#1595, and tracked in hyperium/tonic#1579. that work also depends upon hyperium/hyper#3461. this is complicated by the fact that `axum-server` 0.6.0 depends on `hyper` 1.1.0. this is particularly relevant because of hyperium/hyper#3040. so, we must be mindful that there is now a `tower::Service` and a `hyper::Service`. `hyper-util` provides facilities to convert between the two, and other helper glue. additionally, see the "_upgrading"_ migration guide for more information on the differences between pre- and post-1.0 interfaces in `hyper`. - https://hyper.rs/guides/1/upgrading/ - <https://docs.rs/hyper-util/latest/hyper_util/service/struct.TowerToHyperService.html> at a high level we need to take our `tonic::Router`, and turn that into a `MakeService`, some object that can be used to _generate_ `Service` instances to process a request. this is additionally complicated by the fact that `axum-server` defines its _own_ `MakeService` trait. that trait is sealed, blanket implemented, and is a bound included in `axum_server::server::Server::serve`. ```rust // axum_server::serve impl Server { pub async fn serve<M>(self, mut make_service: M) -> io::Result<()> where M: MakeService<SocketAddr, Request<Incoming>>, // ... { todo!() } } ``` the `SocketAddr` parameter above is of importance. `axum::Router::into_make_service` gives us `IntoMakeService<Router>` - <https://docs.rs/axum/latest/axum/routing/struct.IntoMakeService.html> ```rust impl<S, T> Service<T> for IntoMakeService<S> where S: Clone, { type Response = S; type Error = Infallible; type Future = IntoMakeServiceFuture<S>; ``` ...which means that we are faced with this error: ``` error[E0277]: the trait bound `Router: tower_service::Service<http::request::Request<hyper::body::incoming::Incoming>>` is not satisfied --> crates/bin/pd/src/auto_https.rs:62:20 | 62 | .serve(make_svc) | ----- ^^^^^^^^ the trait `tower_service::Service<http::request::Request<hyper::body::incoming::Incoming>>` is not implemented for `Router` | | | required by a bound introduced by this call | = help: the trait `tower_service::Service<axum::http::Request<axum::body::Body>>` is implemented for `Router` = help: for that trait implementation, expected `axum::http::Request<axum::body::Body>`, found `http::request::Request<hyper::body::incoming::Incoming>` = note: required for `IntoMakeService<Router>` to implement `axum_server::service::MakeService<std::net::SocketAddr, http::request::Request<hyper::body::incoming::Incoming>>` ``` `axum::Router::into_make_service` does not play well with `axum_server::Server`. ...and if we specify the new `hyper::body::incoming::Incoming` as the body type for the axum router, ``` error[E0599]: the method `into_make_service` exists for struct `Router<(), Incoming>`, but its trait bounds were not satisfied --> crates/bin/pd/src/auto_https.rs:57:31 | 57 | let make_svc = router.into_make_service(); | ^^^^^^^^^^^^^^^^^ method cannot be called on `Router<(), Incoming>` due to unsatisfied trait bounds | ::: /home/katie/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hyper-1.1.0/src/body/incoming.rs:47:1 | 47 | pub struct Incoming { | ------------------- doesn't satisfy `hyper::body::Incoming: HttpBody` | = note: the following trait bounds were not satisfied: `hyper::body::Incoming: HttpBody` ``` `HttpBody` was the pre-1.0 trait, that is now `Body` post-1.0. in short, there are some incongruencies at play as different libraries catch up to the new breaking changes in `hyper`. ## 🤨 ...so now what? TODO(kate): write up course of action for tomorrow. --- 🟥 NB: i worry about breakage due to untested load-bearing behavior during a migration like this. see "tertiary requirements", above. consider this a PR to be suspect of when deploying a new testnet. Refs: #3627 Refs: #3646 Refs: #3522

## 👀 overview fixes #3627. this reorganizes the logic in pd's startup code related to automatically managed https functionality. ## 🎨 background & motivation this PR, besides cleaning up the `rustls-acme`-related auto-https logic, is also interested in *creating a state-of-affairs that will dovetail into pr #3522*. in particular, this expression to start the GRPC serve given a bound listener... ```rust tokio::task::Builder::new() .name("grpc_server") .spawn(grpc_server.serve_with_incoming(tls_incoming)) .expect("failed to spawn grpc server") ``` ...should be adjusted so as to work with an `axum::Router`. ### ⚖️ `rustls-acme` and `tokio-rustls-acme` quoth the #3627 description, citing an earlier comment: > In the ~year since this code was written, there may be better options. > `tokio-rustls-acme` seems promising \- <#3522 (comment)> for reference, the repositories for each live here, and here: - <https://github.com/FlorianUekermann/rustls-acme> - <https://github.com/n0-computer/tokio-rustls-acme> after some comparison, i have come to the opinion that `rustls-acme` will still be adequate for our purposes. the latter is a fork of the former, but active development appears to have continued in the former, and i did not see any particular "_must-have_" features for us in the latter. ## 🎴 changes this commit moves some of the auto-https related code from the `main` entrypoint, into standalone functions in `pd::main`. some constants are defined, to keep control flow clear and to help facilitate the addition of future options e.g. a flag to control the LetsEncrypt environment to use. ## 🚰 dropping down to `axum`; a brief note on future upgrades as stated above, we want to switch to an `axum::Router`. this means that we won't be able to use the `AcmeConfig::incoming` function. the `rustls-acme` library provides some "low-level" examples this work is based upon - <https://github.com/FlorianUekermann/rustls-acme/blob/main/examples/low_level_tokio.rs> - <https://github.com/FlorianUekermann/rustls-acme/blob/main/examples/low_level_axum.rs> we also use `tonic` 0.10.2 in pd, and elsewhere in the penumbra monorepo. tonic isn't using hyper 1.x yet. this was being worked on in hyperium/tonic#1583, continued on in hyperium/tonic#1595, and tracked in hyperium/tonic#1579. that work also depends upon hyperium/hyper#3461. we will need to be wait for tonic to finish its migration over to hyper 1.0, see: hyperium/tonic#1579 (comment) this is understandable, but i make note of this situation as a signpost for our future selves when considering a migration to recent versions of axum-server, axum, rustls-acme, or hyper. for now, it's easiest to stay in lock-step with tonic, and we can revisit the upgrade path(s) at a future date. === Refs: #3627 Refs: #3646 Refs: #3522

grod220 · 2024-02-23T15:16:10Z

Status page completed and ready to be added to pd: penumbra-zone/web#592

hdevalence · 2024-02-23T16:02:15Z

Great! I have a rebase of this branch in progress, I'll pick it up next.

cratelyn · 2024-02-23T16:15:47Z

so excited to see this work moving forward! 🚀

hdevalence · 2024-02-25T19:34:01Z

Here's a video of this code in action:

Screen.Recording.2024-02-25.at.11.29.10.AM.mov

Note that sending a transaction fails because the web extension hasn't been updated to the new proving parameters. However, it otherwise seems to work!

This is an experiment towards bundling the web frontend code into the fullnode, with the goal that users who run `pd` nodes can self-host a complete Penumbra frontend, on their custom domain with auto-HTTPS, with just one command: `pd start`. To do this, we - add a new `zipserve` module in `pd`, which takes a `&'static [u8]` with archive data and produces an `axum::Router`; - bundle an `assets/{minifront,node_status}.zip` (tracked with git-lfs) into the `pd` crate; - use `axum::Router::merge` to combine the GRPC and frontend endpoints.

hdevalence temporarily deployed to smoke-test December 15, 2023 01:53 — with GitHub Actions Inactive

hdevalence temporarily deployed to smoke-test December 15, 2023 17:01 — with GitHub Actions Inactive

hdevalence commented Dec 17, 2023

View reviewed changes

hdevalence mentioned this pull request Jan 18, 2024

Clean up pd auto-https code #3627

Closed

cratelyn mentioned this pull request Jan 22, 2024

🧺 pd: self-host complete web frontend #3646

Closed

cratelyn mentioned this pull request Jan 24, 2024

pd: 🔨 rework RootCommand::start auto-https logic #3652

Merged

cratelyn added A-node Area: System design and implementation for node software C-enhancement Category: an enhancement to the codebase labels Feb 8, 2024

hdevalence mentioned this pull request Feb 14, 2024

Create standalone static site for node status page penumbra-zone/web#513

Closed

hdevalence force-pushed the bundle-frontend-into-pd branch from 8042942 to b74ca3a Compare February 25, 2024 19:18

hdevalence had a problem deploying to smoke-test February 25, 2024 19:18 — with GitHub Actions Failure

hdevalence force-pushed the bundle-frontend-into-pd branch from b74ca3a to 1ab2e6a Compare February 25, 2024 19:31

hdevalence had a problem deploying to smoke-test February 25, 2024 19:31 — with GitHub Actions Error

hdevalence marked this pull request as ready for review February 25, 2024 19:32

hdevalence force-pushed the bundle-frontend-into-pd branch from 1ab2e6a to b95bd84 Compare February 25, 2024 19:35

hdevalence temporarily deployed to smoke-test February 25, 2024 19:35 — with GitHub Actions Inactive

hdevalence merged commit 0ce88da into main Feb 25, 2024

hdevalence deleted the bundle-frontend-into-pd branch February 25, 2024 20:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

pd: allow bundling the web frontend into the fullnode #3522

pd: allow bundling the web frontend into the fullnode #3522

Uh oh!

hdevalence commented Dec 15, 2023 •

edited

Loading

Uh oh!

hdevalence Dec 17, 2023

Uh oh!

cratelyn commented Jan 22, 2024

Uh oh!

grod220 commented Feb 23, 2024

Uh oh!

hdevalence commented Feb 23, 2024

Uh oh!

cratelyn commented Feb 23, 2024

Uh oh!

hdevalence commented Feb 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pd: allow bundling the web frontend into the fullnode #3522

pd: allow bundling the web frontend into the fullnode #3522

Uh oh!

Conversation

hdevalence commented Dec 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hdevalence Dec 17, 2023

Choose a reason for hiding this comment

Uh oh!

cratelyn commented Jan 22, 2024

Uh oh!

grod220 commented Feb 23, 2024

Uh oh!

hdevalence commented Feb 23, 2024

Uh oh!

cratelyn commented Feb 23, 2024

Uh oh!

hdevalence commented Feb 25, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

hdevalence commented Dec 15, 2023 •

edited

Loading