Skip to content

Conversation

@senekor
Copy link
Owner

@senekor senekor commented Mar 19, 2023

This is an idea to keep using Unstructured in our control plane API without using unsafe and without lifetime proliferation. Basically, we keep track of which bytes have already been consumed ourselves (with the help of Unstructured::take_rest) and construct new Unstructured values over the remaining bytes for each call to the control plane API.

It involves copying all remaining bytes for each call to the API, so there is a some performance impact. This is because Unstructured may consume bytes from the start of the slice, meaning that we cannot simply truncate our vector to the length of the remaining bytes. But I was able to avoid a heap allocation per API call at least with the tmp buffer.

The API is not generic anymore, because I had issues with the generic lifetime parameter. Maybe it could've been solved, I'm not sure. But I think it wouldn't be too much effort to write one function per data type to be requested if we can still use arbitrary internally.

A consideration about the desired properties of the control plane:

It means that we could later drive the mechanism with some other source of decisions; we're not tied to arbitrary.

I think this would be covered by this approach. We can always create a new function that doesn't make use of arbitrary and uses the vector of bytes as a source of decisions directly.

senekor and others added 3 commits March 22, 2023 15:09
Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>
Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>
Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>
@senekor senekor force-pushed the fuzz-skip-branch-opt branch from be761cf to 727982e Compare March 22, 2023 14:11
Co-authored-by: Falk Zwimpfer <[email protected]>
Co-authored-by: Moritz Waser <[email protected]>
@senekor senekor force-pushed the safe-unstructured branch from 159384b to b737502 Compare March 22, 2023 14:11
@senekor senekor force-pushed the fuzz-skip-branch-opt branch from 23b0d19 to c8e5039 Compare April 3, 2023 16:07
senekor pushed a commit that referenced this pull request Apr 6, 2023
* Integrate experimental HTTP into wasmtime.

* Reset Cargo.lock

* Switch to bail!, plumb options partially.

* Implement timeouts.

* Remove generated files & wasm, add Makefile

* Remove generated code textfile

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>

* Update crates/wasi-http/Cargo.toml

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>

* Extract streams from request/response.

* Fix read for len < buffer length.

* Formatting.

* types impl: swap todos for traps

* streams_impl: idioms, and swap todos for traps

* component impl: idioms, swap all unwraps for traps, swap all todos for traps

* http impl: idiom

* Remove an unnecessary mut.

* Remove an unsupported function.

* Switch to the tokio runtime for the HTTP request.

* Add a rust example.

* Update to latest wit definition

* Remove example code.

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* Fix some warnings.

* bump new deps to latest releases (#3)

* Add tests for wasi-http to test-programs (#2)

* wip: start writing a http test...

* finish writing the outbound request example

havent executed it yet

* better debug output

* wasi-http: some stubs required for rust rewrite of the example

* add wasi_http tests to test-programs

* CI: run the http tests

* bump new deps to latest releases

h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0

---------

Co-authored-by: Brendan Burns <[email protected]>

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs

* wasi-http: fix cargo.toml file and publish script to work together (#4)

unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.

then, add wasi-http to the list of crates to publish.

* Update crates/test-programs/build.rs

* Switch to rustls

* Cleanups.

* Merge switch to rustls.

* Formatting

* Remove libssl install

* Fix tests.

* Rename wasi-http -> wasmtime-wasi-http

* prtest:full

Conditionalize TLS on riscv64gc.

* prtest:full

Fix formatting, also disable tls on s390x

* prtest:full

Add a path parameter to wit-bindgen, remove symlink.

* prtest:full

Fix tests for places where SSL isn't supported.

* Update crates/wasi-http/Cargo.toml

---------

Co-authored-by: Eduardo de Moura Rodrigues <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>
Co-authored-by: Pat Hickey <[email protected]>
@senekor senekor deleted the branch fuzz-skip-branch-opt April 16, 2023 09:58
@senekor senekor closed this Apr 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants