Restructure guest/host error handling #868
Open
+1,347
−1,179
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ludfjig
force-pushed
the
host_error_leak_fix
branch
2 times, most recently
from
d346b32 to
2e45037
Compare
ludfjig
force-pushed
the
host_error_leak_fix
branch
6 times, most recently
from
8818cf5 to
ae355d7
Compare
ludfjig
requested review from
danbugs,
dblnz,
devigned,
syntactically,
marosset,
jprendes and
simongdavies
as code owners
jsturtevant
reviewed
Sep 12, 2025
| assert!( | ||
| matches!(&res, HyperlightError::GuestError(_, msg) if msg == "Host function error!") // rust guest | ||
| || matches!(&res, HyperlightError::GuestAborted(_, msg) if msg.contains("Host function error!")) // c guest | ||
| || matches!(&res, HyperlightError::StackOverflow()) // c guest. TODO fix this. C guest leaks when host func returns error guest panics. |
There was a problem hiding this comment.
do we have an issue for this? was this the case prior?
There was a problem hiding this comment.
This was the case prior. Feedback appreciated how to make this a nicer C-API (for all types, not just i32):
#[unsafe(no_mangle)]
pub extern "C" fn hl_get_host_return_value_as_Int() -> i32 {
get_host_return_value().expect("Unable to get host return value as int")
}
There was a problem hiding this comment.
could we mimic something like wasmtime does? https://github.com/bytecodealliance/wasmtime/blob/7380932631f7784d944cb0326a6ffaaf5dac29fc/crates/c-api/src/component/val.rs#L180-L185
I don't think this needs to be a blocker as we are in this state today
src/hyperlight_common/src/flatbuffer_wrappers/function_types.rs
Outdated
Show resolved
Hide resolved
Signed-off-by: Ludvig Liljenberg <[email protected]>
- Add all.fbs to include all schema files in one place - Restructure function_call_result.fbs to use Result-like union - Add HostError variant to ErrorCode enum in guest_error.fbs - Update flatbuffer generation command in Justfile to use all.fbs - Update documentation for new generation process Signed-off-by: Ludvig Liljenberg <[email protected]>
Update all generated Rust code based on the new schema definitions. This includes new types for error handling and result structures. Signed-off-by: Ludvig Liljenberg <[email protected]>
- Update function_types.rs to handle Result-like return values - Simplify guest_error.rs wrapper implementation - Update util.rs for new generated types - Update mod.rs for new generated types Signed-off-by: Ludvig Liljenberg <[email protected]>
- Remove guest_err.rs from hyperlight_host (replaced by new error handling) - Remove guest_err.rs from hyperlight_guest_bin (replaced by new error handling) - Update func/mod.rs to remove obsolete import Signed-off-by: Ludvig Liljenberg <[email protected]>
- Update initialized_multi_use.rs to use new Result-like error handling - Update mem/mgr.rs to handle host function errors properly - Update sandbox/outb.rs for new error propagation pattern Signed-off-by: Ludvig Liljenberg <[email protected]>
- Update guest/host_comm.rs to use new Result-like return values - Update guest_bin/call.rs to properly handle host function errors - Update guest_bin/lib.rs to remove obsolete error handling import and make GUEST_HANDLE public (for use in C-API) - Update guest_capi/error.rs to support new error types Signed-off-by: Ludvig Liljenberg <[email protected]>
Update sandbox_host_tests.rs to use the new Result-like error handling pattern. Signed-off-by: Ludvig Liljenberg <[email protected]>
Update Cargo.lock and Cargo.toml files to reflect the dependency changes needed for the new error handling implementation. Signed-off-by: Ludvig Liljenberg <[email protected]>
Signed-off-by: Ludvig Liljenberg <[email protected]>
Signed-off-by: Ludvig Liljenberg <[email protected]>
ludfjig
force-pushed
the
host_error_leak_fix
branch
from
6ecad00 to
3e17197
Compare
jsturtevant
reviewed
Sep 29, 2025
jsturtevant
previously approved these changes
Sep 29, 2025
Signed-off-by: Ludvig Liljenberg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR improves error handling between host and guest functions to prevent memory leaks and ensure more reliable deserialization.
Previously, when a host function (invoked from the guest) returned an error, the error was reported immediately without unwinding the guest stack. This left guest-side allocations in an inconsistent state, leading to memory leaks on subsequent entries into the guest.
In addition, error reporting relied on a fragile mechanism: guest errors were manually serialized into a buffer, and the host would attempt to detect them by trying to deserialize an error. If deserialization succeeded, an error was assumed to have occurred. This approach is risky because there was nothing preventing GuestError and FunctionCallResult from possibly having the same serialized format since they are completely separate.
Changes
TODO:
For C guests: If host function returns an error, guest will panic when trying to read an expected good value (because there is only an error instead), and leak memory as a result. Will fix this in follow up PR
Closes #826
Closes #497