Skip to content

Conversation

@bugadani
Copy link
Contributor

@bugadani bugadani commented Aug 14, 2023

Thank you!

Thank you for your contribution.
Please make sure that your submission includes the following:

Must

  • The code compiles without errors or warnings.
  • All examples work.
  • cargo fmt was run.
  • Your changes were added to the CHANGELOG.md in the proper section.
  • You updated existing examples
  • You or added examples: interrupt-mode, cross-core thread-mode
    • esp32
    • esp32s2
    • esp32s3
  • Added examples are checked in CI
    • Make sure to check examples with both timg0 and systimer where available.

Nice to have

  • You add a description of your work to this PR.
  • You added proper docs for your newly added features and code.

The Xtensa executor in embassy-executor is not safe on multiple cores. There are two reasons for this:

  • there exists a single global "there's more work" flag
  • it is not possible for one core to wake the other

This PR remedies this issue, implementing:

  • a thread-mode executor using FROM_CPU_0
  • an interrupt executor, which can use the rest of the FROM_CPU_x interrupts (one per executor)

On single-core (S2) we could probably use other interrupts as well. I'll probably need to add a list of crosscore-compatible, sw pendable interrupts for ESP32 and S3 (which is probably just the 4 FROM_CPU interrupt), so that we can safely disallow arbitrary interrupts on multicore.


We can't implement this in embassy, because there is no cross-core communication possibility in the Xtensa ISA. However, for consistency's sake I've also included non-multicore support (S2), which is basically identical to what's in embassy.


No, this isn't an executor that spreads load to two cores. Instead, this is an executor that is safe to use on both cores where one core's tasks can wake up the other core's tasks.

@bugadani bugadani changed the title Multicore-aware thread-mode executor Multicore-aware executors Aug 14, 2023
@bugadani bugadani force-pushed the executor branch 12 times, most recently from 198f505 to 5fbf286 Compare August 15, 2023 14:21
@bugadani

This comment was marked as outdated.

@bugadani bugadani mentioned this pull request Aug 16, 2023
6 tasks
@bugadani bugadani force-pushed the executor branch 5 times, most recently from 19b35a5 to e07f869 Compare August 16, 2023 20:03
@bugadani

This comment was marked as outdated.

@bugadani bugadani force-pushed the executor branch 3 times, most recently from c0197f3 to 8ffc5db Compare August 20, 2023 04:45
@bugadani

This comment was marked as outdated.

@bugadani bugadani force-pushed the executor branch 8 times, most recently from 9aa84e7 to e2f47a2 Compare August 22, 2023 17:27
@bugadani bugadani marked this pull request as ready for review August 22, 2023 17:29
@bugadani
Copy link
Contributor Author

Who may I curse with this monster? @MabezDev I hope you don't mind if I ping you for a review?

Copy link
Member

@MabezDev MabezDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this! It looks good from my perspective, working on all Xtensa chips :). Just a couple of comments to address.

/// # Safety
///
/// You MUST call this from the interrupt handler, and from nowhere else.
// TODO: it would be pretty sweet if we could register our own interrupt handler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be pretty nice, however, I can't think of a way to do it with the current interrupt registering mechanism. Unless we do some weird stuff with weak linkage, but that would introduce errors on its own. I think the examples for now explain the steps that need to be taken well enough.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about weak linkage, in theory (unless there are details I need to know but don't) we should be able to copy peripherals::__INTERRUPTS into RAM, change https://github.com/esp-rs/esp-hal/blob/6c2659f9e4db7fdc1d5431c267b56ec6caa02ebe/esp-hal-common/src/interrupt/xtensa.rs#L440C9-L440C87 to use the copied handlers and roll with that. I do agree that it's not absolutely necessary

@bugadani
Copy link
Contributor Author

Bleh, global copy-and-replace just made this worse 🤣

@bugadani
Copy link
Contributor Author

Ok I ended up going all in against StaticCell<Executor>, as static cells are unusable anyway after initialization.

Copy link
Member

@MabezDev MabezDev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Copy link
Member

@jessebraham jessebraham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for taking care of this!

@jessebraham jessebraham merged commit e082d47 into esp-rs:main Aug 28, 2023
@bugadani bugadani deleted the executor branch August 28, 2023 17:09
playfulFence pushed a commit to playfulFence/esp-hal that referenced this pull request Sep 26, 2023
* Implement multicore-aware executors

* Add examples

* Use pre-defined config to import SystemPeripheral

* Use static_cell::make_static
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants