Explicit Para Lifecycle w/ Upgrades and Downgrades

roadmap/implementers-guide/src/runtime/paras.md

@@ -1,12 +1,16 @@
 # Paras Module
 
-The Paras module is responsible for storing information on parachains and parathreads. Registered parachains and parathreads cannot change except at session boundaries. This is primarily to ensure that the number of bits required for the availability bitfields does not change except at session boundaries.
+The Paras module is responsible for storing information on parachains and parathreads. Registered
+parachains and parathreads cannot change except at session boundaries. This is primarily to ensure
+that the number of bits required for the availability bitfields does not change except at session
+boundaries.
 
-It's also responsible for managing parachain validation code upgrades as well as maintaining availability of old parachain code and its pruning.
+It's also responsible for managing parachain validation code upgrades as well as maintaining
+availability of old parachain code and its pruning.
 
 ## Storage
 
-Utility structs:
+### Utility Structs
 
 ```rust
 // the two key times necessary to track for every code replacement.
@@ -49,15 +53,69 @@ struct ParaGenesisArgs {
   /// True if parachain, false if parathread.
   parachain: bool,
 }
+
+/// The possible states of a para, to take into account delayed lifecycle changes.
+pub enum ParaLifecycle {
+  /// Para is new and is onboarding as a Parathread.
+  OnboardingAsParathread,
+  /// Para is new and is onboarding as a Parachain.
+  OnboardingAsParachain,
+  /// Para is a Parathread.
+  Parathread,
+  /// Para is a Parachain.
+  Parachain,
+  /// Para is a Parathread which is upgrading to a Parachain.
+  UpgradingToParachain,
+  /// Para is a Parachain which is downgrading to a Parathread.
+  DowngradingToParathread,
+  /// Parachain is being offboarded.
+  OutgoingParathread,
+  /// Parathread is being offboarded.
+  OutgoingParachain,
+}
 ```
 
-Storage layout:
+#### Para Lifecycle
+
+Because the state of parachains and parathreads are delayed by a session, we track the specific
+state of the para using the `ParaLifecycle` enum.
+
+```
+None                 Parathread                  Parachain
+ +                        +                          +
+ |                        |                          |
+ |    (Session Delay)     |                          |
+ |                        |                          |
+ +----------------------->+                          |
+ | OnboardingAsParathread |                          |
+ |                        |                          |
+ +-------------------------------------------------->+
+ | OnboardingAsParachain  |                          |
+ |                        |                          |
+ |                        +------------------------->+
+ |                        |  UpgradingToParachain    |
+ |                        |                          |
+ |                        +<-------------------------+
+ |                        |  DowngradingToParathread |
+ |                        |                          |
+ |<-----------------------+                          |
+ |   OutgoingParathread   |                          |
+ |                        |                          |
+ +<--------------------------------------------------+
+ |                        |    OutgoingParachain     |
+ |                        |                          |
+ +                        +                          +
+```
+
+During the transition period, the para object is still considered in its existing state.
+
+### Storage Layout
 
 ```rust
 /// All parachains. Ordered ascending by ParaId. Parathreads are not included.
 Parachains: Vec<ParaId>,
-/// All parathreads.
-Parathreads: map ParaId => Option<()>,
+/// The current lifecycle state of all known Para Ids.
+ParaLifecycle: map ParaId => Option<ParaLifecycle>,
 /// The head-data of every registered para.
 Heads: map ParaId => Option<HeadData>;
 /// The validation code of every live para.
@@ -83,38 +141,73 @@ FutureCodeUpgrades: map ParaId => Option<BlockNumber>;
 FutureCode: map ParaId => Option<ValidationCode>;
 
 /// Upcoming paras (chains and threads). These are only updated on session change. Corresponds to an
-/// entry in the upcoming-genesis map.
+/// entry in the upcoming-genesis map. Ordered ascending by ParaId.
 UpcomingParas: Vec<ParaId>;
 /// Upcoming paras instantiation arguments.
 UpcomingParasGenesis: map ParaId => Option<ParaGenesisArgs>;
-/// Paras that are to be cleaned up at the end of the session.
+/// Paras that are to be cleaned up at the end of the session. Ordered ascending by ParaId.
 OutgoingParas: Vec<ParaId>;
+/// Existing Parathreads that should upgrade to be a Parachain. Ordered ascending by ParaId.
+UpcomingUpgrades: Vec<ParaId>;
+/// Existing Parachains that should downgrade to be a Parathread. Ordered ascending by ParaId.
+UpcomingDowngrades: Vec<ParaId>;
 ```
 
 ## Session Change
 
 1. Clean up outgoing paras.
-	1. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is because any outdated validation code must remain available on-chain for a determined amount of blocks, and validation code outdated by de-registering the para is still subject to that invariant.
-1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis parameters.
-1. Amend the `Parachains` list to reflect changes in registered parachains.
-1. Amend the `Parathreads` set to reflect changes in registered parathreads.
+  1. This means removing the entries under `Heads`, `ValidationCode`, `FutureCodeUpgrades`, and
+     `FutureCode`. An according entry should be added to `PastCode`, `PastCodeMeta`, and
+     `PastCodePruning` using the outgoing `ParaId` and removed `ValidationCode` value. This is
+     because any outdated validation code must remain available on-chain for a determined amount of
+     blocks, and validation code outdated by de-registering the para is still subject to that
+     invariant.
+1. Apply all incoming paras by initializing the `Heads` and `ValidationCode` using the genesis
+   parameters.
+1. Amend the `Parachains` list and `ParaLifecycle` to reflect changes in registered parachains.
+1. Amend the `ParaLifecycle` set to reflect changes in registered parathreads.
+1. Upgrade all parathreads that should become parachains, updating the `Parachains` list and
+   `ParaLifecycle`.
+1. Downgrade all parachains that should become parathreads, updating the `Parachains` list and
+   `ParaLifecycle`.
 
 ## Initialization
 
-1. Do pruning based on all entries in `PastCodePruning` with `BlockNumber <= now`. Update the corresponding `PastCodeMeta` and `PastCode` accordingly.
+1. Do pruning based on all entries in `PastCodePruning` with `BlockNumber <= now`. Update the
+   corresponding `PastCodeMeta` and `PastCode` accordingly.
 
 ## Routines
 
-* `schedule_para_initialize(ParaId, ParaGenesisArgs)`: schedule a para to be initialized at the next session.
-* `schedule_para_cleanup(ParaId)`: schedule a para to be cleaned up at the next session.
-* `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code upgrade of the given parachain, to be applied after inclusion of a block of the same parachain executed in the context of a relay-chain block with number >= `expected_at`.
-* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head, where the new head was executed in the context of a relay-chain block with given number. This will apply pending code upgrades based on the block number provided.
-* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches the validation code to be used when validating a block in the context of the given relay-chain height. A second block number parameter may be used to tell the lookup to proceed as if an intermediate parablock has been included at the given relay-chain height. This may return past, current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if provided, must be before `at`. If the validation code has been pruned, this will return `None`.
-* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread.
-* `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live parathread or live parachain.
-
-* `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set. This is the `expected_at` number, not the `activated_at` number.
-* `persisted_validation_data(id: ParaId) -> Option<PersistedValidationData>`: Get the PersistedValidationData of the given para, assuming the context is the parent block. Returns `None` if the para is not known.
+* `schedule_para_initialize(ParaId, ParaGenesisArgs)`: Schedule a para to be initialized at the next
+  session. Noop if para is already registered in the system with some `ParaLifecycle`.
+* `schedule_para_cleanup(ParaId)`: Schedule a para to be cleaned up at the next session.
+* `schedule_parathread_upgrade(ParaId)`: Schedule a parathread to be upgraded to a parachain. Noop
+  if `ParaLifecycle` is not `Parachain`.
+* `schedule_parachain_downgrade(ParaId)`: Schedule a parachain to be downgraded to a parathread.
+  Noop if `ParaLifecycle` is not `Parachain`. `ParaLifecycle` is not `Parathread`.
+* `schedule_code_upgrade(ParaId, ValidationCode, expected_at: BlockNumber)`: Schedule a future code
+  upgrade of the given parachain, to be applied after inclusion of a block of the same parachain
+  executed in the context of a relay-chain block with number >= `expected_at`.
+* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head,
+  where the new head was executed in the context of a relay-chain block with given number. This will
+  apply pending code upgrades based on the block number provided.
+* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches
+  the validation code to be used when validating a block in the context of the given relay-chain
+  height. A second block number parameter may be used to tell the lookup to proceed as if an
+  intermediate parablock has been included at the given relay-chain height. This may return past,
+  current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if
+  provided, must be before `at`. If the validation code has been pruned, this will return `None`.
+* `lifecycle(ParaId) -> Option<ParaLifecycle>`: Return the `ParaLifecycle` of a para.
+* `is_parachain(ParaId) -> bool`: Returns true if the para ID references any live parachain, including those which may be transitioning to a parathread in the future.
+* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread, including those which may be transitioning to a parachain in the future.
+* `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live parathread
+  or live parachain.
+* `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of
+  the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set.
+  This is the `expected_at` number, not the `activated_at` number.
+* `persisted_validation_data(id: ParaId) -> Option<PersistedValidationData>`: Get the
+  PersistedValidationData of the given para, assuming the context is the parent block. Returns
+  `None` if the para is not known.
 
 ## Finalization
 

runtime/parachains/src/lib.rs

@@ -43,6 +43,7 @@ mod util;
 mod mock;
 
 pub use origin::{Origin, ensure_parachain};
+pub use paras::ParaLifecycle;
 
 /// Schedule a para to be initialized at the start of the next session with the given genesis data.
 pub fn schedule_para_initialize<T: paras::Config>(