Improve collaborative editing in sync package

[chriszarate]

What?

This draft pull request extends and improves the collaborative editing experiment to provide:

• More performant and stable synchronization.
• Protections to make sure that the core-data package retains control over which entity records are synced and how changes are merged.
• A filter to give plugins the ability to extend a SyncProvider class. This allows plugins to provide their own sync transports and to implement user-facing sync behaviors such as awareness / presence indicators.
• Stubs that allow CRDT docs to be persisted in meta, avoiding the "initialization problem" as described by @dmonad. (These stubs are implemented by our plugin.)
• A Yjs-powered UndoManager that works across multiple synced entities.

It is open now for discussion and guidance while we continue to improve it (see "How?" section).

Why?

This work builds on the pull request opened by @dmonad earlier this year. Our immediate goal is to provide a production-ready implementation of collaborative editing for WPVIP customers using a WebSocket transport.

Gutenberg's sync package must be flexible enough to support a future "default" sync transport that ships with Gutenberg, while also allowing plugins to provide their own transport. Additionally, while collaborative editing is still under development, it must also provide ways for plugins to implement missing features such as awareness / presence indicators and CRDT persistence.

Lastly, Gutenberg's core-data package must provide stable and performant merge algorithms so that entity records are synced in a way that aligns with user expectations.

How?

This pull request is large in service of providing a complete implementation for review. However, the commits are atomic by feature and can be reviewed one-by-one. If desired, we can split this PR up into smaller ones, with the understanding that the branches may not be in a fully functional state.

• Make syncing a side-concern instead of a replacement for local state.
  • This change is conceptually important: Instead of effectively replacing the core-data store with a "database" Yjs provider (IndexedDB), this change makes syncing a side-effect without changing how the entity records are represented and treated within the store. (Kevin's PR took the same approach.)
  • This code change is minor (unwrapping an if/else), but produces a large diff.
• Remove @wordpress/sync from bundled packages
  • This allows the sync package to be imported by plugins and other external code.
  • IN-PROGRESS: The individual exports will be exposed as a private API.
• Remove sync configs for non-post-type entities
  • The shape and spec of entity sync configs has changed, so existing non-post-type sync configs were removed purely for simplicity.
  • TODO: Restore and update these sync configs and test their sync behavior.
• Add "collaborative-editing" post type supports
  • Provide a mechanism for custom post types to opt-in to collaborative editing.
• Add SyncProvider without undo or persistence.
  • Update the SyncProvider to accept "handlers" from core-data actions and resolvers that allow it to safely interact with the store.
  • Update the entity "sync config" to provide CRDT utility functions that determine which properties are synced and how changes are merged.
  • Introduce core.getSyncProvider filter to allow plugins to extend the SyncProvider class and provide custom sync transports.
  • IN-PROGRESS: Unit tests for added code are being written in another branch.
  • IN-PROGRESS: Performance tests to understand the impact on the block editor.
• Add CRDT persistence support
  • Add stubs that allow our plugin to implement CRDT persistence in post meta.
• Add UndoManager support
  • Add a multi-doc-aware UndoManager that provides an undo/redo stack across multiple synced entities.
  • A Yjs-aware UndoManager provides undo behavior that matches sync behavior and user expectations.
• Fix the undefined type problem with API docgen
  • Bugfix for TypeScript re-exports (export { something } from './file').
• Allow post lock dialog to return and retain functionality
  • Disable the post-lock dialog, while allowing it to continually update the post lock meta.
• Disable autosave warnings when collaborative editing is enabled
• Return original blocks when controlledBlocks represent post content
  • Disable block cloning for post content blocks, preserving the synced client IDs.
  • TODO: Understand the initial motivation for cloning blocks in this hook.
• Update experiment language

Additional props to @alecgeatches, @ingeniumed, and @pkevan for their contributions to this branch.

Testing Instructions

This PR can be tested in two ways:

1. Use the existing WebRTC (+HTTP signaling) Yjs provider
   • Check out this branch.
   • Enable the real-time collaboration experiment: Gutenberg > Experiments.
   • Open a post for editing in two different browser environments.
     • Note: This provider uses requests to admin-ajax.php for signaling and the initial connection is delayed.
2. Use WPVIP's Real-Time Collaboration plugin, which ships a WebSocket Yjs provider and provides an awareness implementation.
   • Check out this branch in a folder named gutenberg.
   • Check out the plugin repo in a sibling folder.
   • npm install
   • npm run dev to start the development environment, including WebSocket server. See additional details and instructions, if needed.
   • Open a post for editing in two different browser environments.

Testing Instructions for Keyboard

No UI changes in this pull request.

Screenshots or screencast

WebSocket provider with awareness / presence indicators (WPVIP plugin)

plugin2.mov

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: chriszarate <[email protected]>
Co-authored-by: ingeniumed <[email protected]>
Co-authored-by: pkevan <[email protected]>
Co-authored-by: alecgeatches <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[chriszarate]

The work in this PR is moving forward in separate, discrete pull requests. I'll keep this comment up to date as more are opened:

• Collaborative editing: Make syncing a side-concern instead of a replacement for local state #72114
• Improve entity "sync configs" #72145
• Refactor sync provider into a createSyncManager factory function #72183
• Provide "collaborative-editing" post type support #72147
• Move Yjs providers into subfolder #72224
• Improve CRDT "merge logic" for post entities #72262
• Real-time collaboration: Add support for syncing post meta #72332
• Real-time collaboration: Allow post-locked-modal to be overridden when collaborative-editing is enabled #72326
• Real-time collaboration: Implement CRDT persistence for collaborative editing #72373
• Real-time collaboration: Fix "Show Template" view #72405
• Real-time collaboration: Add UndoManager support for collaborative editing #72407
• Real-time collaboration: Add Yjs awareness #72437
• Real-time collaboration: Add typings in core-data for @wordpress__blocks #72450
• Real-time collaboration: Ensure Yjs imports are inside experimental flag check #72503