Real-time collaboration: Fix persisted document meta and add tests

[chriszarate]

What?

Improve the format and type of persisted CRDT document meta.

Why?

The unknown record value type allowed consuming code to incorrectly nest document meta in a meta property. As a result, persisted CRDT documents could not be correctly validated, resulting in incorrect merge behavior and CRDT document invalidation.

How?

Provide the CRDT document meta correctly, improve the type to prevent nested meta in the future, and add tests.

[youknowriad]

I'm kind of hesitant here, are these the only meta attributes that third-party blocks can have, I wonder if it's possible to have arrays and nested objects too.

[chriszarate]

@youknowriad CRDT document metadata are in-memory metadata of the CRDT document instance (example: a boolean indicating whether the document was loaded from persistence / post meta). The sync package fully controls this metadata and it is not persisted or synced. I've added a comment in 2beb2f4 that hopefully makes this more clear.

[youknowriad]

Ah I thought this was about post meta

[github-actions]

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: chriszarate <[email protected]>
Co-authored-by: youknowriad <[email protected]>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

[github-actions]

Flaky tests detected in 2beb2f4.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/20241061840
📝 Reported issues:

• [Flaky Test] should open and edit templates correctly when active_templates experiment is disabled #73743 in /test/e2e/specs/site-editor/template-id-format.spec.js

[youknowriad]

Why are we messing with these "meta"? Is it because we need to persist the doc in the database or something, so we need some serialized information.

[chriszarate]

The main use case for CRDT document metadata is local state that is colocated with the document. "Local" because we don't want this metadata to be synced to peers or persisted—it applies only to this document instance, in-memory.

We have one use case currently: We use CRDT document metadata to store a marker that the CRDT document was loaded from persistence. This allows us to have code in core-data that runs only when we are validating persisted CRDT documents:

gutenberg/packages/core-data/src/utils/crdt.ts

Lines 282 to 292 in dbcbe3d

	if (
	ydoc.meta?.get( CRDT_DOC_META_PERSISTENCE_KEY ) &&
	editedRecord.content
	) {
	const blocks = ymap.get( 'blocks' ) as YBlocks;
	return (
	__unstableSerializeAndClean(
	blocks.toJSON()
	).trim() !== editedRecord.content.raw.trim()
	);
	}

tl;dr: They are in-memory descriptors for a CRDT document.