Skip to content

Add equaSHion: the shared equation editor (five crates) - #29

Draft
shuff57 wants to merge 1 commit into
storytold:mainfrom
shuff57:add-equashion
Draft

shuff57 wants to merge 1 commit into
storytold:mainfrom
shuff57:add-equashion

Conversation

@shuff57

@shuff57 shuff57 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds equaSHion, the equation editor WordCraft and DeckCraft share, as five crates:

Crate What
equashion-math math tree, OMML / MathML / LaTeX / linear text, templates, caret and selection edit model
equashion-font font database (bundled OFL fonts), metrics, glyph outlines, shaping
equashion-layout TeX-style layout into a backend-neutral scene, caret geometry
equashion-render scenes to RGBA pixels (vello_cpu) and SVG
equashion-egui the editor widget (canvas, palette, linear field), with no host types

It was extracted from WordCraft (provenance and the one behavioural divergence are in
docs/equashion/). Moving it here gives WordCraft, DeckCraft and any other host one copy to depend
on, instead of each app pinning a personal repo or vendoring its own.

Why this is the place for it

It meets the repo's rules: no app code, useful to more than one app, no dependency on any app,
#![deny(clippy::unwrap_used, …)] at every crate root, no unsafe.

Things to decide

  • Names. The repo rule is craft-<name>. These keep equashion-* because WordCraft's and
    DeckCraft's open PRs (wordcraft#801,
    deckcraft#192) and OfficeCraft import them by
    that name. Renaming is mechanical if you want it; say so and I'll do it here and in those PRs.
  • Root-level directories. assets/fonts/, corpus/ and tests/golden/ sit at the repo root,
    as in the standalone repo, so the crates' include_bytes! and test-data paths work unchanged.
    They can move under crates/equashion-* with small path edits if you'd rather.
  • Lints. The equashion-* crates carry the deny-list in their own [lints] tables rather
    than via [lints] workspace = true: adding it to the workspace would also fail the existing
    craft-ui test helpers.
  • Fonts. About 2.4 MB of OFL fonts (Source Serif 4, Inter, STIX Two Math), listed in
    ATTRIBUTION.md.

Checks

cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings and
cargo test --workspace all pass.

After this merges

WordCraft and DeckCraft repoint their equashion-* git dependencies from
shuff57/equaSHion to this repo's merge commit.

🤖 Generated with Claude Code

…, render, egui)

Extracted from WordCraft and shared by WordCraft and DeckCraft. Keeps the equashion-* names;
assets, test corpus and goldens sit at the repo root so the crates' relative paths are unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sullrich

Copy link
Copy Markdown

Nice — this is a well-cut split, and the dependency graph is exactly what I'd want: math ← font ← layout ← render/egui, one-way, no cycles. A few notes from the crate-standards side, then I'll take the repo-organization question to the fleet.

Crate naming. The craft-<shortname> rule in craftrules applies to shared libraries in craft-libs. A family of domain crates with its own internal dependency graph is different, and keeping equashion-* verbatim so WordCraft's and DeckCraft's open PRs compile untouched is the right trade — I won't force a rename. Two conditions, though:

  1. The contract is a carve-out, not a drift. equashion becomes a documented exception in the licensing/standards docs (like craftrules/standards/licensing.md does elsewhere), and every new crate added to the family later goes through the same discussion rather than inheriting the exception by precedent.
  2. Name provenance up front. Once we know where the upstream project actually came from, the equashion name should be traceable to it in docs/equashion/PROVENANCE.md. If it turns out the name is ours (built around the WordCraft equation panel), that's fine too — just say so in the docs so nobody wastes time hunting a nonexistent upstream.

Licensing. WordCraft and DeckCraft are proprietary, so PROPRIETARY.md in each of the five crates is correct and required. If the algorithmic core of -math is genuinely upstream-derived and upstream is OSS-licensed, the clean split is: OSS license on equashion-math (the paper-algorithm layer), proprietary on layout/render/egui (our integration layers). That also makes -math publishable/crates.io-able later without unlicensing anything. If it turns out everything is ours, five PROPRIETARY.mds is right — don't strip them.

Workspace hygiene (worth handling in the landing PR):

  • Each crate inherits workspace lints, which means the #![deny(clippy::unwrap_used, expect_used, panic, ...)] header in lib.rs is mandatory — worth a CI gate so vendored-in code doesn't slip through with unwrap()s.
  • rustfmt.toml doesn't support per-directory overrides, so your equashion/*/ gitignore approach is one way; the other is just committing the family to upstream style once and adding a CI step that excludes equashion/* from cargo fmt --check — explicit and visible, rather than a silent pre-commit skip. Prefer the CI exclusion; gitignore rules for formatted files are easy to lose.
  • -layout's "zero allocs per frame after warmup, hot path only" is the right constraint; suggest a #[cfg(test)] layout-cache test (insert → query → assert cache hit) as the regression guard rather than trusting comments.

Repo fit — I'd lean: not five into craft-libs' main crates/ list (a ten-crate workspace where seven are one feature of two apps adds build-graph weight for everyone), but keep the family together — either a crates/equashion/ workspace-member-with-own-workspace like you propose, or honestly a standalone equashion repo with path-deps from the apps, since it has its own style rules, its own upstream, and only two consumers. That's a fleet-level call, so I'll raise it as #33 and we'll decide once.

Once that lands, DeckCraft takes it from there and the wordcraft#346/wordcraft#345 test gaps can close in CI.

@zachshallbetter

Copy link
Copy Markdown
Contributor

Agreeing with @sullrich's analysis regarding repository fit:

craft-libs is intended for general-purpose technical primitives shared broadly across the suite. Landing 5 domain-specific crates (equashion-*) that only serve WordCraft and DeckCraft introduces several friction points with workspace standards:

  1. Asset Governance: The PR commits ~2.4 MB of raw font binaries (Inter, STIX Two Math, Source Serif 4) directly into git under assets/fonts/. Repository policy avoids storing binary fonts in git trees (shared fonts belong in storytold/craft-fonts).
  2. Namespace & Layout: Placing assets/fonts/, corpus/, and tests/golden/ at the repository root and introducing five equashion-* crates bypasses standard workspace organization.
  3. Build Graph Weight: Adding 5 crates for a single specialized equation editor across two apps increases compilation and dependency maintenance across the entire suite.

As @sullrich suggested, equaSHion is much better suited as a standalone repository (storytold/equashion), where it can maintain its own font assets, golden test suites, and release cycles, and be consumed by WordCraft and DeckCraft via git/path dependencies.

@zachshallbetter zachshallbetter left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, here's a code-level review focusing on memory allocation patterns, crate encapsulation, and workspace conventions across the new equashion-* crates. Hope it's helpful.

  1. Memory Allocations in DOM & Rendering:
    • crates/equashion-math/src/xml.rs:57: Recursive XEl::text() causes repeated heap reallocations when traversing deep XML trees.
    • crates/equashion-render/src/svg.rs:28: fmt2 allocates a heap String on every coordinate in path_d, creating high allocation pressure during scene serialization.
  2. Encapsulation & Asset Governance:
    • crates/equashion-font/src/fontdb.rs:28: include_bytes!("../../../assets/fonts/...") breaks crate boundary encapsulation by reaching into the workspace root, and embeds 2.4 MB of font binaries into every consumer binary.
    • crates/equashion-font/src/fontdb.rs:379: #[allow(clippy::expect_used)] bypasses the workspace panic/expect lint.
  3. Workspace Layout:
    • Cargo.toml:17: equashion-* deviates from the craft-<name> = { path = "crates/<name>" } convention.

for c in &self.children {
match c {
XNode::Text(t) => s.push_str(t),
XNode::El(e) => s.push_str(&e.text()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recursive e.text() allocates a new heap String at every subtree level and copies it into s, causing quadratic allocation overhead on deep XML trees (up to XML_DEPTH).

Consider passing a mutable buffer:

pub fn text_into(&self, out: &mut String) {
    for c in &self.children {
        match c {
            XNode::Text(t) => out.push_str(t),
            XNode::El(e) => e.text_into(out),
        }
    }
}

pub fn text(&self) -> String {
    let mut s = String::new();
    self.text_into(&mut s);
    s
}

This reuses a single buffer across the entire element hierarchy.


#[cfg(not(feature = "extra-fonts"))]
static BUNDLED: &[&[u8]] =
&[include_bytes!("../../../assets/fonts/SourceSerif4-Regular.ttf"), include_bytes!("../../../assets/fonts/SourceSerif4-It.ttf")];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The relative path include_bytes!("../../../assets/fonts/...") breaks crate boundary encapsulation. If equashion-font is moved to a standalone repository (storytold/equashion), published to crates.io, or consumed from an external workspace, this relative path to the repo root will fail to resolve.

Additionally, embedding ~2.4 MB of font binaries into every consumer binary inflates compile times and memory footprint. Per the Craft suite asset guidelines (AGENTS.md §3), font binaries are intended to live in storytold/craft-fonts via CRAFT_FONTS_DIR or runtime directory discovery, rather than being checked into git and baked into static binaries.

// `last_resort_face_parses` test proves it on every run.
#[allow(clippy::expect_used)]
let face = make_face(FontBytes::Static(BUNDLED[0]), 0, FALLBACK_FAMILY.into(), "Regular".into(), Vec::new())
.expect("the compiled-in Source Serif 4 Regular parses");

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#[allow(clippy::expect_used)] with .expect(...) bypasses the workspace #![deny(clippy::expect_used)] policy. If the embedded byte slice fails to parse or becomes corrupted during packaging, this will panic the host application.

last_resort_face() should return a fallible result (Result<Arc<FontFace>, FontError>) or safely propagate initialization errors rather than suppressing the panic lint.

fn path_d(p: &BezPath) -> String {
let mut d = String::new();
let pt = |d: &mut String, x: f64, y: f64| {
let _ = write!(d, "{} {}", fmt2(x), fmt2(y));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling fmt2(x) and fmt2(y) inside path_d allocates a new String via format!("{v:.2}") for every single coordinate point across all path segments (MoveTo, LineTo, QuadTo, CurveTo). In scenes containing dozens of glyphs and complex curves, this produces thousands of short-lived heap allocations per render frame.

Formatting directly into the output buffer without intermediate string allocations:

fn write_fmt2(out: &mut String, v: f64) {
    if !v.is_finite() {
        out.push_str("0.00");
    } else {
        use std::fmt::Write;
        let _ = write!(out, "{:.2}", if v == -0.0 { 0.0 } else { v });
    }
}

avoids heap churn during SVG serialization.

Comment thread Cargo.toml
craft-print = { path = "crates/print" }
craft-ui = { path = "crates/ui" }
equashion-math = { path = "crates/equashion-math" }
equashion-font = { path = "crates/equashion-font", default-features = false }

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The crate naming (equashion-*) and directory structure (crates/equashion-*) diverge from the workspace convention documented in line 14:

# internal: add each crate here as `craft-<name> = { path = "crates/<name>" }`

If these crates remain in craft-libs, standard ecosystem alignment would suggest craft-equation-* = { path = "crates/equation-*" }. However, given that equaSHion forms a specialized 5-crate math/typography engine consumed by WordCraft and DeckCraft, hosting it in a standalone repository (storytold/equashion) as suggested in the maintainer discussion would keep craft-libs focused on universal primitives.

dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…icodeMath (#1013)

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…theta round trip) (#1019)

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* craft-math: an operator name with a script keeps applying to its argument in LaTeX

to_latex wrote \sin^2\theta as {\sin }^{2}{\theta }, which reads back as
(\sin)^2 followed by θ: the braces cut the name off from its argument. An
operator name written as a command now takes its scripts directly
(\sin ^{2}{\theta }). The corpus test no longer needs its exception.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…cked) (#1038)

* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-mime: detect, for bytes that name no character set

is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* Add craft-flow: the flow model the text formats map to, on craft-wordmodel

WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.

* Add craft-markdown: Markdown read and written, on craft-flow

WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.

* WIP: craft-latex sources from WordCraft (unmodified)

* Add craft-latex: LaTeX documents read and written, on craft-flow

WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

* craft-latex: math through craft_math::latex_text; label timing test by ratio

The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…r documents (#1040)

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* craft-docx: WordCraft's crates/docx as it is at wordcraft 07f97737

The source, unchanged, so the next commit shows only what the move changes.
This commit does not build on its own (it still names wordcraft-doc and
wordcraft-geom, and is not a workspace member yet).

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* craft-docx: on craft-wordmodel, craft-opc, craft-xml and craft-offcrypto

- The package (zip reading, part names, relationships, content types,
  zip writing) is craft-opc's; the module keeps WordCraft's bookkeeping
  (passthrough keys, the embedded-part manifest) and its own Rel/Rels shape.
- Parts are parsed with craft-xml's canonical names (the Strict drawing
  names and mc:Choice/@requires through its hooks), then given the tree
  shape the reader was written against.
- Encryption and Restrict Editing hashes are craft-offcrypto's; this crate
  maps its errors and the w:documentProtection attributes.
- wordcraft-doc is craft-wordmodel; wordcraft-geom's helpers are its geom.

Tests that covered what now lives in craft-opc and craft-offcrypto are left to
those crates; the DIFAT regression is craft-cfb's. Test-only zip use follows
zip 9's fallible names. The attribute-cap test times the read only.

* craft-wordmodel: the math modules, symbol_fonts and Rgb are craft-math's

math, math_latex, math_linear, math_symbols, math_tree and symbol_fonts are
re-exported from craft-math at the paths they had, and props::Rgb is
craft-math's Rgb (TextColor stays), so the serde forms and the app's imports
do not change. Their tests are in craft-math.

* craft-docx: equations (OMML) are read and written by craft_math::omml

read/math.rs and write/math.rs keep their functions as adapters: the reader's
tree is handed to craft-math as a craft-xml tree, and the writer's output is
written into this crate's writer. Their tests are in craft-math.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…ft (#1050)

* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-mime: detect, for bytes that name no character set

is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* craft-codepage: one table per legacy code page, for craft-rtf, craft-dxf and craft-mime

craft-rtf and craft-dxf each carried tables for DOS 437, 737, 850, 852 and
857 (and craft-rtf one for 866). They are now one set in craft-codepage,
which takes craft-rtf's codepage module (Windows and Mac tables, OEM glyphs,
font charsets) and craft-dxf's DOS tables. craft_rtf::codepage re-exports it
unchanged; craft-dxf keeps its $DWGCODEPAGE names and Decoder and looks the
DOS tables up there.

Every byte of every code page either crate decodes is unchanged: the
public decoders of both crates were dumped over all 256 bytes of each page
(and the East Asian pairs) before and after, and the dumps are identical.

craft-mime now reads the DOS pages by their IANA names (IBM437, cp850,
ibm-852, 857...), which it read as UTF-8 before.

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* Add craft-flow: the flow model the text formats map to, on craft-wordmodel

WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.

* Add craft-markdown: Markdown read and written, on craft-flow

WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.

* WIP: craft-latex sources from WordCraft (unmodified)

* Add craft-latex: LaTeX documents read and written, on craft-flow

WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

* craft-latex: math through craft_math::latex_text; label timing test by ratio

The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.

* Cargo.lock after the merge

* WIP: WordCraft's HTML document reader and writer, unmodified (crates/formats/src/html at 07f97737)

* craft-html: HTML documents on craft-flow, behind a flow feature

WordCraft's crates/formats/src/html (07f97737) as craft_html::doc: the
tag-soup builder, CSS subset, ruby and MathML reading, page decoding and
the writer, with its 44 tests. It reads tokens from this crate's
tokenizer through a small adapter (decoded text and attributes, names in
lower case) instead of its own lexer, and its character references are
this crate's. Lexer::set_pos is new: the ruby reader hands a token back.
The HTML parts of WordCraft's MathML and cross-format tests come along as
tests/doc.rs, the linear-time check as a ratio.

* craft-math: operator names take their LaTeX scripts directly again (restores #1019)

#1019 (22161d2) wrote \sin^2\theta as \sin ^{2}{\theta } so it reads back as
the same tree. The squash merge of #1038, whose branch predated it, put
math_latex.rs and the corpus test back as they were before. This applies
#1019's change again, unchanged, with its test.

* Cargo.lock: craft-html's flow dependencies

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…from WordCraft (stacked) (#1064)

* Add craft-wordmodel: WordCraft's document model without its editing code

WordCraft's crates/doc (wordcraft-doc, at 07f97737) moved as it is, so the DOCX,
RTF, ODT and other text-format crates can read into and write from one model.
Module paths, type names and serde attributes are unchanged: an app that
re-exports the crate keeps its native file format.

Left in the app: range editing (edit.rs), IME text and equation editing. Not
model, and already in craft-libs: code pages (craft-rtf), the ZIP entry-count
check (craft-opc), the compound-file guard (craft-cfb). The paragraph walks
moved to walk.rs and the unit helpers of wordcraft-geom to geom.rs.

* craft-wordmodel: math_tree, the arguments of an equation node and a walk over its runs

The DOCX reader and writer walk an equation's runs. The walk and the child
accessors it uses were in WordCraft's equation editor (math_edit.rs), which
stays in the app; they are structure, not editing.

* craft-mime: detect, for bytes that name no character set

is_text, decode (BOM, unmarked UTF-16, UTF-8, else the legacy encoding
the bytes look like), legacy and legacy_label, behind a detect feature
that brings in chardetng. Moved from WordCraft's crates/formats/src/txt.rs
at 06c2ae6f with its text-sniffing tests. Also charset::is_known, so a
caller can tell a label decode would read as UTF-8 from one it knows.

* craft-wordmodel: keep the document operations (edit.rs) with the model

The DOCX and RTF readers use them: alternative-format chunks are merged into
a document with adopt_blocks, and a reader formats ranges with format_range
and inserts objects. They operate on the document, not on a view, so they
belong beside it. The tests that drove them come back with them; the
paragraph walks stay in walk.rs and edit re-exports them.

base64 0.23, as WordCraft and craft-offcrypto use.

* craft-codepage: one table per legacy code page, for craft-rtf, craft-dxf and craft-mime

craft-rtf and craft-dxf each carried tables for DOS 437, 737, 850, 852 and
857 (and craft-rtf one for 866). They are now one set in craft-codepage,
which takes craft-rtf's codepage module (Windows and Mac tables, OEM glyphs,
font charsets) and craft-dxf's DOS tables. craft_rtf::codepage re-exports it
unchanged; craft-dxf keeps its $DWGCODEPAGE names and Decoder and looks the
DOS tables up there.

Every byte of every code page either crate decodes is unchanged: the
public decoders of both crates were dumped over all 256 bytes of each page
(and the East Asian pairs) before and after, and the dumps are identical.

craft-mime now reads the DOS pages by their IANA names (IBM437, cp850,
ibm-852, 857...), which it read as UTF-8 before.

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* Add craft-flow: the flow model the text formats map to, on craft-wordmodel

WordCraft's crates/formats/src/model (07f97737) moved as it is: Flow,
FBlock, Para, Inline, Fmt, tables and cells, pictures, to_doc and
from_doc, picture sniffing, colours, base64 and the Dropped report. Only
the paths changed (wordcraft_doc is craft_wordmodel); the report helpers
are public so the format crates can call them; the image re-encoding
helper stays with WordCraft's RTF writer.

* Add craft-markdown: Markdown read and written, on craft-flow

WordCraft's crates/formats/src/markdown (07f97737) with its 35 tests,
moved onto craft-flow and craft-wordmodel. Character references come from
craft-html. The table cell budget lives in craft-flow, which names it in
its warning. Two test lines that wrote plain text through WordCraft's txt
module stay in WordCraft.

* WIP: craft-latex sources from WordCraft (unmodified)

* Add craft-latex: LaTeX documents read and written, on craft-flow

WordCraft's crates/formats/src/latex (07f97737) with its 21 tests, moved
onto craft-flow and craft-wordmodel; bytes are decoded by craft-mime's
detect. The LaTeX parts of WordCraft's cross-format tests (nesting,
column specs, labels, no TeX commands from the document, the fuzz and
round-trip properties) come along as tests/hostile.rs. The ligature test
builds its three-font paragraph as a flow instead of reading HTML.

* craft-math: latex_text, LaTeX math and the linear format converted as text

WordCraft's crates/formats/src/latex/math.rs (07f97737): the converter its
LaTeX document reader and writer use, which keeps the source's spacing.
It moves here so craft-math is the one place that converts LaTeX math;
craft-latex calls it, and its output does not change. Its tests (from
WordCraft's latex tests and cross-format tests) come along.

* craft-math: latex_text's group finders are public, for craft-latex's length parser

* craft-latex: math through craft_math::latex_text; label timing test by ratio

The document math converter moved to craft-math (#1040) unchanged; this
crate calls it there (and its length parser uses its group finders), so
export output is the same. The label test compared one read with a
ten-second wall clock and failed on a loaded machine; it now compares the
fastest of three reads of 25,000 and 100,000 labels, which a linear read
keeps near 4x and a quadratic one near 16x.

* Cargo.lock after the merge

* WIP: WordCraft's HTML document reader and writer, unmodified (crates/formats/src/html at 07f97737)

* craft-html: HTML documents on craft-flow, behind a flow feature

WordCraft's crates/formats/src/html (07f97737) as craft_html::doc: the
tag-soup builder, CSS subset, ruby and MathML reading, page decoding and
the writer, with its 44 tests. It reads tokens from this crate's
tokenizer through a small adapter (decoded text and attributes, names in
lower case) instead of its own lexer, and its character references are
this crate's. Lexer::set_pos is new: the ruby reader hands a token back.
The HTML parts of WordCraft's MathML and cross-format tests come along as
tests/doc.rs, the linear-time check as a ratio.

* Cargo.lock after the merge

* WIP: WordCraft's ODT reader and writer, unmodified (crates/formats/src/odt at 07f97737)

* craft-odf: OpenDocument Text documents on craft-flow, behind a text feature

WordCraft's crates/formats/src/odt (07f97737) as craft_odf::text, with
its 467 tests. Moved onto this crate's own layers: packages are read with
Package and written with PackageWriter, password protection is crypt
and flat files are flat (the copies in the WordCraft module are gone),
and its XML event stream is craft-xml's pull reader instead of quick-xml.
Paths: craft_flow, craft_wordmodel, craft_math::mathml, craft_metafile.
The date helper it borrowed from the spreadsheet reader came along.
Tests read packages through Package instead of zip.

* craft-math: operator names take their LaTeX scripts directly again (restores #1019)

#1019 (22161d2) wrote \sin^2\theta as \sin ^{2}{\theta } so it reads back as
the same tree. The squash merge of #1038, whose branch predated it, put
math_latex.rs and the corpus test back as they were before. This applies
#1019's change again, unchanged, with its test.

* Cargo.lock: craft-html's flow dependencies

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
dylanmtaylor added a commit that referenced this pull request Nov 12, 2026
…aft-math (#1027)

* Add craft-math: the equation tree with OMML, MathML, LaTeX and UnicodeMath

The math tree, LaTeX, linear-format, symbol and tree-walk modules come
from craft-wordmodel (#963, WordCraft's crates/doc at
07f97737) unchanged, with the run colour (Rgb) and the symbol-font tables
they use. MathML is WordCraft's crates/formats/src/mathml.rs and OMML is
WordCraft's crates/docx/src/read/math.rs and write/math.rs (06c2ae6f /
07f97737), moved onto craft-xml. The HTML-specific MathML tests stay in
WordCraft; the rest moved with the code.

mathml::parse and omml::parse/read_str/write_str are new: string entry
points for callers that have no tree of their own.

* craft-math: read MathLive's and other spellings of LaTeX math commands

\abs{...} (an absolute value), \lvert \rvert \lVert \rVert, \nparallel,
\backprime, \infinity, \cross, \differentialD, \exponentialE and
\imaginaryI (MathLive's d, upright e and i), \emph, \textsf, \hbox,
\mathnormal and \underleftarrow / \underrightarrow /
\underleftrightarrow were unknown and read as literal text. They are read
as equaSHion's typed-input parser reads them (#29).

* craft-math: equaSHion's equation corpus as test data

corpus/equations.tsv, corpus/latex.tsv and the OMML samples of
#29 (7a6ef6b), unmodified.

* craft-math: corpus round trips, GridCraft's OMML cases, entry points and limits tested; README

The corpus equations go through OMML, MathML and LaTeX and back; the OMML
samples are written back as kept and from the tree. GridCraft's OMML to
LaTeX test cases (crates/xlsx/src/omml.rs) give LaTeX that reads back the
same through the tree. omml::parse and read_str are tested against other
prefixes, the strict namespace, kept OMML, junk, the element cap and deep
nesting.

* craft-wordmodel: the math modules, symbol_fonts and Rgb are craft-math's

math, math_latex, math_linear, math_symbols, math_tree and symbol_fonts are
re-exported from craft-math at the paths they had, and props::Rgb is
craft-math's Rgb (TextColor stays), so the serde forms and the app's imports
do not change. Their tests are in craft-math.

---------

Co-authored-by: shuff57 <62350898+shuff57@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants