Skip to content

fix(ts): port the From: header rules the Python side already has - #422

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
PerryLink:fix/ts-email-from-header
Sep 25, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
PerryLink:fix/ts-email-from-header

Conversation

@PerryLink

Copy link
Copy Markdown
Contributor

fix(ts): port the From: header rules the Python side already has

The TypeScript port kept the pre-#338 form of all three header patterns, so it deleted the rest
of a request whose body contained a line starting From: .

  • QUOTE_HEADERS matched /^\s*From:\s.+$/i where Python requires an address
    (/^\s*From:\s.*[@<]/i), so a prose line — "From: my side the integration works, but the
    downstream job still times out." — cut the message there.
  • HEADER_FROM_NAME was /^\s*De:\s+\S/i and HEADER_NEXT had no Sent: and no Date:, so
    the English half of the pair was missing: a bare "From: Maria Souza" header was never
    recognised even with its own Sent: line beside it. Both now mirror laya/email.py:46-47.

#371 changed exactly these three lines in laya/email.py and tests/test_email.py and merged
on 2026-09-24; the port was not updated with it. laya-ts/tests/ had no From: case at all,
which is why this stayed green — the same reason the Python suite did, as #371's own description
notes.

Verified against Python rather than by eye. Every expectation in the new tests was produced by
running laya.clean_email_body on the same input first:

                         Python result
From prose   -> the body, unchanged
From addr    -> 'Please refund my order'
From+Sent    -> 'Please refund order 123.'
From bare    -> the body, unchanged
De+Date      -> 'Please refund order 123.'

Before the change the same five cases give, in the copy at laya-ts/:

F1 keeps the request (Python: clean_email_body)
AssertionError: expected 'Hi team, the export failed again this…' to be 'Hi team, the export failed again this…'
  - From: my side the integration works, but the downstream job still times out.
  - Could you take a look before Friday?

F4 cuts 'De: Maria Souza' + 'Sent:' (Python does)
  + De: Maria Souza
  + Sent: Monday

After it, all five pass and the suite goes 253 → 258:

$ npx tsc -p tsconfig.json
$ npx vitest run --reporter=basic
 Test Files  16 passed (16)
      Tests  258 passed (258)

Deliberate choices tested:

  • The regexes are copied from Python, not re-derived. These three lines have a history on the Python side — #338, #346, #371 — and the port's job is to match the reference, so the fix is the reference's text plus a comment saying why each part is there. A re-derived equivalent would be a second place for the two sides to drift.
  • The comments are ported too. The Python source explains why From: needs an address and why a bare From: needs its neighbours; without that, the next person tightening the pattern has no way to know which half is load-bearing.
  • One PR for both patterns. They are the same rule seen from two sides — the marker that cuts on an address, and the neighbour rule that cuts on a header line — and #371 changed them in one commit for the same reason.
  • No other port gaps touched. The audit that found this also found divergences in lang.ts and common.ts; they are separate PRs, so this diff is three lines of source and the tests for them.

Known limitation: this is laya-ts only. The SIGNOFF_TAIL class in email.ts:25 is deliberately
not equivalent to Python's [^\W\d_a-zß-öø-ÿ] — Python's class admits lowercase letters outside
Latin-1, so Python cuts a sign-off like Regards, āā and the port keeps it. The port matches the
Python comment's stated intent, so that divergence is a bug in laya/email.py, not here, and is
not addressed by this PR.

The TypeScript port kept the pre-NandhaKishorM#338 form of all three header patterns, so it deleted the rest
of a request whose body contained a line starting `From: `.

- `QUOTE_HEADERS` matched `/^\s*From:\s.+$/i` where Python requires an address
  (`/^\s*From:\s.*[@<]/i`), so a prose line — "From: my side the integration works, but the
  downstream job still times out." — cut the message there.
- `HEADER_FROM_NAME` was `/^\s*De:\s+\S/i` and `HEADER_NEXT` had no `Sent:` and no `Date:`, so
  the English half of the pair was missing: a bare "From: Maria Souza" header was never
  recognised even with its own `Sent:` line beside it. Both now mirror `laya/email.py:46-47`.

`NandhaKishorM#371` changed exactly these three lines in `laya/email.py` and `tests/test_email.py` and merged
on 2026-09-24; the port was not updated with it. `laya-ts/tests/` had no `From:` case at all,
which is why this stayed green — the same reason the Python suite did, as NandhaKishorM#371's own description
notes.

Verified against Python rather than by eye. Every expectation in the new tests was produced by
running `laya.clean_email_body` on the same input first:

```
                         Python result
From prose   -> the body, unchanged
From addr    -> 'Please refund my order'
From+Sent    -> 'Please refund order 123.'
From bare    -> the body, unchanged
De+Date      -> 'Please refund order 123.'
```

Before the change the same five cases give, in the copy at `laya-ts/`:

```
F1 keeps the request (Python: clean_email_body)
AssertionError: expected 'Hi team, the export failed again this…' to be 'Hi team, the export failed again this…'
  - From: my side the integration works, but the downstream job still times out.
  - Could you take a look before Friday?

F4 cuts 'De: Maria Souza' + 'Sent:' (Python does)
  + De: Maria Souza
  + Sent: Monday
```

After it, all five pass and the suite goes 253 → 258:

```
$ npx tsc -p tsconfig.json
$ npx vitest run --reporter=basic
 Test Files  16 passed (16)
      Tests  258 passed (258)
```

Deliberate choices tested:

- **The regexes are copied from Python, not re-derived.** These three lines have a history on the Python side — `NandhaKishorM#338`, `NandhaKishorM#346`, `NandhaKishorM#371` — and the port's job is to match the reference, so the fix is the reference's text plus a comment saying why each part is there. A re-derived equivalent would be a second place for the two sides to drift.
- **The comments are ported too.** The Python source explains why `From:` needs an address and why a bare `From:` needs its neighbours; without that, the next person tightening the pattern has no way to know which half is load-bearing.
- **One PR for both patterns.** They are the same rule seen from two sides — the marker that cuts on an address, and the neighbour rule that cuts on a header line — and `NandhaKishorM#371` changed them in one commit for the same reason.
- **No other port gaps touched.** The audit that found this also found divergences in `lang.ts` and `common.ts`; they are separate PRs, so this diff is three lines of source and the tests for them.

Known limitation: this is `laya-ts` only. The `SIGNOFF_TAIL` class in `email.ts:25` is deliberately
not equivalent to Python's `[^\W\d_a-zß-öø-ÿ]` — Python's class admits lowercase letters outside
Latin-1, so Python cuts a sign-off like `Regards, āā` and the port keeps it. The port matches the
Python comment's stated intent, so that divergence is a bug in `laya/email.py`, not here, and is
not addressed by this PR.
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you for bringing the TypeScript email cleaner in line with #371. The three patterns now match laya/email.py exactly. I ran a set of bodies through both implementations and they agree on every one, including prose lines starting with From: that used to delete the rest of a request. Generating the test expectations from the Python function first is a great habit. Merging.

@NandhaKishorM
NandhaKishorM merged commit c454ee0 into NandhaKishorM:main Sep 25, 2026
19 checks passed
NandhaKishorM pushed a commit that referenced this pull request Sep 27, 2026
`[^\W\d_a-zß-öø-ÿ]` excludes the lowercase letters only where Latin-1 spells them, so a
name starting with ł, ż, д or α counted as capitalised-in-any-script, the line was cut as
a sign-off, and whatever followed it went with the cut:

$ python -c "from laya.email import clean_email_body as c; print(c('Hi,\n\nPlease refund invoice 4411.\nThanks, żaneta\nCould you confirm by Friday?'))"
Hi,

Please refund invoice 4411.

The TS port asks the case question per character instead (`[\p{Lu}\p{Lt}\p{Lo}]`), which is
the rule #245's comment describes — "a name is capitalised in any script" — so Python moves
to it: the tail is matched structurally, and each token's first character is judged by
Unicode category (Lu/Lt/Lo). A combining mark rides along with the letter before it
(`Jose\u0301` spells `José`), which `\w` alone does not cover.

$ python tests/test_email.py      # before the fix, with the new cases
105 passed, 5 failed
$ python tests/test_email.py      # after
110 passed, 0 failed

Deliberate choices tested:

- **Categories, not a predicate.** `token[0].isalpha() and not token[0].islower()` accepts
  modifier letters (`Thanks, ʻŌlelo`), which the port rejects; the category check agrees
  with the port on the whole 58-case corpus, the predicate does not.
- **A callable marker, not a wider class.** `re` cannot say "not a lowercase letter in any
  script"; listing ranges is what failed. The English sign-off joins `_SIGNATURE_MARKERS` as
  a callable (the port's list is callables too), and `_marker_matches` dispatches it.
- **Marks via `\u0300-\u036f`.** The combining diacriticals are the block a decomposed
  Latin-script name actually carries; other mark blocks are not covered, matching `\w`.

Known limitation: a mark from another block (a Devanagari matra, say) still ends the token
in Python where the port's `\p{M}` carries it. The port needs no code change, only its
comment, which quoted the old class. Related: #422 touches other hunks of
`laya-ts/src/email.ts`; no line overlap.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants