Repository navigation
fix(research): make the Chinese benchmark audit platform-independent - #216
Merged
NandhaKishorM merged 1 commit intoSep 23, 2026
Merged
Conversation
The feishu-benchmark CI job runs audit.py and its unittest suite with no dependencies, but it cannot run on a platform whose locale encoding is not UTF-8, for two independent reasons. 1. read_text()/write_text() without an explicit encoding fall back to the locale encoding, so reading the Chinese data/cases.jsonl raises UnicodeDecodeError on Windows (cp1252). 2. SOURCE.json pins a sha256 of the raw bytes of the frozen dataset and of every archived run, and verify_archive() checks them with read_bytes(). With git's default core.autocrlf=true on Windows those files check out as CRLF, so the integrity check fails on a clean clone. All nine archived files hash correctly only with LF, so the archive is pinned to eol=lf. Writers also get newline='\n', so a run produced off Linux stays byte-comparable with the committed archive. No logic and no numbers changed. Verified on Windows 11, Python 3.11, cp1252: both CI steps fail on main and pass after the change, and audit.py --write-summary now reproduces results/v1/summary.json byte-for-byte. Co-Authored-By: Claude Code <noreply@anthropic.com>
Contributor
|
Short answer: on a UTF-8-locale platform nothing changes, exactly as the PR describes - from the macOS side this is good to merge. Independent check on macOS (Darwin arm64, Python 3.14.4), checked out at 033d013 on top of 010bace (v0.3.7):
The explicit encoding='utf-8' reads and the LF pinning behave as advertised, and the .gitattributes comments document why. |
Owner
|
Thank you, and thanks @aashish254 for the independent macOS check. The feishu audit is the one suite that runs with no dependencies, so it should run on every platform. Explicit UTF-8 plus |
This was referenced Sep 24, 2026
baninaveen
pushed a commit
to baninaveen/laya
that referenced
this pull request
Sep 24, 2026
…andhaKishorM#216) The feishu-benchmark CI job runs audit.py and its unittest suite with no dependencies, but it cannot run on a platform whose locale encoding is not UTF-8, for two independent reasons. 1. read_text()/write_text() without an explicit encoding fall back to the locale encoding, so reading the Chinese data/cases.jsonl raises UnicodeDecodeError on Windows (cp1252). 2. SOURCE.json pins a sha256 of the raw bytes of the frozen dataset and of every archived run, and verify_archive() checks them with read_bytes(). With git's default core.autocrlf=true on Windows those files check out as CRLF, so the integrity check fails on a clean clone. All nine archived files hash correctly only with LF, so the archive is pinned to eol=lf. Writers also get newline='\n', so a run produced off Linux stays byte-comparable with the committed archive. No logic and no numbers changed. Verified on Windows 11, Python 3.11, cp1252: both CI steps fail on main and pass after the change, and audit.py --write-summary now reproduces results/v1/summary.json byte-for-byte. Co-authored-by: Claude Code <noreply@anthropic.com>
This was referenced Oct 6, 2026
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
feishu-benchmarkCI job runsaudit.pyand its unittest suite with no dependencies, which makes it the one suite a contributor can run anywhere. It cannot run on a platform whose locale encoding is not UTF-8, for two independent reasons.1. Text IO without an explicit encoding.
read_text()/write_text()fall back to the locale encoding, so reading the Chinesedata/cases.jsonlraisesUnicodeDecodeError: 'charmap' codec can't decode byte 0x8don Windows (cp1252). On Linux nothing changes —encoding='utf-8'is what the locale already supplied.2. No
.gitattributes.SOURCE.jsonpins a sha256 of the raw bytes of the frozen dataset and of every archived run, andverify_archive()checks them withread_bytes(). With git's defaultcore.autocrlf=trueon Windows those files check out as CRLF, so the integrity check fails on a clean clone:ValueError: Archived source bytes changed: data/cases.jsonl. All nine archived files hash correctly only with LF, so the archive is pinned toeol=lf. The stored blobs are already LF; this only stops a checkout from rewriting them.Writers (
run.py, andaudit.py --write-summary) also getnewline='\n', so a run produced off Linux stays byte-comparable with the committed archive instead of silently differing by line ending.No logic and no numbers changed.
Verification
On Windows 11, Python 3.11, cp1252 locale — both CI steps fail on
mainand pass after the change:And the strongest check:
audit.py --write-summaryon Windows now reproducesresults/v1/summary.jsonbyte-for-byte (sha256 ec4a60b4…), identical to the committed Linux-produced archive. Before the change it raisedUnicodeDecodeError.py_compileclean on all four touched scripts.git ls-files --eolreportsi/lf w/lffor every archived file.Related but separate: #212 adds a Windows lane for
tests/, which does not cover this job.