Repository navigation
Conversation
…d when it cannot On transformers 4.x `build_model` reaches `transformers.modeling_utils` for `no_init_weights`, and importing that module runs a chain that ends in `import tensorflow`: ``` laya/common.py:579 _no_init_weights -> transformers/modeling_utils.py:70 -> transformers/loss/loss_utils.py:21 -> transformers/loss/loss_d_fine.py:21 -> transformers/loss/loss_for_object_detection.py:32 -> transformers/image_transforms.py:48 -> tensorflow/__init__.py:40 -> Fatal Python error: Bus error ``` Laya's model path is torch-only, so in an environment where TensorFlow is installed but cannot load, `laya.load()` dies **natively** — not an exception that can be caught, the interpreter is gone — because of a library nothing here uses. Reported in NandhaKishorM#915 with the full traceback. The fix is the one the reporter suggested: `os.environ.setdefault("USE_TF", "0")` at the top of `laya/__init__.py`, above every import. **`setdefault`, not assignment.** A caller who set `USE_TF` deliberately keeps it, including `1` for someone who wants TensorFlow in the same process. Verified for both `"1"` and `"0"`. **It has to be in `__init__.py` and above the imports**, because transformers decides TensorFlow's availability when it is first imported. `import laya` deliberately imports no transformers at all — the torch-backed names are lazy (`_LAZY_ATTRS`) — so this runs first in the normal case, and that is checked rather than assumed: a subprocess asserts `"tensorflow" not in sys.modules` after `import laya` and that no `transformers.*` module was pulled in by it. ## The repository already knew this `USE_TF=0` is set by CI, by the Dockerfile, and by six test files as a precondition: ``` tests/test_load_errors.py:30 tests/test_download.py:20 tests/test_head_checkpointing.py:12 tests/test_empty_questions.py:13 tests/test_calibration_persistence.py:16 sdk_server_fixture.py:9 ``` and `grep USE_TF laya/` returned **nothing** before this change. The test suite worked around a hazard the library created and never defended against, so a plain `import laya` script hit it. This moves the knowledge to where it takes effect — the same shape as the `lang_temperatures` drift in NandhaKishorM#428. ## Tests Five checks in `tests/test_lazy_import.py`, in subprocesses because an in-process assertion could not distinguish "the guard worked" from "it was too late but harmless": ``` USE_TF/import laya succeeds with tensorflow blocked -> 0 USE_TF/is set to 0 on import -> '0' USE_TF/tensorflow is not imported -> False USE_TF/a caller's '1' is preserved -> '1' USE_TF/a caller's '0' is preserved -> '0' ``` The TensorFlow blocker raises `ImportError` rather than segfaulting, so if laya's import chain reaches for TensorFlow at all the check fails loudly instead of taking the test process with it. **Confirmed the tests catch the bug:** with the guard line replaced by `pass`, the suite reports `12 passed, 1 failed — USE_TF/is set to 0 on import: got 'None', want '0'`. ``` $ python tests/test_lazy_import.py 13 passed, 0 failed (9 before) $ python tests/test_imports.py All imports tests passed. $ python -m pytest tests/test_serve.py tests/test_router_batch.py \ tests/test_system_one_lang.py tests/test_audit_regressions.py -q 308 passed $ python -m ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 All checks passed! ``` ## Limitation, stated plainly **I could not reproduce the reported crash, and this is not a fix I observed working.** This box runs transformers **5.17.0**, where `transformers.initialization.no_init_weights` exists and the `modeling_utils` fallback is never taken — so the TensorFlow import chain does not run here at all, and the crash needs a *broken* TensorFlow to trigger. The evidence I have is: - the reporter's traceback, which names `common.py:579` and the exact chain above; - the reporter's own controls — `USE_TF=0` makes the same environment work in ~5 s, and a fresh venv (transformers 5) never crashes; - that the guard is set before transformers can be imported, measured; - that `setdefault` preserves a caller's value, measured. So the mechanism is confirmed and the guard is verified to do what it claims; the end-to-end crash is verified only by the reporter's environment. Worth a note on NandhaKishorM#915 asking them to confirm.
…` on a re-gate
A `results` payload gated once and then gated again kept the first call's flag, because
`flag_low_confidence` only ever **adds** `low_confidence: True` (`confidence.py:158`) and never takes
one back — correct for a function whose docstring says the flag "is added", but wrong underneath a
second gate. `abstention` *was* recomputed each call, so the two halves of the report disagreed:
```python
a = {"choice": "refund", "answer_confidence": 0.8}
r = {"answers": {"intent": a}}
apply_confidence_gate([r], min_confidence=0.9)
# {'low_confidence': True, 'abstention': 'abstained', 'abstention_threshold': 0.9}
apply_confidence_gate([r], min_confidence=0.5) # 0.8 clears 0.5
# before: {'low_confidence': True, 'abstention': 'abstained', 'abstention_threshold': 0.5}
# ^ stale ^ stale ^ updated
```
The result is **self-contradictory**: the threshold says 0.5 and the verdict says `abstained`, a
decision that was never made at that threshold. A fresh dict at 0.5 gives
`{'low_confidence': None, 'abstention': 'passed', 'abstention_threshold': 0.5}`.
Reusing a result dict is the normal way this API is used, because the function **mutates in place
and returns `None`** — there is nothing to signal that a second call needs fresh input. Reported in
NandhaKishorM#910.
## The fix
Clear `low_confidence` before delegating, so what the gate reports is decided at the threshold of
*this* call and nowhere else:
```python
for a in gated:
a.pop("low_confidence", None)
flag_low_confidence(results, min_confidence)
```
The answers are now gathered once into `gated` and walked twice, instead of walking `results` twice
with the same filter — the clearing and the state-writing are two rules over one set.
Deliberate choices tested:
- **Cleared in `apply_confidence_gate`, not in `flag_low_confidence`.** That function's contract is "the flag is added", it is public and exported in `laya.__all__`, and someone calling it directly on a accumulated payload may well be relying on the flag persisting. The gate is the layer that claims to *report* a decision, and it already owns writing `abstention`/`abstention_threshold` onto the answers — so it is where an answer's gate state should be made whole.
- **The direction is asserted both ways.** Loosening clears the flag; tightening sets it again. Clearing only in one direction would be a pass-through rather than a re-decision.
- **`0.0` clears too.** A re-gate at 0.0 is a gate that ran and nothing can fail, so it reports `passed` and carries no flag — the same state a fresh 0.0 gate produces, which the existing `flag_low_confidence` no-op is what gives it.
- **A per-bucket map re-gate is covered as well**, since the flag there is decided by the map's threshold for that answer's bucket rather than by the scalar the earlier call used.
- **The strongest assertion is equivalence, not the individual fields:** after re-gating, the answer must equal what a fresh gate at that threshold produces. That is the property a caller actually needs, and it catches disagreements between the flag and the state that checking either alone would miss.
## Tests
Eleven checks added to `tests/test_confidence.py`, covering looser, tighter, `0.0` and map
re-gates, plus the fresh-gate equivalence.
**Confirmed the tests catch the bug:** with the clearing loop removed, the suite reports
`126 passed, 7 failed`, including the exact contradictory state from the issue —
```
FAIL gate/re-gated looser matches a fresh gate:
got {..., 'low_confidence': True, 'abstention': 'abstained', 'abstention_threshold': 0.3}
want {..., 'abstention': 'passed', 'abstention_threshold': 0.3}
```
```
$ python tests/test_confidence.py 133 passed, 0 failed (122 before)
$ python tests/test_conformal_abstention.py 20 passed, 0 failed
$ python tests/test_batch.py 78 passed, 0 failed
$ python tests/test_cli.py all CLI tests passed
$ python tests/test_structured.py all structured tests passed
$ python -m pytest tests/test_serve.py tests/test_router_batch.py \
tests/test_system_one_lang.py tests/test_audit_regressions.py -q
308 passed
$ python -m ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!
$ python -m compileall -q laya/ tests/
```
Known limitation: this makes a re-gate authoritative for the answers it is given, which is a
behaviour change for anyone deliberately calling the gate twice to accumulate the *strictest*
verdict across thresholds. I do not think that is a supported pattern — the function reports "the
gate's state" and echoes one threshold, so a mixture of two thresholds is not representable in its
own output — but it is worth a word from the maintainer if it was intended. The alternative shape
would be to add a separate accumulate flag rather than change the default.
|
Closing this one, because both halves of it shipped in 0.3.27 through separate pull requests.
The reason I split them rather than taking this PR is That is a process note, not a criticism of either fix. Both were right, and your version of the confidence fix was equivalent to the one that shipped. One thing from this PR that did not land and that I would take as its own change: |
|
You are right and I was wrong — I checked before replying, and the stacking is real.
so the diff opened against 0.3.26 carried all four files — Worth recording how the mistake happened, since it is a process failure rather than a typo: I Two things from your note, taken up: The confidence fix — nothing to follow up. I ran my PR's eleven re-gate checks against 0.3.27 The The thing I did take up is #935, the same per-call hook gap on the |
What this fixes
A
esults payload gated once and then gated again kept the first call's flag. lag_low_confidence
only ever adds
low_confidence: True(confidence.py:158) and never takes one back — correctfor a function whose docstring says the flag "is added", but wrong underneath a second gate.
abstentionwas recomputed each call, so the two halves of the report disagreed:The result is self-contradictory: the threshold says 0.5 and the verdict says
abstained, adecision that was never made at that threshold. A fresh dict at 0.5 gives
{'low_confidence': None, 'abstention': 'passed', 'abstention_threshold': 0.5}.Reusing a result dict is the normal way this API is used, because the function mutates in place and
returns
None— there is nothing to signal that a second call needs fresh input. Reported in #910.The fix
Clear
low_confidencebefore delegating, so what the gate reports is decided at the threshold ofthis call and nowhere else. The answers are now gathered once into
gatedand walked twice,instead of walking
resultstwice with the same filter — clearing and state-writing are two rulesover one set.
Deliberate choices tested:
apply_confidence_gate, not inflag_low_confidence. That function's contract is "the flag is added", it is public and exported inlaya.__all__, and a caller using it directly on an accumulated payload may be relying on the flag persisting. The gate is the layer that claims to report a decision, and it already owns writingabstention/abstention_thresholdonto the answers.0.0clears too — a gate that ran and nothing can fail reportspassedwith no flag, which is the state a fresh 0.0 gate produces.Tests
Eleven checks added to
tests/test_confidence.py: looser, tighter,0.0and map re-gates, plusthe fresh-gate equivalence.
Confirmed the tests catch the bug — with the clearing loop removed:
One thing worth your view
This makes a re-gate authoritative for the answers it is given, which is a behaviour change for
anyone calling the gate twice deliberately to accumulate the strictest verdict across thresholds.
I do not think that is a supported pattern — the function reports "the gate's state" and echoes a
single threshold, so a mixture of two is not representable in its own output — but if it was
intended, the alternative shape is an explicit accumulate argument rather than changing the default.
Happy to switch it.