Repository navigation
fix(agent): return empty predictions without inference - #144
NandhaKishorM merged 1 commit into
Conversation
Empty question sets produce no batch, so inference dereferences None. Return an empty response before encoding to make dynamic zero-question requests safe while preserving normal predictions.
|
Review for PR #144: fix(agent): return empty predictions without inference Zero-Allocation Short-Circuit Analysis
|
|
Thanks for reviewing, @chilkotiKartik. Yes, the early return preserves the normal response shape and zero token usage while skipping encoding, batching, and model inference. The regression tests cover both One small clarification: the response dictionaries are still allocated, so this guarantees no tokenization or model inference, rather than literally zero allocations. No latency improvement has been measured. |
…#144) Empty question sets produce no batch, so inference dereferences None. Return an empty response before encoding to make dynamic zero-question requests safe while preserving normal predictions.
Summary
Calling
agent.predict("hello", {})oragent.system_one("hello", {})currently raisesTypeError: 'NoneType' object is not subscriptable: the empty question set produces no batch, which inference then dereferences. Return the standard response with empty answers and zero token usage before encoding, batching, or model inference.Changes
system_one, also used by itspredictalias. Leave nonempty inference and the shared collator unchanged.Testing
The new tests reproduced the original crash before the fix and pass afterward. Validation used Python 3.12, PyTorch 2.14.0+cpu, and Transformers 5.17.0 with Hugging Face offline mode. No pretrained weights were downloaded and no training ran.
test_download.pyretains the five known path-separator assertion failures addressed separately by #140.Ran
python tests/test_empty_questions.pyand the existingtest_download.py,test_router.py,test_criteria.py,test_shortlist.py,test_decision_model.py,test_packaging.py, andtest_email.pyscripts. The combined build also ran the calibration-persistence and temperature-loading regressions. Targeted Ruff checks,compileall, andgit diff --checkpassed. The checkpoint-dependenttest_local_e2e.pywas not run.Notes for reviewer
QuestionError; this focused PR instead defines an empty SDK request as a successful no-op. That difference needs reconciliation if both land. feat: TypeSafe Jev wire-compatible FastAPI server (/v1/systemone) + Pydantic v2 schemas #3's REST schemas separately reject empty question mappings; this PR does not change those HTTP schemas.mainand does not include the other contribution fixes. Integration checks used separate temporary copies; their branches remain unchanged.