Skip to content

fix(agent): return empty predictions without inference - #144

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
MycelAI:alpha912/fix/empty-question-prediction
Sep 23, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
MycelAI:alpha912/fix/empty-question-prediction

Conversation

@alpha912

Copy link
Copy Markdown
Contributor

Summary

Calling agent.predict("hello", {}) or agent.system_one("hello", {}) currently raises TypeError: 'NoneType' object is not subscriptable: the empty question set produces no batch, which inference then dereferences. Return the standard response with empty answers and zero token usage before encoding, batching, or model inference.

Changes

  • Add a small early return in system_one, also used by its predict alias. Leave nonempty inference and the shared collator unchanged.
  • Document the empty-question behavior in the direct SDK guide and method docstring.
  • Add four CPU-only regression tests using an in-memory, randomly initialized tiny BERT and local tokenizer. Cover both methods and supported state shapes, skipped encoding/batching/forward calls, independent response containers, and normal choice/score/noul predictions before and after an empty request.

Testing

The new tests reproduced the original crash before the fix and pass afterward. Validation used Python 3.12, PyTorch 2.14.0+cpu, and Transformers 5.17.0 with Hugging Face offline mode. No pretrained weights were downloaded and no training ran.

Validation Result
This branch on Linux CPU Four new tests and all seven existing suites passed.
This branch on Windows CPU Four new tests and six existing suites passed. test_download.py retains the five known path-separator assertion failures addressed separately by #140.
Isolated combination with #139, #140, and #142 All ten suites passed on both Windows and Linux CPU. Matching source hashes verified the same code was tested.

Ran python tests/test_empty_questions.py and the existing test_download.py, test_router.py, test_criteria.py, test_shortlist.py, test_decision_model.py, test_packaging.py, and test_email.py scripts. The combined build also ran the calibration-persistence and temperature-loading regressions. Targeted Ruff checks, compileall, and git diff --check passed. The checkpoint-dependent test_local_e2e.py was not run.

Notes for reviewer

Empty question sets produce no batch, so inference dereferences None. Return an empty response before encoding to make dynamic zero-question requests safe while preserving normal predictions.
@chilkotiKartik

Copy link
Copy Markdown
Contributor

Review for PR #144: fix(agent): return empty predictions without inference

Zero-Allocation Short-Circuit Analysis

  1. Inference Short-Circuit:
    When questions is empty ({}), immediately returning {"model": ..., "answers": {}, "usage": {"input_tokens": 0, "output_tokens": 0}} avoids tokenizing state and constructing dummy PyTorch tensors for the cross-attention heads.

  2. API Contract Compatibility:
    The response schema remains 100% compliant with standard system_one() and predict() output dictionaries, ensuring downstream agent middleware does not encounter missing keys.

@alpha912

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing, @chilkotiKartik. Yes, the early return preserves the normal response shape and zero token usage while skipping encoding, batching, and model inference. The regression tests cover both system_one() and predict(), verify those calls are skipped, and check that nonempty predictions still work before and after an empty request.

One small clarification: the response dictionaries are still allocated, so this guarantees no tokenization or model inference, rather than literally zero allocations. No latency improvement has been measured.

@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you. Returning an empty, well-formed response without running the model is the friendlier contract for a server or a batch job, and it slots in cleanly next to the question validation from #183. Merging. #209 proposed raising instead, and I'll let them know we went this way.

@NandhaKishorM
NandhaKishorM merged commit fb14021 into NandhaKishorM:main Sep 23, 2026
baninaveen pushed a commit to baninaveen/laya that referenced this pull request Sep 24, 2026
…#144)

Empty question sets produce no batch, so inference dereferences None. Return an empty response before encoding to make dynamic zero-question requests safe while preserving normal predictions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants