Adding prompt caching

[jamesbraza]

This PR adds cache_breakpoint throughout the code base cache_control_injection_points to LiteLLM Router so we can get some caching and save money

[cursor]

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on March 5.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[dosubot]

Related Documentation

Checked 1 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[Copilot]

Pull request overview

This PR implements prompt caching support to reduce LLM API costs by adding cache_breakpoint=True to system messages that are reused across multiple requests.

Changes:

• Added cache_breakpoint=True to system messages in evidence gathering, query processing, and agent operations
• Updated dependency fhaviary from version 0.27 to 0.33 to support the cache_breakpoint feature
• Enhanced test coverage with assertions to verify prompt caching behavior for Anthropic models

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/paperqa/core.py	Added cache_breakpoint to system messages in evidence-gathering function
src/paperqa/docs.py	Added cache_breakpoint to reusable system message in query operations
src/paperqa/agents/main.py	Added cache_breakpoint to agent system prompt that's reused on every turn
pyproject.toml	Updated fhaviary dependency to v0.33 for cache_breakpoint support
tests/test_paperqa.py	Extended Anthropic test with caching assertions and padding to meet minimum token requirements

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[sidnarayanan]

Approving, but I haven't looked at why CI is failing