Skip to content

ollama: send keep_alive at top level #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chinmaynadgir wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ollama: send keep_alive at top level #267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions langextract/providers/ollama.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -348,10 +348,12 @@ def _ollama_query(
)

options: dict[str, Any] = {}
if keep_alive is not None:
options['keep_alive'] = keep_alive
else:
options['keep_alive'] = _DEFAULT_KEEP_ALIVE
# Compute keep_alive once; retain in options for back-compat and
# also send at the top level to match Ollama API.
keep_alive_value = (
keep_alive if keep_alive is not None else _DEFAULT_KEEP_ALIVE
)
options['keep_alive'] = keep_alive_value

if seed is not None:
options['seed'] = seed
Expand Down Expand Up @@ -402,6 +404,8 @@ def _ollama_query(
'raw': raw,
'options': options,
}
# Canonical: keep_alive should be at the top level per Ollama API
payload['keep_alive'] = keep_alive_value

if structured_output_format is not None:
payload['format'] = structured_output_format
Expand Down
4 changes: 4 additions & 0 deletions tests/inference_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,8 @@ def test_ollama_extra_kwargs_passed_to_api(self, mock_post):
call_args = mock_post.call_args
json_payload = call_args.kwargs["json"]

# keep_alive should be present at top-level for Ollama API
self.assertEqual(json_payload["keep_alive"], 600)
self.assertEqual(json_payload["options"]["keep_alive"], 600)
self.assertEqual(json_payload["options"]["num_thread"], 8)
# timeout is passed to requests.post, not in the JSON payload
Expand Down Expand Up @@ -238,6 +240,8 @@ def test_ollama_defaults_when_unspecified(self, mock_post):
call_args = mock_post.call_args
json_payload = call_args.kwargs["json"]

# keep_alive defaults at the top-level too
self.assertEqual(json_payload["keep_alive"], 300)
self.assertEqual(json_payload["options"]["temperature"], 0.1)
self.assertEqual(json_payload["options"]["keep_alive"], 300)
self.assertEqual(json_payload["options"]["num_ctx"], 2048)
Expand Down
Loading