feat: Add Ollama integration with Docker examples and CI tests (#62)

- Add quickstart example and documentation for local LLM usage
- Include Docker setup with health checks and docker-compose
- Add integration tests and update CI pipeline
- Secure setup: localhost-only binding, containerized deployment

Signed-off-by: Akshay Goel <[email protected]>

.github/workflows/ci.yaml

@@ -28,7 +28,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.10", "3.11"]
+        python-version: ["3.10", "3.11", "3.12"]
     steps:
       - uses: actions/checkout@v4
 
@@ -79,3 +79,54 @@ jobs:
             exit 0
           fi
           tox -e live-api
+
+  ollama-integration-test:
+    needs: test
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request'
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Detect file changes
+        id: changes
+        uses: tj-actions/changed-files@v44
+        with:
+          files: |
+            langextract/inference.py
+            examples/ollama/**
+            tests/test_ollama_integration.py
+            .github/workflows/ci.yaml
+
+      - name: Skip if no Ollama changes
+        if: steps.changes.outputs.any_changed == 'false'
+        run: |
+          echo "No Ollama-related changes detected – skipping job."
+          exit 0
+
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+
+      - name: Launch Ollama container
+        run: |
+          docker run -d --name ollama \
+            -p 127.0.0.1:11434:11434 \
+            -v ollama:/root/.ollama \
+            ollama/ollama:0.5.4
+          for i in {1..20}; do
+            curl -fs http://localhost:11434/api/version && break
+            sleep 3
+          done
+
+      - name: Pull gemma2 model
+        run: docker exec ollama ollama pull gemma2:2b || true
+
+      - name: Install tox
+        run: |
+          python -m pip install --upgrade pip
+          pip install tox
+
+      - name: Run Ollama integration tests
+        run: tox -e ollama-integration

.github/workflows/validate_pr_template.yaml

@@ -11,31 +11,31 @@ jobs:
   check:
     if: github.event.pull_request.draft == false   # drafts can save early
     runs-on: ubuntu-latest
-    
+
     steps:
       - name: Fail if template untouched
         env:
           PR_BODY: ${{ github.event.pull_request.body }}
         run: |
           printf '%s\n' "$PR_BODY" | tr -d '\r' > body.txt
-          
+
           # Required sections from the template
           required=( "# Description" "Fixes #" "# How Has This Been Tested?" "# Checklist" )
           err=0
-          
+
           # Check for required sections
           for h in "${required[@]}"; do
             grep -Fq "$h" body.txt || { echo "::error::$h missing"; err=1; }
           done
-          
+
           # Check for placeholder text that should be replaced
           grep -Eiq 'Replace this with|Choose one:' body.txt && {
-            echo "::error::Template placeholders still present"; err=1; 
+            echo "::error::Template placeholders still present"; err=1;
           }
-          
+
           # Also check for the unmodified issue number placeholder
           grep -Fq 'Fixes #[issue number]' body.txt && {
             echo "::error::Issue number placeholder not updated"; err=1;
           }
-          
+
           exit $err

README.md

@@ -17,6 +17,8 @@
 - [Quick Start](#quick-start)
 - [Installation](#installation)
 - [API Key Setup for Cloud Models](#api-key-setup-for-cloud-models)
+- [Using OpenAI Models](#using-openai-models)
+- [Using Local LLMs with Ollama](#using-local-llms-with-ollama)
 - [More Examples](#more-examples)
   - [*Romeo and Juliet* Full Text Extraction](#romeo-and-juliet-full-text-extraction)
   - [Medication Extraction](#medication-extraction)
@@ -256,13 +258,13 @@ result = lx.extract(
 LangExtract also supports OpenAI models. Example OpenAI configuration:
 
 ```python
-from langextract.inference import OpenAILanguageModel
+import langextract as lx
 
 result = lx.extract(
     text_or_documents=input_text,
     prompt_description=prompt,
     examples=examples,
-    language_model_type=OpenAILanguageModel,
+    language_model_type=lx.inference.OpenAILanguageModel,
     model_id="gpt-4o",
     api_key=os.environ.get('OPENAI_API_KEY'),
     fence_output=True,
@@ -272,6 +274,29 @@ result = lx.extract(
 
 Note: OpenAI models require `fence_output=True` and `use_schema_constraints=False` because LangExtract doesn't implement schema constraints for OpenAI yet.
 
+## Using Local LLMs with Ollama
+
+LangExtract supports local inference using Ollama, allowing you to run models without API keys:
+
+```python
+import langextract as lx
+
+result = lx.extract(
+    text_or_documents=input_text,
+    prompt_description=prompt,
+    examples=examples,
+    language_model_type=lx.inference.OllamaLanguageModel,
+    model_id="gemma2:2b",  # or any Ollama model
+    model_url="http://localhost:11434",
+    fence_output=False,
+    use_schema_constraints=False
+)
+```
+
+**Quick setup:** Install Ollama from [ollama.com](https://ollama.com/), run `ollama pull gemma2:2b`, then `ollama serve`.
+
+For detailed installation, Docker setup, and examples, see [`examples/ollama/`](examples/ollama/).
+
 ## More Examples
 
 Additional examples of LangExtract in action:
@@ -325,6 +350,17 @@ Or reproduce the full CI matrix locally with tox:
 tox  # runs pylint + pytest on Python 3.10 and 3.11
 ```
 
+### Ollama Integration Testing
+
+If you have Ollama installed locally, you can run integration tests:
+
+```bash
+# Test Ollama integration (requires Ollama running with gemma2:2b model)
+tox -e ollama-integration
+```
+
+This test will automatically detect if Ollama is available and run real inference tests.
+
 ## Development
 
 ### Code Formatting

examples/ollama/.dockerignore

@@ -0,0 +1,35 @@
+# Ignore Python cache
+__pycache__/
+*.pyc
+*.pyo
+*.pyd
+.Python
+
+# Ignore version control
+.git/
+.gitignore
+
+# Ignore OS files
+.DS_Store
+Thumbs.db
+
+# Ignore virtual environments
+venv/
+env/
+.venv/
+
+# Ignore IDE files
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# Ignore test artifacts
+.pytest_cache/
+.coverage
+htmlcov/
+
+# Ignore build artifacts
+build/
+dist/
+*.egg-info/

examples/ollama/Dockerfile

@@ -0,0 +1,23 @@
+# Copyright 2025 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+FROM python:3.11-slim-bookworm
+
+WORKDIR /app
+
+RUN pip install langextract
+
+COPY quickstart.py .
+
+CMD ["python", "quickstart.py"]

examples/ollama/README.md

@@ -0,0 +1,32 @@
+# Ollama Examples
+
+This directory contains examples for using LangExtract with Ollama for local LLM inference.
+
+For setup instructions and documentation, see the [main README's Ollama section](../../README.md#using-local-llms-with-ollama).
+
+## Quick Reference
+
+**Local setup:**
+```bash
+ollama pull gemma2:2b
+python quickstart.py
+```
+
+**Docker setup:**
+```bash
+docker-compose up
+```
+
+## Files
+
+- `quickstart.py` - Basic extraction example with configurable model
+- `docker-compose.yml` - Production-ready Docker setup with health checks
+- `Dockerfile` - Container definition for LangExtract
+
+## Model License
+
+Ollama models come with their own licenses. For example:
+- Gemma models: [Gemma Terms of Use](https://ai.google.dev/gemma/terms)
+- Llama models: [Meta Llama License](https://llama.meta.com/llama-downloads/)
+
+Please review the license for any model you use.

examples/ollama/docker-compose.yml

@@ -0,0 +1,42 @@
+# Copyright 2025 Google LLC.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+services:
+  ollama:
+    image: ollama/ollama:0.5.4
+    ports:
+      - "127.0.0.1:11434:11434"  # Bind only to localhost for security
+    volumes:
+      - ollama-data:/root/.ollama  # Cross-platform support
+    command: serve
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:11434/api/version"]
+      interval: 5s
+      timeout: 3s
+      retries: 5
+      start_period: 10s
+
+  langextract:
+    build: .
+    depends_on:
+      ollama:
+        condition: service_healthy
+    environment:
+      - OLLAMA_HOST=http://ollama:11434
+    volumes:
+      - .:/app
+    command: python quickstart.py
+
+volumes:
+  ollama-data: