Ollama PDF Processor

A comprehensive PDF processing system with Ollama integration for document Q&A, built with FastAPI, SQLModel, PostgreSQL, and Docker.

Features

• PDF Upload & Processing: Upload PDF files and extract text content automatically
• Document Management: Store, retrieve, and manage PDF documents with metadata
• Semantic Search: Search through document content using intelligent text matching
• Document Q&A: Chat with your documents using Ollama language models
• Background Processing: Asynchronous PDF processing with status tracking
• REST API: Complete RESTful API with OpenAPI documentation
• Docker Support: Containerized deployment with PostgreSQL and pgAdmin

Architecture

• FastAPI: Modern Python web framework for building APIs
• SQLModel: Type-safe database models with Pydantic integration
• PostgreSQL: Robust relational database for document storage
• Ollama: Local LLM integration for document Q&A
• Docker: Containerized deployment and development

Prerequisites

• Python 3.11+
• Docker & Docker Compose
• Ollama (running locally)

Quick Start

1. Install Ollama

First, install and start Ollama on your system:

# Install Ollama (visit https://ollama.ai for platform-specific instructions)
# Pull a model (e.g., llama2)
ollama pull llama2

2. Clone and Setup

# Clone the repository
git clone <your-repo-url>
cd ollama-pdf-processor

# Create uploads directory
mkdir uploads

3. Environment Configuration

Copy and modify the .env file if needed:

# The default configuration should work for most setups
# Modify OLLAMA_MODEL if you want to use a different model
#Sample Config:
# Database Configuration
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=postgres

# pgAdmin Configuration
[email protected]
PGADMIN_DEFAULT_PASSWORD=admin123

# Application Configuration
APP_NAME=Ollama PDF Processor
APP_VERSION=1.0.0
DEBUG=True
HOST=0.0.0.0
PORT=8000

# Ollama Configuration
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2-vision

# File Upload Configuration
MAX_FILE_SIZE_MB=50
UPLOAD_DIR=./uploads
ALLOWED_FILE_TYPES=pdf

4. Run with Docker Compose

# Start all services
docker-compose up -d

# Check service status
docker-compose ps

This will start:

• PostgreSQL on port 5432
• pgAdmin on port 5050 ([email protected] / admin123)
• FastAPI on port 8000

5. Verify Installation

Visit these URLs to verify everything is working:

• API Documentation: http://localhost:8000/docs
• Health Check: http://localhost:8000/health
• pgAdmin: http://localhost:5050

API Usage

📋 For detailed PDF upload instructions, see PDF_UPLOAD_GUIDE.md

Quick Start: Upload a PDF

Method 1: Web Interface (Easiest)

1. Visit http://localhost:8000/docs
2. Find POST /documents/upload
3. Click "Try it out" and choose your PDF
4. Click "Execute"

Method 2: Command Line

curl -X POST "http://localhost:8000/documents/upload" \\
     -H "accept: application/json" \\
     -H "Content-Type: multipart/form-data" \\
     -F "[email protected]"

Method 3: PowerShell (Windows)

$uri = "http://localhost:8000/documents/upload"
$filePath = "C:\\path\\to\\your\\document.pdf"
Invoke-RestMethod -Uri $uri -Method Post -Form @{ file = Get-Item -Path $filePath }

Method 4: Upload Scripts (Windows)

For convenience, we've included upload scripts:

# Simple upload
upload-pdf.bat "path\\to\\your\\document.pdf"

# Upload and wait for processing
upload-pdf.bat "path\\to\\your\\document.pdf" -wait

Or use the PowerShell script directly:

.\\upload-pdf.ps1 -PdfPath "path\\to\\your\\document.pdf" -WaitForProcessing

List Documents

curl -X GET "http://localhost:8000/documents"

Search Documents

curl -X POST "http://localhost:8000/search" \\
     -H "Content-Type: application/json" \\
     -d '{
       "query": "your search query",
       "limit": 10
     }'

Chat with Documents

curl -X POST "http://localhost:8000/chat" \\
     -H "Content-Type: application/json" \\
     -d '{
       "message": "What is this document about?",
       "context_limit": 5
     }'

API Endpoints

Document Management

• POST /documents/upload - Upload a PDF file
• GET /documents - List all documents
• GET /documents/{id} - Get specific document
• DELETE /documents/{id} - Delete document
• GET /documents/{id}/chunks - Get document chunks

Search & Chat

• POST /search - Semantic search across documents
• POST /chat - Chat with documents using Ollama

System

• GET /health - Health check and system status
• GET /ollama/models - List available Ollama models
• POST /ollama/pull/{model} - Pull new Ollama model

Development

Run Locally (without Docker)

1. Install dependencies:

pip install -r requirements.txt

2. Set up PostgreSQL locally or use Docker:

docker run --name postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres:15

3. Run the application:

uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

Database Management

The application uses SQLModel with PostgreSQL. Database tables are created automatically on startup.

To access the database:

• pgAdmin: http://localhost:5050
• Direct connection: postgresql://postgres:postgres@localhost:5432/postgres

Logging

Application logs are available in the container:

docker-compose logs api

Configuration

Key environment variables in .env:

• DATABASE_URL: PostgreSQL connection string
• OLLAMA_BASE_URL: Ollama service URL
• OLLAMA_MODEL: Default model for document Q&A
• MAX_FILE_SIZE_MB: Maximum PDF file size
• CHUNK_SIZE: Text chunk size for processing
• CHUNK_OVERLAP: Overlap between text chunks

Troubleshooting

Common Issues

1. Ollama connection failed:

   • Ensure Ollama is running: ollama serve
   • Check if the model is available: ollama list
   • Pull required model: ollama pull llama2

2. Database connection error:

   • Verify PostgreSQL is running: docker-compose ps
   • Check database logs: docker-compose logs postgres

3. PDF processing fails:

   • Check file size limits
   • Verify PDF is not password-protected
   • Check application logs: docker-compose logs api

Health Check

Use the health endpoint to diagnose issues:

curl http://localhost:8000/health

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

For issues and questions:

1. Check the troubleshooting section
2. Review the API documentation at /docs
3. Check application logs
4. Open an issue on the repository