Demo ollama on OpenShift

This repo contains information on how to deploy ollama on OpenShift.

Requirements

• OpenShift >= 4.15
• A GPU worker node with at least 16GB of GPU memory.
  • AWS
    • g4dn.2xlarge
    • g5.2xlarge

Installation Quickstart

Use CPU only

# setup ollama
until oc apply -k deploy; do : ; done

Use Nvidia GPU

# setup nvidia gpu nodes (prerequisite)
until oc apply -k deploy/nvidia-gpu-autoscale; do : ; done

# setup ollama w/ gpu
until oc apply -k deploy; do : ; done
until oc apply -k deploy/ollama-gpu; do : ; done

Setup Web Terminal (optional)

until oc apply -k deploy/web-terminal; do : ; done

See these additional notes on how to pull and test models.

Links

• Containerfile / Dockerfile
• Github - ollama
• docker.io - ollama container
• Github - ollama UBI container