GitLab Repository Migration Tool

A powerful Python script to automatically migrate repositories between GitLab instances with support for structural reorganization, repository renaming, and selective migration.

🚀 Features

• Cross-Instance Migration: Transfer repositories between different GitLab instances
• Structural Mapping: Reorganize group structures during migration
• Repository Renaming: Change repository names during the transfer
• Selective Migration: Choose specific repositories using a whitelist
• Dry Run Mode: Preview migration without making changes
• Interactive Confirmations: Get prompted for confirmation at critical steps
• Detailed Logging: Comprehensive logs with emoji indicators for better readability

📋 Prerequisites

• Python 3.6 or higher
• GitLab access tokens with appropriate permissions:
  • Source GitLab: read_repository, api
  • Target GitLab: read_repository, api, write_repository

🔧 Installation

1. Set up a virtual environment:

python3 -m venv gitlab-migration
source gitlab-migration/bin/activate  # On Windows: gitlab-migration\Scripts\activate

1. Install dependencies:

pip install python-gitlab GitPython

1. Download the script:

curl -O https://raw.githubusercontent.com/yourusername/gitlab-migration/main/migration.py
# OR
# wget https://raw.githubusercontent.com/yourusername/gitlab-migration/main/migration.py

⚙️ Configuration

Edit the script to configure the following:

1. GitLab Connection Settings:

SOURCE_GITLAB_URL = "https://git.source.com"
TARGET_GITLAB_URL = "https://git.target.com"

1. Structure Mapping:

python

STRUCTURE_MAPPING = {
    "source-repo-group": "reponew/target-repo-group",
    # Add more mappings as needed
}

1. Repository Name Mapping:

python

REPO_NAME_MAPPING = {
    "source-repo": "target-repo",
    # Add more mappings as needed
}

1. Whitelist (Optional):

python

WHITELIST_REPOS = [
    "allowed-repo-one",
    "allowed-repo-two",
    # Add more repositories as needed
]

🔐 Authentication

Set your GitLab access tokens as environment variables:

bash

export SOURCE_GITLAB_TOKEN="your_source_gitlab_token"
export TARGET_GITLAB_TOKEN="your_target_gitlab_token"

🖥️ Usage

Basic Migration

Migrate all repositories with interactive confirmations:

bash

python gitlab-migration.py

Preview Migration (Dry Run)

See what would happen without making any changes:

bash

python gitlab-migration.py --dry-run

Migrate Specific Repositories

Migrate only repositories in the whitelist:

bash

python gitlab-migration.py --whitelist

Automated Mode (No Confirmations)

Run migration without prompting for confirmations:

bash

python gitlab-migration.py --no-confirm

Combine Options

Options can be combined as needed:

bash

python gitlab-migration.py --whitelist --dry-run
python gitlab-migration.py --whitelist --no-confirm

🔄 Automated Scheduled Migration

Using Cron

Add a cron job to run nightly migration:

bash

# Edit crontab
crontab -e

# Add this line to run daily at 2 AM
0 2 * * * cd /path/to/script && source /path/to/gitlab-migration/bin/activate && python gitlab-migration.py --no-confirm --whitelist >> /path/to/logs/migration_$(date +\%Y\%m\%d).log 2>&1

Using Docker

Create a Dockerfile:

dockerfile

FROM python:3.9-slim

WORKDIR /app

# Install dependencies
RUN pip install python-gitlab GitPython

# Copy script
COPY migration.py /app/

# Set environment variables (to be provided at runtime)
ENV SOURCE_GITLAB_TOKEN=""
ENV TARGET_GITLAB_TOKEN=""

# Add crontab entry for nightly run
RUN apt-get update && apt-get -y install cron &&\
    echo "0 2 * * * cd /app && python gitlab-migration.py --no-confirm --whitelist >> /var/log/migration_\$(date +\%Y\%m\%d).log 2>&1" >> /etc/cron.d/gitlab-cron &&\
    chmod 0644 /etc/cron.d/gitlab-cron &&\
    crontab /etc/cron.d/gitlab-cron

# Volume for logs
VOLUME /var/log

# Run cron in foreground
CMD ["cron", "-f"]

Build and run:

bash

docker build -t gitlab-migration .
docker run -d --name gitlab-migration\
  -e SOURCE_GITLAB_TOKEN="your_source_token"\
  -e TARGET_GITLAB_TOKEN="your_target_token"\
  -v /path/on/host/logs:/var/log\
  gitlab-migration

📊 Migration Phases

1. Connection Phase: Connect to source and target GitLab instances
2. Analysis Phase: Scan source GitLab structure and determine what to migrate
3. Migration Phase: Create groups, clone repositories, and push to target

⚠️ Best Practices

1. Always run with --dry-run first to see what would happen
2. Start with a whitelist containing a small number of repositories
3. Use interactive mode for important migrations
4. Consider target repositories as read-only to prevent conflicts
5. Schedule migrations during off-hours to minimize impact

🔎 Troubleshooting

Connection Issues

If you encounter connection problems:

❌ Failed to connect to GitLab at https://git.example.com

Check:

• GitLab token has correct permissions
• GitLab URL is correct
• Network connectivity to GitLab server

Push Failures

If repository push fails:

❌ Failed to migrate repository: ...

Check:

• Target GitLab token has write_repository permission
• Target group exists and you have access
• Repository size isn't exceeding target GitLab limits

Python Environment Issues

If you see externally-managed-environment errors:

error: externally-managed-environment

Use a virtual environment as shown in the installation section.

⚖️ Limitations

• Only migrates repository content (not issues, merge requests, etc.)
• One-way synchronization (changes in target will be overwritten)
• Requires GitLab API access (may not work with strict firewalls)
• Very large repositories may take a long time to migrate

📜 License

This script is provided as-is under the MIT License.