🚀 macos backup and restore strategy

Automated macOS backup and restore strategy for techies

A comprehensive, idempotent backup and restore strategy that configures your mac for modern software development. Supports both Intel and Apple Silicon macs with automatic architecture detection.

The script is idempotent — every step checks whether its work is already done before executing, so you can safely re-run after a partial failure without undoing completed steps. Each skipped step logs the reason, so you can see at a glance what was already in place.

All of the folder structures and the setup/backup operations are governed by the environment variables defined here. Please read the explanation of each variable in the same and edit appropriately.

✨ Features

• 🔍 Auto-detects architecture - supports both Intel x86_64 and Apple Silicon arm64
• 🔄 Idempotent — safe to run multiple times
• 📝 Comprehensive logging — shows all logs with colors for ease of debugging and checking status
• 🛡️ Safe — retains your pre-existing configs instead of overwriting them

📋 What Gets Installed

🛠️ Essential Development Tools

• Homebrew — Package manager
• Modern CLI and GUI tools — See the full list in the Brewfile

🐚 Shell Configuration

• antidote — Static zsh plugin manager
• Starship — Modern cross-shell prompt
• Plugins — autosuggestions, syntax highlighting, selected OMZ libs and plugins managed via antidote
• Aliases — Convenient shortcuts and functions

🛠️ How to adopt/customize the scripts to your own settings

If you want to be able to re-image a new machine with your own settings (and overridden choices), and you do not want to repeat these steps manually, you would want to fork my repo and make appropriate changes into your fork.

Note:

• DO NOT clone this repo into your local machine while forking/initial adoption. The setup script will put it in the correct folder and setup the PATH environment variable as well.
• Make the following changes via the Github web UI/portal itself.
• If you end up with multiple commits on top of the parent repo, you can squash them at the end.

In your forked repo, make the following changes, commit and push via the Github web-UI itself (for the first time before running the script). Once the above steps are done, and committed into your fork, then everytime you need to run the setup, you can run the curl commands that point to your fork:

1. Before running the bootstrap command in GettingStarted.md, you MUST customize the following environment variables in the curl command to match your setup:

   • GH_USERNAME='vraravam' — REQUIRED: Change to your GitHub username (this determines where the script clones the dotfiles repo from)
   • DOTFILES_BRANCH='master' — OPTIONAL: Change to a different branch name if you want to test changes before merging to master (see How to test changes in your fork)
   • FIRST_INSTALL='true' — DO NOT CHANGE: Required for vanilla OS setup
   • CACHE_BUST_HEADERS='true' — DO NOT CHANGE: Ensures latest version is fetched
   • CURL_RETRY_OPTS='true' — DO NOT CHANGE: Enables retry logic for network issues

2. In files/--HOME--/.shellrc: Update the hardcoded username defaults to match your setup:

   • export GH_USERNAME='vraravam' — Change to your GitHub username (must match the value used in the curl command above)
   • export KEYBASE_USERNAME='avijayr' — Change to your Keybase username, or comment out if not using Keybase
   • export DOTFILES_BRANCH='master' — Typically leave as 'master' unless testing a specific branch

3. In scripts/utilities/env_vars.rb: Update the fallback defaults to match your usernames:

   • GH_USERNAME = ENV.fetch('GH_USERNAME', 'vraravam').freeze — Change the fallback 'vraravam' to your GitHub username
   • KEYBASE_USERNAME = _normalize_optional_string(ENV.fetch('KEYBASE_USERNAME', 'avijayr')) — Change the fallback 'avijayr' to your Keybase username, or leave as-is if not using Keybase

4. In this file (README.md) and GettingStarted.md: Find and replace any remaining references to vraravam and avijayr with your usernames for consistency in documentation.

5. Review and update path-related env vars in files/--HOME--/.shellrc to match your preferred folder layout:

   • PROJECTS_BASE_DIR (default: ${HOME}/dev) — root folder where all your git repos will be cloned
   • PERSONAL_BIN_DIR (default: ${HOME}/personal/dev/bin) — folder for personal scripts and executables
   • PERSONAL_CONFIGS_DIR (default: ${HOME}/personal/dev/configs) — folder for private config files and repo catalogs
   • PERSONAL_PROFILES_DIR (default: ${HOME}/personal/${USER}/browser-profiles) — folder for browser profile backups

6. If you are not using Keybase (or want to defer setting it up):

   • In files/--HOME--/.shellrc: Comment out all lines starting with KEYBASE_
   • In scripts/utilities/env_vars.rb: Change the fallback values to empty strings for KEYBASE_USERNAME, KEYBASE_HOME_REPO_NAME, and KEYBASE_PROFILES_REPO_NAME (e.g., change 'avijayr' to '', 'home' to '', 'profiles' to ''). You cannot comment out the entire lines as they are constants that Ruby scripts depend on.
   • The script will skip Keybase-dependent steps silently when these variables are empty or not set.

7. Review all entries in the files/--HOME--/Brewfile, and ensure that there are no unwanted libraries/applications. If you have any doubts (if comparing with my Brewfile), you will need to search the internet for the uses of those libraries/applications and decide whether to retain each one or not.

8. If you changed PROJECTS_BASE_DIR from its default (~/dev), update the corresponding entries in files/--HOME--/custom.gitignore — specifically the /dev/ entry in the "HOME DIRECTORY TOP-LEVEL FOLDERS" section and all /dev/**/ entries in the "DEV WORKSPACE" section. Prefer editing the repo source file directly, then run install-dotfiles.rb to propagate. See Technical Deep Dive § 9 for details on how install-dotfiles.rb resolves conflicts between the repo copy and an existing file on disk.

How to upgrade / catch-up to new changes

1. My recommendation is to always have all your customizations as a single commit on top of the upstream. This allows to easily rebase and adopt new changes in the future.

2. Run the git -C "${DOTFILES_DIR}" fetch --all command.

   • Run the git -C "${DOTFILES_DIR}" upreb command. Most of the times, this should simply rebase your changes on top of the latest upstream master.

   • As an alternative to the above step, if there are too many commits to catch-up to, AND your fork had only 1 commit on top of any of my historical commits, then you can quickly re-apply your changes (remember: single commit) using the following script:

     latest_head="$(git -C "${DOTFILES_DIR}" rev-parse HEAD)"
     git -C "${DOTFILES_DIR}" reset --hard upstream/master
     git -C "${DOTFILES_DIR}" cherry-pick ${latest_head}
     # TODO: manually fix any conflicts

3. Hint: Before pushing your changes to your remote, if you want to ensure (diff) that your old changes are retained (for eg in Brewfile) and no new/unnecessary changes are present, you can run the following 2 commands and review the diffs manually

    git -C "${DOTFILES_DIR}" diff @{u}  # will diff your local HEAD against the remote HEAD of your own fork. Please remember that this diff will show new changes that I have made in my repo, and which are now going-to-be-adopted into yours. It's a good idea to remove entries in Brewfile that you won't need
   
    git -C "${DOTFILES_DIR}" diff upstream/`git br`  # will diff your local HEAD against the remote HEAD of the parent repo. These changes should be exactly the changes that you had done previously (most likely only in GettingStarted.md, files/--HOME--/.shellrc and files/--HOME--/Brewfile)

4. You will have to force-push to your fork's remote after the above step. To accomplish this, I recommend using git -C "${DOTFILES_DIR}" push --all --force-with-lease

5. After the above step, it is always recommended to run the install-dotfiles.rb script once to ensure all (non symlinked) changes are setup on your machine correctly.

6. In case there are any other changes that might be needed after updating, these steps will be detailed in the changelog. In such rare cases, you might have to run the appropriate steps in sequence as detailed out in that section for that version.

7. After updating/catching-up, it is recommended to quit and restart the terminal app so that all "in session memory" aliases, etc are up-to-date and the dotfiles are sourced correctly.

How to test changes in your fork before raising a Pull Request

1. Especially if you are making changes to the fresh-install scripts and want to test it out on a vanilla OS, you can change the github urls to refer to your branch in these files GettingStarted.md and files/--HOME--/.shellrc. For eg, if your PR branch is called zdotdir-fixes, you can search for DOTFILES_BRANCH= in those files, and replace master with zddotdir-fixes. Once your PR is tested and approved, please remember to revert zddotdir-fixes back to master and then merge the PR into the main working branch.

📝 Pre-requisites

If you want to capture data from your current mac, please follow the instructions here

🏗️ Complete setup

The backup strategy is split into 2 stages - both of which are run by the same script. The basic "getting started" provides the instructions for the most common/basic setup. This covers everything that a typical user might need - without the need to backup other parts of the existing laptop.

The "advanced" setup captures application preferences (both system and custom apps) and backs them up into an encrypted remote repository. This requires Keybase for the encrypted private storage. Keybase is entirely optional — if you skip it, everything else (dotfiles, Homebrew packages, zsh config, mise language versions, cron jobs) still works. Simply comment out the KEYBASE_* environment variables in files/--HOME--/.shellrc and the script will skip the Keybase-dependent steps silently.

If you want to automate the repetitive running of these scripts/commands, you can use the system-level cronjobs to set this up, the details of which can be found in the Extras file, by which you can reduce more manual efforts.

🎯 Finally...

The softwares in the files/--HOME--/Brewfile will be run with the bare minimum of formulae initially; the full Brewfile install continues automatically in the background once the base setup completes.

Once the above is done, and if you have setup the keybase-based home repo, browser profiles repo, etc - you can then re-import your exported preferences from the pre-requisites section.

Of course, you will have to manually take snapshots of your machine for backup from time-to-time as an ongoing activity. This can be done using the scripts/capture-prefs.rb script and pushing into the remote repo of your home folder. (More details can be found in the next section.)

As a summary, these files will typically have changes between your setup and mine:

• GettingStarted.md (references to your usernames instead of mine, and typically any other changes that you introduce in the files/--HOME--/.shellrc - look below)
• files/--HOME--/.gitconfig (the IncludeIf line to match your global/base configuration filename)
• files/--HOME--/.shellrc (GH_USERNAME, KEYBASE_USERNAME, path env vars such as PROJECTS_BASE_DIR / PERSONAL_CONFIGS_DIR / PERSONAL_BIN_DIR / PERSONAL_PROFILES_DIR, and other changeable env vars to control which steps to perform vs which to bypass). See ENV_VARS.md for a complete reference of all environment variables.
• files/--HOME--/Brewfile (the list of applications and command-line utilities that you choose to install in your local machine)
• scripts/data/capture-prefs-allowed-list.txt (the preference domains you choose to back up — add or remove entries to match your installed apps)
• scripts/data/capture-prefs-denied-list.txt (domains that must never be exported or imported — edit only to add newly discovered unsafe domains; do not remove existing entries)

For a complete reference of all environment variables (where they're defined, how to access them in shell and Ruby, which ones require customization), see ENV_VARS.md.

For a deeper understanding of how the scripts work internally — the logging system, startup optimisation, .shellrc vs .aliases architecture, cron safety, and more — see the Technical Deep Dive.

🔄 Ongoing tasks to keep your backup up-to-date on a regular basis

The backup strategy is not a one-off activity. It will require you to take snapshots from time-to-time. Similarly, adherance to maintainence of the "catalogs" will need to be strictly upheld for the backup strategy to be effective. Most of the generic maintenance tasks can be automated using cronjobs, the details of which can be found in the Extras file.

🧰 Extras/Details

Some utility scripts have been provided in this repo - which you can use to manage the backup strategy in a better fashion. Details can be found here

🙏 Attributions & Thanks

These folks have contributed to this codebase till date:

• @arunvelsriram
• @shaz-ahammed
• @jotheeswaran-dev